To assume that one’s voice and cultural contribution don’t count because one was born into “privilege” is as narrow and toxic as to deny one’s voice because one was born into poverty.

This is by no means to discount the idea that, as Shankar Vedantam put it in his elegant and nuanced meditation on unconscious bias, “those who travel with the current will always feel they are good swimmers; those who swim against the current may never realize they are better swimmers than they imagine.” But the simple fact remains that some of humanity’s greatest, most enduring, most influential minds came from one direction of the current and some from the other. Marcus Aurelius was born into a prominent family and became Emperor of Rome. Today, he is celebrated as one of the most influential philosophers of the ages. Incidentally, he was a pupil of Epictetus and was profoundly influenced by his teacher’s ideas. Epictetus was born a slave.

And when we point the privilege finger, where do we draw the line anyway? The concept itself is so abstract and nebulous. Is privilege merely about coming from a family of means and prominence? Or is it also about benefiting from nurturing parenting and a happy childhood, of which wealth and prominence are no guarantee?

Pioneering astronomer Maria Mitchell, who paved the way for women in science, was born into a poor Quaker family of nine children, decades before women had access to formal education or the right to vote. But she had a father who insisted on giving his daughter an education equal to that of his sons, who introduced her to astronomy at an early age, and who created the kind of nurturing intellectual and creative environment in which her talent could flourish. What part is the poverty and what the privilege, and does either warrant that we dismiss Mitchell’s monumental contributions to science and culture?

It is equally narrow to mistake purposeful achievement for privilege. Take Seth Godin. He spent decades toiling and received hundreds of rejections from the establishment before publishing his own books and becoming a best-selling author with one of the most popular sites on the internet. Millions of people heed his advice and devour his perspective. The “privilege” of this hard-earned status is a direct result of the sheer doggedness of writing daily for years, about things that matter and ideas that are helpful and enriching and ennobling for his readers. And yet when people tell Godin, as they often do, “Well that’s easy for you because you have a really popular blog,” misguided accusations of privilege emanate from the complaint — as though, of course, his ideas would be worth any less in the first place if he did indeed have the ease of “privilege.”

Behind every accusation of “privilege” is the self-pitying complaint that the accuser wasn’t granted the same advantage by the fickle bestowers of favor. To that, Joseph Brodsky had the only appropriate answer: “A pointed finger is a victim’s logo.”

To deny a person’s merit or talent or voice on account of the circumstances with which he or she was blessed or cursed — without any say in the matter — is not only to victimize ourselves as individuals but to cheat ourselves, as a culture, of the essential gift of the human spirit.