Mild cases can seem comical, but severe prosopagnosia afflicts millions in the U.S. Illustration by Jean-FranÇois Martin

It is with our faces that we face the world, from the moment of birth to the moment of death. Our age and our gender are printed on our faces. Our emotions, the open and instinctive emotions that Darwin wrote about, as well as the hidden or repressed ones that Freud wrote about, are displayed on our faces, along with our thoughts and intentions. Though we may admire arms and legs, breasts and buttocks, it is the face, first and last, that is judged “beautiful” in an aesthetic sense, “fine” or “distinguished” in a moral or intellectual sense. And, crucially, it is by our faces that we can be recognized as individuals. Our faces bear the stamp of our experiences and our character; at forty, it is said, a man has the face he deserves.

At two and a half months, babies respond to smiling faces by smiling back. “As the child smiles,” Everett Ellinwood writes, “it usually engages the adult human to interact with him—to smile, to talk, to hold—in other words, to initiate the processes of socialization. . . . The reciprocal understanding mother-child relationship is possible only because of the continual dialogue between faces.” The face, psychoanalysts consider, is the first object to acquire visual meaning and significance. But are faces in a special category as far as the nervous system is concerned?

I have had difficulty recognizing faces for as long as I can remember. I did not think too much about this as a child, but by the time I was a teen-ager, in a new school, it was often a cause of embarrassment. My frequent inability to recognize schoolmates would cause bewilderment, and sometimes offense—it did not occur to them (why should it?) that I had a perceptual problem. I usually recognized close friends without much difficulty, especially my two best friends, Eric Korn and Jonathan Miller. But this was partly because I identified particular features: Eric had heavy eyebrows and thick spectacles, and Jonathan was tall and gangly, with a mop of red hair. Jonathan was a keen observer of postures, gestures, and facial expressions, and seemingly never forgot a face. A decade later, when we were looking at old school photographs, he still recognized literally hundreds of our schoolmates, while I could not identify a single one.

It was not just faces. When I went for a walk or a bicycle ride, I would have to follow exactly the same route, knowing that if I deviated even slightly I would be instantly and hopelessly lost. I wanted to be adventurous, to go to exotic places—but I could do this only if I bicycled with a friend.

At the age of seventy-seven, despite a lifetime of trying to compensate, I have no less trouble with faces and places. I am particularly thrown if I see people out of context, even if I have been with them five minutes before. This happened one morning just after an appointment with my psychiatrist. (I had been seeing him twice weekly for several years at this point.) A few minutes after I left his office, I encountered a soberly dressed man who greeted me in the lobby of the building. I was puzzled as to why this stranger seemed to know me, until the doorman greeted him by name—it was, of course, my analyst. (This failure to recognize him came up as a topic in our next session; I think that he did not entirely believe me when I maintained that it had a neurological basis rather than a psychiatric one.)

A few months later, my nephew Jonathan Sacks came for a visit. We went for a walk—I lived in Mount Vernon, New York, at the time—and it started raining. “We had better get back,” Jonathan said, but I couldn’t find my house or my street. After two hours of walking around, during which we both got thoroughly soaked, I heard a shout. It was my landlord; he said that he had seen me pass the house three or four times, apparently failing to recognize it.

In those years, I had to take the Boston Post Road to get from Mount Vernon to the hospital where I worked, on Allerton Avenue in the Bronx. Though I took the same route twice a day for eight years, the road never became familiar to me, I never recognized the buildings on either side, and I often took the wrong turn, realizing it only when I came to one of two landmarks that were unmistakable, even for me: at one end, Allerton Avenue, which had a large sign, or the Bronx River Parkway, which loomed over the Boston Road.

I had been working with my assistant, Kate, for about six years when we arranged to rendezvous in a midtown office for a meeting with my publisher. I arrived and announced myself to the receptionist, but failed to note that Kate had already arrived and was sitting in the waiting area. That is, I saw a young woman there but did not realize that it was her. After about five minutes, smiling, she said, “Hello, Oliver. I was wondering how long it would take you to recognize me.”

Parties, even my own birthday parties, are a challenge. (More than once, Kate has asked my guests to wear name tags.) I have been accused of “absent-mindedness,” and no doubt this is true. But I think that a significant part of what is variously called my “shyness,” my “reclusiveness,” my “social ineptitude,” my “eccentricity,” even my “Asperger’s syndrome,” is a consequence and a misinterpretation of my difficulty recognizing faces.

My problem with recognizing faces extends not only to my nearest and dearest but also to myself. Thus, on several occasions I have apologized for almost bumping into a large bearded man, only to realize that the large bearded man was myself in a mirror. The opposite situation once occurred at a restaurant. Sitting at a sidewalk table, I turned toward the restaurant window and began grooming my beard, as I often do. I then realized that what I had taken to be my reflection was not grooming himself but looking at me oddly.

In 1988, I met Franco Magnani, the “memory artist,” and during the next couple of years I spent weeks with him, talking about his paintings, his life, and even travelling to Italy with him to visit the village where he grew up. When I finally submitted an article about him to The New Yorker, Robert Gottlieb, who was then the magazine’s editor-in-chief, read the piece and said, “Very nice, fascinating—but what does he look like? Can you add some description?” I parried this awkward (and, to me, unanswerable) question by saying, “Who cares what he looks like? The piece is about his work.”

“Our readers will want to know,” Bob said. “They need to picture him.”

“I will have to ask Kate,” I said. Bob gave me a peculiar look.

I assumed that I was just very bad at recognizing faces, as my friend Jonathan was very good at it—that this was within the limits of normal variation, and that he and I just stood at opposite ends of a spectrum. It was only when I went to Australia to visit my older brother Marcus, whom I had scarcely seen in thirty-five years, and discovered that he, too, had exactly the same difficulties recognizing faces and places, that it dawned on me that this was something beyond normal variation, that we both had a specific trait, a so-called prosopagnosia, probably with a distinctive genetic basis.