It is something of an irony that the most well-known photograph of Virginia Woolf, captured by George Beresford in 1902, was taken well before she had composed her most famous works; in fact, it was taken before she had even become a Woolf. That image of the 21-year-old Virginia Stephen—ethereal, elegant, beautiful, haunted—has always struck me as intensely, almost painfully private. No matter that it was almost certainly posed; Virginia feels impossibly distant, possessed of an infinite inwardness, and what she offers the viewer pales in comparison to what she withholds. Her eyes, slightly downcast, seem attuned to the ineffable rhythms of an invisible world. Whenever I reread To the Lighthouse, something I try to do once a year, this is the image that rises to the surface of my mind, floating freely above the text. A gaze of both great softness and great intensity, a far-away face from whose slightly parted lips leapt the immortal lines: “All the being and the doing, expansive, glittering, vocal, evaporated; and one shrunk, with a sense of solemnity, to being oneself, a wedge-shaped core of darkness, something invisible to others.”

But when we let our gaze linger on the photographed faces of our literary heroes, how much of what we feel can be attributed to readerly intuition—and how much to fantasy? Affixed to the backs of book jackets, touched and smudged and gazed at longingly (or enviously, as it were), the image of the author has become an underappreciated accessory to the ritual of modern reading. These photographs, with the drama inherent to a human face, offer up a kind of ready-made narrative potential that we seize on, often before we’ve read a single page. We take in the glowing eyes, the spectral hands, the starched collar, the frozen smile, and in that encounter a feeling—a story—begins to unspool in our minds. That reflexive flash becomes a kind of echo that stays with us when the actual reading begins, a conversation between text and image, image and text. If ours is a culture of visual primacy, is it any wonder the photos of our authors come to inhabit the texts they write?

Take Franza Kafka, whose final photograph, taken months before he died of a tuberculosis that had stalked him for years, reveals a lean, ascetic, almost unbearable intensity in which a visage has transformed into a kind of physical aphorism: Kafka’s face as a Kafkaesque story in and of itself. In that gaze I find the burrower, the hunger artist, and the insect, K. wandering outside the castle, the surreally thwarted country doctor; however, I also find a confrontation, as if Kafka is daring me to look upon him; as if, even after reading every one of his gorgeous obscurities, it is the mystery and terror of a face that lingers as his final literary challenge.

Or Sylvia Plath, who stares out at us as though seeing through a glittering darkness, an enigmatic smile playing about her lips. Plath’s confessional poetry, a ferociously varied tapestry of blight and sex and death, often revealed a consciousness at odds with itself as her productive periods ran aground on the agonies of a psychological disintegration. Plath’s face, half-shadowed, seems to speak to this divided consciousness, her features gathering in a perfect ambiguity: the poetess receding even while fully present. The beauty and mystery of her work is wreathed around this face and I can’t help but see it when I hear her name, an apparition that emerges from the ecstasy of her art.

But often, too, these faces are not what we expect them to be, as the physical particulars of a writer do not match with what we know, with what we’ve heard, with what we’ve read. I remember seeing a photograph of William Burroughs after having read Naked Lunch and Junky as a teenager. I don’t know what I was expecting—the louche elegance of a young Keroauc, perhaps, or the disheveled romance of Bukowski—but when I saw the tall, thin, nattily dressed, physically unremarkable man gazing back at me, I felt there had to be some sort of mistake. This was the man whose cut-up technique had dazzled and angered and opened me? This?

Similarly, I remember Jean-Paul Sartre’s legendary ugliness taking my breath away. The traumatic haircut, the right eye not wandering so much as fleeing his face, the sense of a gargoyle-like grotesquerie, Sartre’s physical unsightliness is forever at odds with the intellectual beauty of his celebrated work. Sartre owned his ugliness though; when Camus found him laying it on thick with a girl at a Parisian bar he couldn’t understand it. “Why are you going to so much trouble?” Camus asked. To which Sartre replied: “Have you had a proper look at this mug?”

Camus, of course, occupied the other side of the spectrum, the intellectual Adonis par excellence. With his Bogart-style trenchcoat and a Gauloise cigarette dangling from his lip, Camus was the picture of literary elegance, envied and lusted after, the James Dean of existentialism—though this didn’t always work to his advantage. As the Australian cultural critic Clive James said, “The widespread notion that Camus’s mind was not really very complex at all is the penalty he paid for being blessed with good looks, the Nobel Prize, too many women, and too much fame.” One of his most famous lines from The Rebel, “Tyrants conduct monologues above a million solitudes,” is poetically rich and effortlessly authentic—as it ought to be; Camus’s exile from his homeland of Algeria constituted a permanent form of solitude that always haunted his good looks, a melancholy that lurked behind his eyes. Of course, Camus being Camus, this only made him more handsome. “No modern writer I can think of, besides Camus, has aroused love,” said Susan Sontag, though one suspects he aroused a great deal more besides.

Emily Dickinson is an interesting case in that we’ve had to recalibrate our relationship with her image after a recent discovery. For decades, there was thought to be only one photograph of her: the much-discussed, much-studied image of the poet as a teenager in 1847. Austere and ethereal, her bone-white face set like a lamp against a gray backdrop, this striking photograph was the only visual record of her existence, and into that singular presence we were able to cast the dreams and presentiments that emerged from her staggering body of work. It made everything easier, in a way; a single photograph; a single face; a familiarity to tame the terse power of her paeans to death.

Then, in 2013, a new daguerreotype was found, soon authenticated as Dickinson, a grown woman now in her late twenties. The difference is striking: in the new image she shows a previously unseen strength and serenity, her arm thrown protectively around the back of a recently widowed friend. I resisted the new image when I first saw it, for reasons that weren’t clear to me then; looking back, I think I was attached to a particular representation, a particular visual understanding of her that nourished the written work itself, or at least my relationship to it. And even though I realize how unfair, even infantile, such a hang up might be, I couldn’t seem to help myself. Our investment in an author’s photograph can make us perceptually rigid, unresponsive to change. I learned then that there is nothing passive about our literary iconographies. We are always active, and often intractable, participants in such enshrining.

But really, how could we ever hope to resist? If, as Cicero had it, “the face is a picture of the mind,” we are its natural born interpreters, embellishers, idolaters. From the radiant humor and flaring intellectual force in the photographs of the ever-penetrating James Baldwin, to Eugene O’Neill’s studied brand of dark-eyed melancholy (the son of an actor, after all), part of the thrill of the author’s image is our willingness to submit to a visual seduction. But in crafting our own relation to these cherished photos, in allowing the image to enhance and intersect with the text, we, too, seduce our images in turn. Perhaps that is as good a definition of reading in an image-rich culture as any: a mutual manipulation.

Virginia Woolf last sat for a professional photographer in 1939; she would be dead in less than two years, drowned in the River Ouse. These photos, so important to me and my reading of her, have become something of a parallel text, one requiring a new kind of literacy. I often wonder what she was thinking here, perched on the edge of her genius and her madness. To better understand, I read, of course; but I also look.