I’m waiting for the elevator in a medieval-themed hotel in Fort Mitchell, Kentucky, when the elevator doors open to reveal a heated exchange between a bald man in a Hawaiian shirt and a puppet shaped like a toucan. My presence brings an uncomfortable end to their private imbroglio. Both stare at me silently as I enter the elevator, and for five awkward floors I’m brought into direct contact with what George Bernard Shaw described as the “unvarying intensity of facial expression” of puppets, an attribute he believed makes them more compelling actors than humans.

I’m at the Vent Haven ConVENTion where, each July, hundreds of ventriloquists, or “vents,” as they call themselves, gather from all over the world. For four days, they attend lectures on the business, getting advice on AV equipment, scriptwriting, or creating an audience through social networking. They listen to a keynote address by Comedy Central’s ventriloquist-in-residence, Jeff Dunham, who exhorts his notoriously defensive colleagues to “quit complaining that people say we’re weird. We talk to dolls. We are weird, ok. Just own it.” They eat at a Denny’s off the highway and visit the creationist museum down the road. And they don’t go anywhere without the accompaniment of their alter egos.

At the convention, the puppets are a slim but boisterous majority. They crowd in around you. They critique you. They grope you. They chatter continuously. Being around them approximates what it would be like to read people’s minds. It is a most unpleasant experience—a great deal more unsettling, of course, isn’t what they say but that they say anything at all. All over the hotel, in conference rooms, in hallways, at the bar, ventriloquism is practiced in its purest form: not as a stage show, but as an ongoing, unscripted social interaction, a live conversation between humans and their golems. At a drunken party one night, in the hotel’s “hospitality suite,” I witness one dummy operating another dummy, as the human source of both voices sits silently nearby, pretending to compose a text message. The mini bar has lips, which cruelly insult anyone who walks by, the origin of its voice impossible to determine. Almost as soon as I join the party, I am molested by a busty lady puppet, a faded showgirl. She swoons onto my shoulder. “Godaaamn,” she slurs. “Where have you been?” Her vent is a burly, unsmiling dude with a shaved head, a muscle shirt, and camo shorts. He smells strongly of whiskey.

Since ancient times, ventriloquists have been highly prized and despised. While the Biblical writer of the Book of Samuel paints an ugly picture of the Witch of Endor, it is clear that the witch’s talent as a necromancer and prophet—or, to be more precise, as a ventriloquist using the dead prophet Samuel as a dummy—earned her a meeting with the great king. Even so, Hippocrates considered ventriloquism a sickness. The history of the ventriloquism is populated by sinister characters like Louis Brabant, a bankrupt courtier of Francis I, who used his ventriloquial skills to conjure up his victims’ dearly departed. In this way he managed to hoodwink an heiress into marriage, then terrorize an emotionally vulnerable banker out of his fortune.

But even if ventriloquial voices don’t emanate from beyond the grave surely they speak to us from some distant realm. At the convention, I encountered two types of dummies: the Freudian Ids, brightly colored, creaturely puppets who acted out the dormant longings of their masters, and the Superegos, usually wise-cracking boys like the famous Charlie McCarthy dummy, or squinting, censorious characters attired in three-piece suits. In all of these interactions I, at first, played it cool. I’d always address the vent, never the puppet. But it didn’t take long for me to begin talking directly to the dummy—to many dummies, in fact, and candidly and for long stretches—because, as odd as it was to talk to dolls, it felt less odd than ignoring them.

As I later learned, my brain was contending with the “ventriloquist effect,” first noted in a study in the 1890s but named by a research team in the 1960s. The basic insight, that “visual information biases the spatial localization of auditory events,” is a key finding in behavioral and neurological research. When it comes to spatial processing, human vision is, by dint of evolutionary adaptation, generally stronger than the auditory sense. (Which is perhaps why it’s infuriatingly difficult to locate a cellphone that is ringing two feet away from you if it’s concealed under a couch pillow.) Human perception, which functions by fusing simultaneous streams of sensory information, works on the assumption that if auditory and visual stimuli occur in proximity—close in both space and in time—they must be caused by a single source, the one you see. So when we watch lips moving in sync with an unrelated sound, our brain simply denies the confusion, the strange coincidence of these two events, and instead processes them as though they were one very normal speech act. Thus, a ventriloquist can modulate his voice to make it sound near or far, as though it were muffled in a box, or gurgling up from underwater, but he doesn’t actually “throw his voice” in any particular direction; he just tosses it to the audience and they—their eyes, their brain—place it in the lips of the dummy.

This accounts for why post-Enlightenment ventriloquism continues to be bothersome. The problem lies not in the nefarious methods or motivations of the ventriloquist but rather in what it says about us, about our capacity for self-deception. True, the effect may be orchestrated by a sordid stranger in a Hawaiian shirt, but it is we who carry out the illusion. Nor does an understanding of the ventriloquist effect free us from its power. On the contrary, it forces us to witness just how pitifully our brain glosses over problems, how seamlessly it weaves its convenient answers. Our sensory system is as much a puppet as the duck with the googly eyes. As it turns out, there is wisdom in that old routine in which the dummy turns to his master and says, “No, you’re the dummy.”

Ventriloquists tend to think of themselves as living on the cusp of extinction. (Even the word haven in Vent Haven suggests this sense of besiegement). In conversation, they deal with their angst by talking nostalgically of the days of vaudeville and of the era when Edgar Bergen and Charlie McCarthy were household names, back when vents were appreciated. But the best contemporary practitioners know from experience that when a ventriloquist act is executed well it is as mesmerizing and oddly irresistible as ever. An audience for it exists, as it always has; vents continue to be disdained and admired. The rise of Terry Fator is a case in point. Fator was a struggling vent from rural Texas who made his way to America’s Got Talent. When he walked up to the mic, one of the judges rolled his eyes and said, “Oh no, a ventriloquist?” By the end of the performance, however, the judges and the audience were on their feet. Fator won the competition and went on to sign a $100 million contract to headline at The Mirage, one of the largest entertainment deals in Vegas history. As long as there are humans and decoy lips, the ventriloquist effect will live on.

For the vents and dummies who travel to Kentucky for the annual meeting, a visit to Vent Haven is a way to commune with ancestors. On the last day of the convention, the vents visit the museum and archive, which turns out to be a converted suburban home, indistinguishable from the other houses on its quiet block. (Though I am told neighbors have complained of the uncanny dolls gazing from Vent Haven’s windows.) The vents take pictures and pose in front of famous dummies. Inside, rows and rows of puppets are lined up, as though eternally posing for a class picture. There are Victorian-era dummies with mutton chops and top hats, grizzled vaudevillian veterans, modern dummies d’art. Lining the walls are old photos of vents and their dolls. As the vents get older in the pictures, the dummies remain ageless. Now, of course, these vents are dead, and their dummies are here, still wide-eyed. I find myself stopping at Mary Lou, a dummy in tattered rags who washed up ashore as the sole survivor from a 1908 shipwreck that killed her owner.

As I walk through the rows of these frozen faces and the silence of this melancholy little museum, I ask one of my new ventriloquist friends to conjure up a voice, to give speech to one of these figures. I suggest Mary Lou. He scolds me in a hushed tone. “Oh no no no,” he says. “Absolutely not.” This is the one place at the convention where these vents won’t practice their art—if they speak at all in here, they whisper. Perhaps they should go a step farther. Perhaps they should follow the tradition of the Indonesian puppeteers who are, I’m told, buried with their puppets. After four days enduring the prodding and harassment of myriads of clamoring muppets I’ve come to the conclusion that the only thing more unsettling than a talking dummy is a silent one.

Avi Steinberg is the author of Running the Books, a memoir of his adventures as a prison librarian.