As one of the world's most respected sleight-of-hand artists, Ricky Jay has amassed a peer group of top magicians and cardsharps, and a stable of famous pals, from Bob Dylan to David Mamet. These days, however, he probably spends more time hanging out with book dealers, he says. "I like them. They live by their wits and their knowledge."

They also live off their earnings from insatiable collectors like him. At the New York Antiquarian Book Fair, held recently in the cavernous Park Avenue Armory, he wandered the booths with some new acquisitions in hand, two books about hand shadows, and greeted many dealers by first name.

One seller asked his advice about a book whose blank pages reveal images when you blow on them, a specialty of Mr. Jay's. Another tried to tempt him with a book of illustrations of London street people, published in 1649 and bearing a price tag of $75,000. To one silver-haired gentleman who tipped him off to a set of miniature playing cards, Mr. Jay said, "Thanks, kid."

He incorporates the arcane history he studies into the patter of his one-man shows, several of which have been directed by Mr. Mamet. He has unearthed bizarre characters from the past—automatons, ceiling walkers, Lilliputians, master pickpockets—and used them to anchor his lectures and books, some of which are collectors' items in their own right. These real-life characters no doubt informed the fictional characters he has played in TV shows and movies such as "Deadwood" and "Boogie Nights."

Mr. Jay is preparing to share his expertise, especially in magic, at an event that he's calling the Congress of Wonders. On July 11-14, in the upstate New York town of Rhinebeck, he'll reveal "the truth about dishonesty," or so the schedule promises. Organized by a promoter who books fantasy music camps with rock stars, the symposium also features a handful of speakers picked by Mr. Jay. They include Jules Fisher, a Tony-winning lighting designer who will discuss the craft's often overlooked role in magic, and Michael Weber, Mr. Jay's partner in a business called Deceptive Practices, which advises Hollywood productions on how to create illusions with props.