On Manhattan’s Upper East Side, home to some of the most expensive real estate on earth, exists the crown jewel of the city’s residential town houses. With its 15-foot-high oak door, huge arched windows, and nine floors, it sits on—or, rather, commands—the block of 71st Street between Fifth and Madison Avenues. Almost ludicrously out of proportion with its four- and five-story neighbors, it seems more like an institution than a house. This is perhaps not surprising—until 1989 it was the Birch Wathen private school. Now it is said to be Manhattan’s largest private residence.

Inside, amid the flurry of menservants attired in sober black suits and pristine white gloves, you feel you have stumbled into someone’s private Xanadu. This is no mere rich person’s home, but a high-walled, eclectic, imperious fantasy that seems to have no boundaries.

The entrance hall is decorated not with paintings but with row upon row of individually framed eyeballs; these, the owner tells people with relish, were imported from England, where they were made for injured soldiers. Next comes a marble foyer, which does have a painting, in the manner of Jean Dubuffet … but the host coyly refuses to tell visitors who painted it. In any case, guests are like pygmies next to the nearby twice-life-size sculpture of a naked African warrior.

Despite its eccentricity the house is curiously impersonal, the statement of someone who wants to be known for the scale of his possessions. Its occupant, financier Jeffrey Epstein, 50, admits to friends that he likes it when people think of him this way. A good-looking man, resembling Ralph Lauren, with thick gray-white hair and a weathered face, he usually dresses in jeans, knit shirts, and loafers. He tells people he bought the house because he knew he “could never live anywhere bigger.” He thinks 51,000 square feet is an appropriately large space for someone like himself, who deals mostly in large concepts—especially large sums of money.

Guests are invited to lunch or dinner at the town house—Epstein usually refers to the former as “tea,” since he likes to eat bite-size morsels and drink copious quantities of Earl Grey. (He does not touch alcohol or tobacco.) Tea is served in the “leather room,” so called because of the cordovan-colored fabric on the walls. The chairs are covered in a leopard print, and on the wall hangs a huge, Oriental fantasy of a woman holding an opium pipe and caressing a snarling lionskin. Under her gaze, plates of finger sandwiches are delivered to Epstein and guests by the menservants in white gloves.

Upstairs, to the right of a spiral staircase, is the “office,” an enormous gallery spanning the width of the house. Strangely, it holds no computer. Computers belong in the “computer room” (a smaller room at the back of the house), Epstein has been known to say. The office features a gilded desk (which Epstein tells people belonged to banker J. P. Morgan), 18th-century black lacquered Portuguese cabinets, and a nine-foot ebony Steinway “D” grand. On the desk, a paperback copy of the Marquis de Sade’s The Misfortunes of Virtue was recently spotted. Covering the floor, Epstein has explained, “is the largest Persian rug you’ll ever see in a private home—so big, it must have come from a mosque.” Amid such splendor, much of which reflects the work of the French decorator Alberto Pinto, who has worked for Jacques Chirac and the royal families of Jordan and Saudi Arabia, there is one particularly startling oddity: a stuffed black poodle, standing atop the grand piano. “No decorator would ever tell you to do that,” Epstein brags to visitors. “But I want people to think what it means to stuff a dog.” People can’t help but feel it’s Epstein’s way of saying that he always has the last word.