If all was as it should be in the world, Rory Duffy, 33, a lanky fifth generation master tailor from County Monaghan, Ireland, would be celebrated as the finest tailor in New York City, draping the shoulders of oligarchs and dynamos in his bespoke suits. He’d be held up as proof that the proud tradition of Savile Row tailoring is alive and well. But since all is not as it should be, Duffy is working from the back of a small soon-to-shutter menswear shop called Against Nature on Manhattan’s Lower East Side. His name is known only to a few in the trade and a small subculture of menswear pedants. But why?

All sharp angles, pale skin, and slender bones, Duffy cuts a striking figure. Greeting me in his shop, he looks like someone in an Egon Schiele painting in his high-waisted breeches, suspenders, and burgundy linen waistcoat beneath an 1860s frock coat.

The theatrics, as clothiers like Turnbull & Asser, Berluti, and even Duncan Quinn know, involve not simply fabric and the human touch, but the brandishing of wealth and privilege. The tailor’s fitting platform is as much as proscenium as it is functional and the self, reflected in triplicate, is both actor and audience. One often goes in for bespoke not so much for the garment but for the self-regard it affords. So what one sees in the mirror is important. Often this mise-en-scene involves wainscotting, deep leather chairs, and whiskey, signifiers of an Ur-Alpha Male. But all those things cost money, too. And the rub of truly bespoke suiting is that it gobbles time and money, leaving little in the way of investment in scenery for the independent tailor.

For years, for instance, Duffy worked out of his Williamsburg apartment, which was nice but, you know, a Williamsburg apartment. Now his atelier is larger but still ramshackle. In the backroom of the store, past a doorway guarded by a pair of taxidermied albino peacocks, it is very much a workshop. One rack is laden with finished garments. Another hangs heavy with patterns. A shop boy named Pablo responds to emails in the corner. Trimmings and cloth, drawers full of buttons, linings, canvases, shoulder pads in boxes—it can’t help but be chaos.

Duffy charges $5,000 to $6,000 for each of his handcrafted bespoke suits, which puts him around the starting price for an off-the-rack Kiton suit. However, while those are instantaneous, Duffy’s take around 80 hours to make and are completely handsewn. Duffy is busy but at bandwidth. “I have a lot of clients,” he says, “but not enough to sustain a business.” Almost alone among modern tailors—though he isn’t, properly, speaking modern in anyway but the chronological—Duffy eschews standardized block patterns. He handcuts his patterns based upon the body of each of his clients and this, he says, takes hidden skill.

“Measurements aren’t taken straight, they’re taken from a slight angle,” he says, “You have to know anatomy to do it properly.” Using a large pair of steel shears, he liberates from sheets of brown craft paper the blazers, trousers, waistcoats, two-piece front patterns, one-piece fronts, and the so-called corpulent cuts of his client’s future clothing. These hang from a rack, the human form deconstructed.

When a then-25-year-old Duffy arrived in the United States in 2011, fresh from winning the Golden Shears, the Oscars of Savile Row, he was encouraged by what he saw as a softness in the market. “Despite how trendy it was, I was the only master tailor in the city,” he recalls. He found that what we commonly call bespoke is, in fact, made-to-measure.

“Bespoke is bench-made by one tailor,” he says, “and made-to-measure is factory made. Nobody really wants to say they’re doing made-to-measure, but they are. I was the only one who could do true bespoke.”

Even companies that do make made-to-measure often outsource their tailoring to Asia. They aren’t alone. According to Duffy, even Savile Row tailors like Gieves & Hawkes outsource their work. “It’s not that Asian tailors don’t make quality garments,” he says, “but what they lack is the knowledge. Because they don’t have a tradition of bespoke Western tailoring, they often reverse engineer Savile Row garments, without knowing the techniques.”

Duffy, on the other hand, was almost born knowing those techniques. Reared in a tailoring family—his great-great-great-grandfather, Patrick McCabe, entered the trade in the early 19th century—Duffy began his apprenticeship at age 19 with master tailor Joseph Martin on the west coast of Ireland, Louis Copeland in Dublin, and David Young in Galway. He also studied at the London School of Fashion and interned at hallowed Savile Row clothiers like Henry Poole. But despite his expertise, it’s been a struggle. “Bespoke just isn’t scalable,” he says. “And since Brooks Brothers introduced ready-to-wear, there’s not even the heritage left anymore.”

What he means is that the American market might be soft on bespoke clothing for a reason. We’re a nation reared on ready-to-wear, which, as Duffy notes, Brooks Brothers introduced in 1850. So for more than 150 years, we’ve been sold a story that suits of the highest quality can—and should—be readily available and rather affordable. The only problem is that they aren’t really. Bespoke suits suffer from the Big Mac dilemma. Sure, the 99-cent version is tasty, but the costs are hidden, the similarities with a grass-fed burger are cosmetic, and the underlying quality is missing.

So Duffy, who barely breaks even by the time he’s added up the cost of his material, his time, and his overhead, is left with an insoluble dilemma: Either betray the tradition of bespoke or live hand-to-mouth, like the tailor’s son he is. Naturally, Duffy chose the latter.

It is perhaps no coincidence that he works out of a store, Against Nature, named after an obscure Symbolist novel by J.K. Huysman about a Viennese aesthete who, out of touch with his times, moves to his family’s countryside manor to keep the world at bay. In some ways, Duffy is an outsider for outsiders. “The men who come to me,” he said in an Irish brogue that makes everything sound poetic, “can’t wear a regular suit.” Duffy cuts for the shortnecks, the unevenly shouldered, and the portly. And, using the magic passed down from the hands of masters, he makes them all appear ennobled. For the corpulent, introducing extra darts to jimmy a slimmer silhouette. For the slope-shouldered, increasing the shoulder angle. For the short-necked, cutting a higher shoulder.

And yet, perhaps there are simply not enough misfits looking to fit in. In a few months, Duffy says, Against Nature will close. He will return to Ireland to teach his methods in the workshop of his old master, Joseph Martin. He’ll alternate months in New York but, he says, he too will send much of his production abroad—not to Asia but back to Ireland, to County Monaghan, where he’ll rely on a workshop of tailors trained in the old ways. As for his own whereabouts, he’ll live in an old house in the Irish countryside where his grandparents once lived—a house with 13-inches of thick flagstone walls and 45 acres of farmland to keep the world at bay.