People who live in New York might agree that there is very little reason to find yourself between Fourteenth and Forty-Second Streets unless you absolutely have to. Go past Union Square, and you’re liable to bump into everything from confused tourists to people selling knockoff Louis Vuitton and Fendi bags worse than the ones you can purchase on Canal Street in Chinatown. The twenties into the thirties can look like a never-ending row of scaffolding at certain stretches, with C-grade delis and fast food chains hidden beneath, leading you finally to the terrifyingly bright lights of Times Square.

For the better part of the decade in which I’ve lived in New York, this experience is probably what has kept me from the middle of the city. But when I moved from Brooklyn into Manhattan, and started taking daily walks up the various avenues from the West Village to an office on Twenty-Eighth, I began to learn the history of certain buildings I passed along my way: admiring the townhouse at 28 E. Twentieth Street where President Theodore Roosevelt was born; the splendor and history of Gramercy Park; the row of buildings in the Flower District that seems unremarkable, until you realize that this block of Twenty-Eighth between Fifth and Sixth was once known as Tin Pan Alley, and filled the American Songbook. With each block, the twenties became more and more magical, especially on the days when I managed to avoid the crowds scuttling down the sidewalks—those less hectic New York days when I could look up and admire the various gargoyles and the golden dome of the Sohmer Piano Building. The architecture of the twenties distracted me from my daily grind, but it was on an evening trip to the grocery store that the area I once shunned suddenly took on an entirely new meaning. That night I noticed the red plaque on a doorway next to a Starbucks at 14 W. Twenty-Third Street that read, “This was the childhood home of Edith Jones Wharton, one of America’s most important authors.”

Prior to noticing the plaque, I’d had in mind a specific set of things that, to me, were quintessentially New York: Woody Allen 1977 to somewhere in the early nineties; the lyrics in the Television song “Venus” where Tom Verlaine sings that “Broadway looked so medieval”; Anthora coffee cups; and getting to Russ & Daughters early enough to beat the line on a Sunday morning. Edith Wharton’s books, though undeniably part of the American canon, never quite conjured the spirit of the New York that I knew. I loved Wharton’s writing, but it was hard for me, a Jewish guy who grew up in the Midwest in the 1980s, to relate to her world of Gilded-Age manners.

My discovery of Wharton’s birthplace couldn’t have come at a better time, as I had just revisited her novels as research for a book project. I had recently finished, and absolutely loved, The Custom of the Country, and decided to revisit The Age of Innocence after using the CliffsNotes for a high school paper. I found myself particularly interested in Wharton’s description of the Beaufort’s home, one of very few that “possessed a ball-room” that was “shuttered in darkness” every day of the year save for the night of the family’s annual ball. This ostentatious ballroom “was felt to compensate for whatever was regrettable in the Beaufort past.” I became infatuated with Wharton’s description of New York houses, and tried to imagine them nestled in between the busy shopping districts of today. I was paying close attention to the detail which Wharton would describe the Fifth Avenue that she once knew, and the New York that came before her that she describes when talking of Newland Archer walking from Greenwich Village to around Gramercy Park, where his mother always told him “more agreeable and cultivated society had been.”

I finished The Age of Innocence and decided to keep reading as much Wharton as I could, and after awhile, I began to understand what was truly wonderful and great about her work. I realized that when Edith Wharton wrote about New York, she was writing about the New York that she once loved, a New York that she saw vanish as she grew older. It wasn’t some bucolic portrait of a different time and place, nor was it as cynical as some critics of Wharton’s may claim. Rather, the work was honest about Gilded Age New York, a moment in time you could hardly fault her for being born into. As I was reading her story “A Cup of Cold Water,” I even came upon a line that seemed to perfectly sum up the situation that got me so interested in Wharton and her work in the first place: “The buildings which had been so unobtrusively familiar stood out with sudden distinctness: he noticed a hundred details which had escaped his observation.” Just like the character she was writing about, I, too, was noticing the same details for the first time, and I began to read Wharton in a whole different way.

As I was midway through (essentially working backwards) Wharton’s work, I realized that this was the year of her one hundred fiftieth birthday. Wharton and her work suddenly were in vogue—literally, as Vogue ran an Annie Lebovitz spread that featured (among others) Jeffrey Eugenides posing as Henry James, Junot Diaz as diplomat Walter Berry, and Russian model Natalia Vodianova as the WASPy Wharton, all against the backdrop of Wharton’s famed Massachusetts estate, the Mount. While there was plenty of discussion about the interesting casting, the editorial hardly matched the outcry caused by Jonathan Franzen’s New Yorker piece, when Franzen said, quite bluntly, that Wharton “wasn’t pretty.” But while all the talk about Wharton could be looked at as promising, it all failed to truly engage with her work; instead, we got the response to Edith Wharton: the writer of and from privilege. Franzen, Vogue, and a hundred blog posts focused on her wealth and the novels and short stories chronicling the tragic Lily Bart of The House of Mirth and the conniving social climber Undine Spragg of The Custom of The Country. What they left out was that Wharton was in many ways the Gilded Age insider who helped pull back the curtain on the time; her fiction played a part, of course, but it was also Wharton’s keen interest in design that she not only chronicled in The Decoration of Houses, but which also plays a part throughout her fiction.

“Codman shared my dislike of these sumptuary excesses, and thought as I did that interior decoration should be simple and architectural,” Wharton writes in her autobiography, A Backwards Glance, about the beginning of her collaborations with the architect and designer. While the two would go on to write Wharton’s first book together, The Decoration of Houses, Wharton and Ogden Codman first began working together when she and her husband hired Codman to help decorate the “ugly wooden house half an acre of rock and illimitable miles of Atlantic Ocean” they bought in Newport, Rhode Island, known as Land’s End. The “sumptuary excesses” Wharton wrote of included many of the furnishings and decorations popular among the new generation of wealthy Americans during the post–Civil War era. The grand American kingdoms along the East Coast being built and decorated by the nouveau riche seemed ostentatious and ugly to Wharton, monuments to excess and poor taste. Wharton, whose own family lineage stretched back to old New York families like the Jones (supposedly the basis for the saying “keeping up with the Joneses”) and the Rhinelanders, used arriviste types as the basis of many of her characters. Many times these portraits were less than friendly, as in the case of the British banker Julius Beaufort from The Age of Innocence, who Wharton supposedly modeled after the Jewish financier August Belmont, and Undine Spragg’s uncultured and uncouth family from the Midwest. Wharton, as she stated in the introduction to The Decoration of Houses, believed there to be two ways to decorate a home: “by a superficial application of ornament totally independent of structure,” or “by means of those architectural features which are part of the organism of every house, inside as well as out.” Wharton rallied against the “black art” and “dubious eclecticism” that was the house decoration of her day. Thick curtains, dinner tables covered in velvet, bric-a-brac of the era, and “a great deal of gilding” were, in the mind of Wharton, totally out.

While it isn’t easy to agree with some of her views (Wharton biographer Hermione Lee writes that Wharton talked on her deathbed of “her dislike of the Jews”; she also called herself a “rabid imperialist”), Wharton was a Gilded Age iconoclast in terms of how she believed a house should look. Wharton is one of the few great fiction writers whose work takes on a different meaning when you begin to understand her views on design. There is a longing for the simpler times of her youth; not just the people who lived it but also the aesthetic of the time.

That said, it would be hard to argue that Wharton was a minimalist, as she left behind a treasure trove of items worthy of the biggest auction houses in the world. (Lee’s Edith Wharton mentions items like a sapphire and diamond broach, a moleskin coat, a pearl-and-enamel Cartier watch, chinchilla and sable wraps.) Wharton was hardly unhappy with wealth; she just disliked its changing face. And while she by no means enjoyed herself as a child, calling her younger years “safe, guarded, monotonous,” she clearly missed the more innocent times; and that might be the most relatable aspect of her writing. Simpler and more organic is what she is calling for in her books on interior design and gardening—a less is more approach—and a rallying cry against what she saw as the McMansions of her day. While Wharton did spend her last years in France, and once wrote that “The American landscape has no foreground and the American mind no background,” she believed that landscape could be changed. Her embrace of European gardens and affinity for the classical idea of proportion and symmetry helped make her a tastemaker for her time, but these qualities also potentially help readers understand her motivation as a writer. In other words, Wharton’s nostalgia was primarily aesthetic.

Today, when I walk through the neighborhood where young Edith Wharton lived as a child, when I retrace the pathways that gave her a first-hand look at the world that she would eventually commit to paper, I still can’t quite find her exact footsteps. Instead, I stop into nearby Eataly to gaze into the meat case and buy fresh mozzarella. I sometimes take money out of the ATM a few feet away, and every summer find myself drawn to the Shake Shack across the street, inevitably waiting in line for a hot dog and beer to take to watch the US Open on the Madison Square Park big screen. I always take a moment to pause when I’m walking down Twenty-Third to look up and down and try to imagine what it looked like over one hundred fifty years ago. I try and see what Wharton saw; what she loved and what she despised. I try to see the way things were when she was young; the way things were changing as she grew, the way she wanted things to look, and I try to reconcile that with what remains. When I do that, I have an easier time understanding Edith Wharton, her writing, and that little slice of New York I once tried so hard to avoid.

Jason Diamond is a writer who lives in Brooklyn. He’s the founder of Vol. 1 Brooklyn, an editor at Flavorpill, and can be found on Twitter at @ImJasonDiamond.