James Baldwin in Paris in 1975. Photograph by Sophie Bassouls / Sygma / Corbis

Last May, my wife, Valentine, and I and our friend Shahin took an overnight train from Paris to the Cote d’Azur. The bunks on these couchettes_ _are not comfortable, nor is the trip even a bargain against the high-speed option, which gets you there in less than half the time. But it is worth it for the stretch, early in the morning, when day breaks over Marseille and the train shifts from its southern descent and veers east along the corridor linking Toulon, Saint-Raphaël, and Cannes. Suddenly, the sky fills with pastels that turn to gold and shatter on the sea. Standing in the narrow hallway, hands against the glass, your whole body lets you know that you are in the South. From Nice, the terminus, we took a car twenty minutes inland through nondescript suburban sprawl that opens onto the pristine medieval hilltop village of Saint-Paul-de-Vence. As a belated wedding gift, Shahin had booked a duplex in the Colombe d’Or, an unfussy, family-run hotel perched at the base of the ancient ramparts. Since the nineteen-twenties, the hotel has provided a haven to all manner of guests, many of them artists eager to exchange canvases with the forward-thinking patron for room and board. A short list includes Matisse, Braque, Picasso, Léger, Miró, Calder, Cocteau, and Chagall—all of whom have work hanging casually, almost negligently, in the dining room or built into the spectacular landscape around the hotel’s cloistered terrace and swimming pool. Writers have come, too, and we were on a pilgrimage to retrace the steps of one we hold especially dear. For the last seventeen years of his life, until his death in 1987, James Baldwin, lifelong “transatlantic commuter,” emblem of a free black man, was a regular at the bar of the Colombe d’Or, and called home a sprawling ten-acre property, just down the hill, on the Route de la Colle.

When we’d checked in and had some coffee by the pool, we made the ten-minute trek in the withering midday heat to see Baldwin’s “spread,” as he liked to call it. Over the years, a slew of visitors distinguished enough to rival the Colombe d’Or’s dined and slept there, too, from Miles Davis and Josephine Baker to Nina Simone and Ella Fitzgerald. But the story I’ve always preferred is that when Baldwin was awarded the Légion d’Honneur, the year before his death, it was his housekeeper, Valérie, and his former landlady, Jeanne Fauré, that he brought with him to meet the President of France. I thought about that as we approached the property, an extremely wide and shallow expanse of overgrown grass, orange trees, cypresses, wild lavender, and palms that gives sweeping views of the walled town above, the sun-drenched valley below, and, in the distance, the Mediterranean’s rippling sheen. The stone barrier wall had been broken a truck’s width and re-sealed with a chain-link fence that begged to be circumvented. I crouched and pulled out the cinderblock that stabilized it; we slipped in easily. I had seen pictures of the lot before, and there was the sense of déjà vu that accompanies walking into any photographic scene. But what remained of the three-hundred-year-old farmhouse and the gatehouse, where Baldwin’s Swiss lover, Lucien Happersberger, lived, had lapsed into a powerful state of disrepair. Birds flew in and out of the second level, and Shahin hoisted himself through a rectangular opening in the side of the first, reporting back that it was trashed and stripped bare.

I don’t know what we’d been expecting to uncover. A short article that Valentine had dug up indicated that the land had recently been sold to a large developer and was to be subdivided into three smaller plots with brand-new villas, but work had been sporadic and then reached a standstill. We had imagined that we would find a way to intervene and gather support for a French-American cultural center in Baldwin’s name. Several weeks earlier, we had written a letter and begun compiling a list of potential allies to seek out. I had already spoken with some knowledgeable friends in Paris, including a novelist who had recently finished a play on the author’s fraught dynamic with his mentor Richard Wright, and a hip-hop artist whose mother knew Baldwin in Istanbul and visited him in Saint-Paul-de-Vence. Shahin had even made some bluffing preliminary inquiries from his office at the Ministry of the Economy, which did get the developer’s attention and fuelled our hope, but when we saw the reality of the house’s condition, combined with the obvious value of those panoramic views, it complicated the plan. We picked a few sour oranges and slogged back to the Café de la Place, another of Baldwin’s watering holes, across from the Colombe d’Or. In his honor, we ordered glasses of pastis_ _and gazed out at the square, an Instagram-ready fantasy of the South of France if ever there were one, with tables of tanned families nibbling niçoises salads and an immaculate clay pitch populated by timeless old men, barely moving yet somehow never still, nursing rosé and bickering and patting each other’s backs under the guise of playing boules. Above the trees, the sky had achieved a perfect state of blue. At that moment, I understood why Baldwin never resettled in gloomy Paris (let alone Harlem), and for the first time I also thought I understood why some other blacks had come to if not hate him then certainly look at him askew.

Today, among my generation of black writers and readers, James Baldwin is almost universally adored. In this climate, it’s easy to forget the degree to which the man was disparaged and even savaged by both black and white critics, to both his right and his left, while still alive. There was of course Eldridge Cleaver’s venomous homophobic assault in “Soul on Ice,” only a fraction of which is quotable here: “There is in James Baldwin’s work the most grueling, agonizing, total hatred of the blacks, particularly of himself, and the most shameful, fanatical, fawning, sycophantic love of the whites that one can find in the writings of any black American writer of note in our time.” All but irrelevant today, Cleaver, the Black Panther Party’s minister of information, once the darling of white liberals, easily outflanked Baldwin from the left. Calling him a “jive-ass,” he disavowed his initial admiration for “the cover and camouflage of the perfumed smoke screen of his prose.”

Meanwhile, more serious black intellectuals than Cleaver both publicly and privately picked Baldwin apart. “Within the frame of superficial social insights,” a pivotal scholar of black intellectual history, Harold Cruse, wrote, “Baldwin’s literary skills have seduced many people to accept as profound a message that was, from the first, rather thin, confused and impressionistic.” The brilliant and iconoclastic writer Albert Murray echoed this sentiment, accusing Baldwin of failing as an artist for precisely the same reasons that Baldwin once had denounced Richard Wright. Baldwin’s “difficulties and confusions as a serious writer,” Murray argued, stemmed from his “involvement with oversimplified library and laboratory theories and conjectures about the negative effects of racial oppression.” He accused Baldwin of foregoing “the rich, complex, and ambivalent sensibility of the novelist” and acquiring the “thinness” of the polemicist. “What Baldwin writes about is not really life in Harlem. He writes about the economic and social conditions of Harlem, the material plight of Harlem. But far from writing in terms of a U.S. Negro tradition,” which for Murray was a southern, rural one, “he confuses everything with Jewish tradition and writes about life in a black ghetto!” Ralph Ellison was even more dismissive—and acidly homophobic—writing privately to Murray, “Take a look at their works, I don’t think either is successful, but both are interesting examples of what happens when you go elsewhere looking for what you already had at home. Wright goes to France for existentialism when Mose, or any blues, could tell him things that would make that cock-eyed Sartre’s head swim. As for Baldwin, he doesn’t know the difference between getting religion and going homo.” Perhaps most damningly, even Martin Luther King, Jr., in a conversation secretly recorded by the F.B.I., expressed ambivalence about appearing with the author on television, claiming to be “put off by the poetic exaggeration in Baldwin’s approach to race issues.”