Perhaps it's the influence of his stories, with their subtle narrators and exquisite understatement, but to me the smiles on Isaac Babel's face in the black-and-white photographs in the Odessa State Literary Museum all seem ironic. It's hard, too, knowing a little about his life and how it ended, not to suspect that one of the ironies Babel might have been contemplating was the transience of success, even the violence that might one day supersede the accolades the Stalinist state had heaped upon him. Looking at the wiry spectacles pinned to the wall, I think of the pair that must have perched on the pale, fragile dome of Babel's head when the secret police took him to the Lubyanka.

Babel is Odessa's best-known literary son. But this wonderful museum, housed in a pale-blue tsarist-era palace, isn't devoted only to Odessa's own, or to its magical and dreadful history, though it encompasses both. Because of its location—on the Black Sea, at what was the Russian and then the Soviet empire's sunny southern limit—many of those empires' greatest authors were exiled to Odessa, fled through its docks, or came here for their health or a debauch. Embracing the transients and flâneurs, this is, in effect, a museum of Russian literature. And, being Russian, it becomes a museum of censorship and repression as well as art: of genius and bravery, blood and lies.

There are lots of museums devoted to famous writers, but fewer dedicated purely to literature. This one was conceived and founded by Nikita Brygin, a bibliophile and ex- KGB officer. He left the KGB in murky circumstances, but remained sufficiently well-connected to secure a handsome venue near the sea for his eccentric scheme—the ceilings are cracking, but the chandeliers and reliefs conjure the mood of the aristocratic balls for which the palace was built. He sent a team of young women across the Soviet Union to secure writerly artefacts for the collection, which is arranged in a suite of bright first-floor rooms reached by a grand double staircase. Opened in 1984, the museum survived Odessa's transition from the defunct Soviet Union to independent Ukraine. Today, it is overseen by elderly attendants whose sternness yields to solicitous enthusiasm when one of their infrequent visitors approaches. The place runs on love.

For me, this is a memento of the years I spent travelling across the former Soviet Union as a foreign correspondent—the most exhilarating, frustrating, sad and privileged years of my life. I loved both Odessa and the museum when I first came in 2006, but the stories I wrote on that trip were bleak ones, about smuggling through the port and sex trafficking through the ferry terminal. A woman from a charity that helps victims of trafficking told me how to spot them among the passengers disembarking the ships from Istanbul: hungry, hangdog expressions; no luggage; clothes ill-suited to the season. The sex trade is the dark side of the licence and loucheness for which Odessa has always been renowned.

Pushkin is partly to blame for the city's raunchy reputation. In the margins of the manuscripts of "Eugene Onegin", which he started writing in Odessa, he doodled portraits of some of the women he slept with here. Facsimiles, complete with lavish crossings-out, are on view in the museum. You look at the sketches and think of the young poet, bored by his own genius. Legend has it that, exiled from St Petersburg by the tsar, Pushkin began an affair with Countess Vorontsova, the wife of Odessa's governor. Another of his local flames, Karolina Sobanska, was also the some-time lover of Adam Mickiewicz (a cherished Polish poet commemorated in the museum), and a long-term spy for the tsarist secret police. Count Vorontsov, the peeved governor, dispatched Pushkin to write a report about a locust infestation, before running him permanently out of town.

You can still sense Odessa's erotic tension and potential in its balmy passeggiata and suggestively latticed balconies. In my novel "Snowdrops", mostly set in Moscow, I send the narrator, Nick Platt, down to Odessa as the dénouement approaches. Nick is a chronic self-deluder and a man of fatally easy virtue. Here, he says, "you can somehow make things seem better than they truly are. You can make things be what you want them to be." Odessa, a breeding ground for fabulists, seemed the right place for him.

Yet, like many ports, Odessa stands for freedom as well as sleaze. Revolutionaries and their ideas have been smuggled in and out along with contraband goods. Each of the museum's rooms represents a period in the city's intellectual history, evoking a particular era through furniture and design, and often concentrating on a characteristic genre. In the room depicting the 1850s and 1860s, there is a run of issues of the Bell, the journal published by Alexander Herzen during his London exile, which was sneaked into Russia through Odessa's docks.

Freedom, in fact, was the point of Odessa. Founded by Catherine the Great in 1794 as a free port, it soon became probably the most cosmopolitan city in the world, drawing in Greeks, Poles, Germans, Italians, Tatars, Turks, Armenians, runaway serfs and Jews fleeing the anti-Semitic restrictions in force across the rest of the Russian empire. Even today, with its Mediterranean architecture and post-Soviet ricketiness, Odessa seems to belong to many other places, and at the same time only to itself. Poetry and polemic in a variety of languages, forgotten heroes of foreign struggles and minority masterpieces are remembered in the museum's portraits and manuscripts.

At the opposite end of the empire from St Petersburg, warm where the former capital is chilly, Odessa is its opposite in other ways, too. Whereas St Petersburg is a "premeditated city", as Dostoyevsky put it, built by Peter the Great on the blood and bones of slaves, Odessa grew up spontaneously on commerce.

The blood came later. At the turn of the 20th century the region was overtaken by pogroms and then the Russian revolution. Anyone inclined to believe that revolutions are chiefly a pretext for bloodletting and score-settling will find that view amply corroborated in "Cursed Days", Ivan Bunin's diary of life in the city during the Russian civil war. In the museum you can sit at the dressing table rescued from Bunin's rooms and see yourself in his mirror, wondering, perhaps, how you might have coped, what compromises you might have made, in the chaos of rumour and mob thuggery that "Cursed Days" describes. The mirror is half patched with letters and family photographs and the remaining, exposed glass is mottled. The blotches aren't blood, but they make you think of blood.

Revolutionary Odessa, Bunin writes, was "a dead, burned-out city", a nightmare place stalked by tramps and drunks armed with revolvers and cutlasses. His is a more reliable account of revolutionary upheaval than Sergei Eisenstein's brilliant but wildly mendacious "Battleship Potemkin", a propaganda version of a naval mutiny in 1905. The film has a famous scene of a pram careening down the steps from Odessa's seafront boulevard to the water. A cartoon sketch of Eisenstein and a poster advertising the film's release appear in a room that deals with the cinema and drama of the 1920s. These days, visitors who stroll along from the museum to the "Potemkin Steps" can have themselves photographed with an impressive array of exotic animals, for a small consideration.

Exhausted by fear and depravity, Bunin caught the last boat out of Odessa with the retreating French forces in 1920. When, a few years later, Isaac Babel published his "Odessa Tales", one of the cycles of short stories that made his reputation, the milieu they portrayed had already been swallowed by strife.

They all came to Odessa—all the classic Russian writers I have learned to revere. Gogol came south for his health. There is a sort of shrine to him in the museum, in a shadowy alcove formed by a dark, cruciform bookshelf and an ornate, frieze-covered wall. Chekhov passed through, on his way to and from the penal colony of Sakhalin Island, and so did Tolstoy: the young writer in one of the photographs in the collection looks a lifetime away from the iconically bearded sage. At least, almost all of them came. The notable exception is Dostoyevsky, who never visited Odessa—which might explain a lot.

The Soviet-era masters, too. Anna Akhmatova was born here. Mayakovsky fell in love in Odessa, as one of his poems records: "It happened. It happened in Odessa…" His mad visage stares from a photo in the museum, appearing as the 19th-century style of the earlier galleries gives way to constructivist patterns, slanting collages and striking red frames that signify the great disjuncture. In the lovely, lush statue garden that runs along one of the walls, abode of dead writers and live cats, there is a monument to Vladimir Vysotsky, a hellraising Russian chanteur and one of the Soviet Union's finest poets. Vysotsky was also an actor; he made films in Odessa's studios, and sang about a longing to come here. If I could go back in time to attend any gig, I would choose one of Vysotsky's semi-clandestine performances in the 1970s.

Still, for me, Babel, born in Odessa in 1894, is the presiding genius. It's partly the Jewish connection, which adds another layer of darkness and light to the Odessa story. My first book was about my Jewish forebears' emigration from what was then Galicia, and is now western Ukraine, to the East End of London at the end of the 19th century. Anyone researching Jewish life in that region at that time encounters the legend of old Odessa. Once among the great Jewish cities of Europe, it was a sort of seedier, commercial version of fin-de-siècle Vienna: an incubator of self-confident, enlightened Jews, where opera and cantorial music mingled, and Jewish families sent violinists and chancers out into the world in roughly equal measure. In early-20th-century London, I discovered, there was a fearsome Jewish gang called the Odessans.

Then I read Babel's "Odessa Tales", colourful early editions of which are mounted alongside the photographs of him in the museum. The tales feature women who wear scented lingerie, flap black fans and stake gold coins; and they star Benya Krik—Benya the King—the leader of Odessa's Jewish gangsters. Benya burns down the police station on the night of his sister's wedding. "Everyone makes mistakes, even God," he tells a mother whose son has just been shot by one of his henchmen. When you've had enough of Odessa's palaces, you can take a tour of the Moldavanka, Benya's neighbourhood, and peer into its courtyards, now populated by defeated dogs, noisy children, men working on ancient cars, and young women stepping out of hovels but looking like $2m.

So I was primed to admire Babel by my background, but also, I think, by the pattern of my career. Babel, for me, is the supreme journalist-turned-fiction-writer. Serving as a correspondent during the Polish-Soviet war of 1920, he rode into battle with the Cossack cavalry, improbable company for a Jewish intellectual. Afterwards he transmuted those experiences into "Red Cavalry", the collection of quietly appalled war stories that are his finest achievement. He published very little after these, riskily referring to himself as a "master of the genre of silence". But he stayed in the Soviet Union when he might have defected: a classic, passive Russian heroism.

Babel was shot in 1940, around the same time as several lesser writers mentioned in the museum, some of whom might have gone on to be as distinguished as he was. Not long afterwards Jewish Odessa was destroyed in violence that, even by the standards of the second world war, is difficult to read about. The war and its cultural impact are discussed in a set of rooms on the museum's ground floor. There are blown-up photos of destroyed streets and liberating Soviet soldiers; also, less conspicuously, preserved orders from the Romanian occupiers for Jews to assemble and ghettoise. Like many Jews, when I look at these blandly genocidal documents I have a dim, irrational sense that this could have been me and mine.

In a way there is surprisingly little about Babel in the museum. But perhaps this is as it should be. The exhibition has remained largely unchanged since the 1980s. By then Babel had been rehabilitated, but he wasn't quite a whole person again; nor was the whole truth about Soviet crimes accepted and circulated. The displays are coy about the fates of some of the writers they mention. The photos of Babel do not include the haggard mug shots taken after his arrest.

And so, in a mild form, the museum is itself an example of the censorship that both thwarted and stimulated its subjects. Nikita Brygin, its founder, knew that about his project from first-hand experience. Party bosses, reputedly nervous about the enlightened atmosphere Brygin was creating, forced him out of the museum before it opened. He died in 1985 without ever seeing the finished article. Even without his final guidance, the exhibition is in some ways daring for its time, both in design and content. But it remains an artefact of its era. Perhaps that makes it even more valuable. Like the city, the history and the culture it celebrates, it is a glorious achievement with an underside.

The years I spent in Russia broadened my literary and moral horizons. And they made me think about the relationship between the two: between art and the suffering that sometimes seems to gestate it. Odessa has been a theatre of both art and suffering and the museum is a testament to that synergy. I love the city, the museum and Russian literature; but, in the end, they aren't and couldn't be worth it. They aren't worth all the blood that has gone into them.

The Odessa State Literary Museum is open Tuesday to Sunday, 10am-6pm; museum-literature.odessa.ua