The members’ bar of the Southbank Centre in London is not the first place you’d look for a plot to change the face of British culture. But that’s where you’ll find Barbara Roche, the former Labour immigration minister, and Sophie Henderson, an immigration lawyer taking a break from her usual job. This is the temporary office of the Migration Museum Project, which is backed by several MPs and counts Roche as chair of trustees and Henderson as director. Both were inspired by Robert Winder’s book Bloody Foreigners, which told the centuries-long tale of immigration to Britain, and suggested in a footnote that a museum might reduce our widespread ignorance of the subject.

“It’s everybody’s story,” says Roche. “The only difference between people is how long ago they came.” That’s something that seems to have been forgotten in all the current talk of a “crisis” in Calais and a “swarm” of migrants. The story of immigration to the UK can be stretched to take in our entire history: right back through the waves of Protestant, Jewish and other refugees, through the Norman and Roman invasions, to the first humans to cross over from mainland Europe. Britain has many museums that tell part of this story, such as the new Huguenot Museum in Rochester – but nothing to rival, say, France’s national museum of immigration, which opened in Paris in 2007.

“The museum could be rather earnest, and that would be its death knell,” says Roche, whose own background is in Jewish east London. “It’s got to be properly challenging but fun.”

“We’ll be looking for a super-storyteller,” Henderson adds. “We don’t want to be preachy. You have to have something positive and engaging and we are very mindful about footfall.” Henderson points out that there is no national museum of British history, with the National Portrait Gallery the closest thing; she says polls show that around half of British people do not hold strongly pro- or anti-immigration views and are happy to engage in discussion. The new museum, which is envisaged to cover emigration as well as immigration, would have a cafe, bookshop and auditorium, as well as a genealogy area with resources for those interested in tracing their own roots.

Personal stories of family migration by visitors to the Adopting Britain exhibition at the Southbank Centre in London. Photograph: David Levene for the Guardian

A series of temporary shows has tested the market, and a selection of photographs and “keepsakes” – toys, gifts and other personal items – are included in the Southbank Centre’s current show, Adopting Britain, with its theme of “70 years of migration”. Some of these are hugely appealing: a reversible doll with one black head and one white was lent by the mathematician Sir Michael Atiyah, whose mother made hundreds of them when the family lived in Sudan. A separate touring exhibition, which opened in Edinburgh last month and travels to Aberdeen next week, looks at the history of German immigration to the UK, from doctors to musicians, clockmakers and sailors.

But exactly which objects, or kinds of objects, a new permanent exhibition would feature is not yet clear. “The exhibits are the most difficult thing – what should we show?” says Aude Pessey-Lux, director of France’s Musée de l’Histoire d’Immigration, when asked what those behind the London project could learn from Paris. Her museum has three main collections: a social history archive; donated mementoes, jewellery and crafts; and a growing amount of contemporary art. Knitted wigs by Beninese artist Meschac Gaba feature in a current exhibition.

Pessey-Lux says the job of working with historians, including those on the museum’s own staff, has been hard. “In a museum,” she says, “you have to find things that explain what you want to say. A museum is not a book.” Her team has also been forced to confront the controversial history of their own art deco building, the Palais de la Porte Dorée, which was built to celebrate French colonialism in 1931, and retains wall paintings and friezes from that time, as well as a tropical aquarium. Opposite the palace stands a monument to Jean-Baptiste Marchand, who helped spread French rule in Africa.

‘In a museum you have to find things that explain what you want to say. A museum is not a book’ … France’s national museum of immigration, which opened in Paris in 2007. Photograph: Stephane De Sakutin/AFP/Getty

For other immigration museums, the heritage and atmosphere of their physical surroundings has been the key to their success. At Ellis Island Museum in New York, visitors are taken through a recreation of the experience of the 12m new immigrants who entered the US via this gateway between 1892 and 1924. As the poem on the pedestal of the Statue of Liberty puts it: “Give me your tired, your poor, your huddled masses yearning to breathe free.” Also in New York, the Lower East Side Tenement Museum shows the conditions in which some of those immigrants lived.

The fact that London already has its own similar museum, 19 Princelet Street in Spitalfields, arguably makes the challenge facing the Migration Museum’s backers even harder. Princelet Street is a listed Georgian house that was home to Huguenot silk weavers, later lived in by Irish and Jews who built a synagogue in the garden, and then by the celebrated scholar and recluse David Rodinsky before he vanished without a trace. But although it has been owned by a charity since 1983, and proudly declares itself Europe’s first museum of immigration, Princelet Street is rarely open to the public and mainly deals with private groups.

If it can raise the £3-4m needed for restoration, this will change, but as chair Susie Symes says as she shows me around, the house is not a museum of objects but a “museum of ideas”. Its curators are more likely to commission pieces by new artists than to mount a historical exhibition about the generations of people who have moved to the UK.

‘Migration is everybody’s story. The only difference between people is how long ago they came’ … Barbara Roche (left) and Sophie Henderson of the Migration Museum Project. Photograph: David Levene for the Guardian

But even if their visions of what it should look like differ, Symes, Henderson and Roche all agree that a museum is the ideal forum for the discussion about immigration they think Britain urgently needs. With a migration crisis gripping Europe, and three months after Ukip won almost 4m votes in the general election with a campaign that seemed to suggest cutting the number of immigrants was the answer to every question, you might think their time has come. In France, the museum of immigration was a direct consequence of the Front National’s breakthrough in the 2002 presidential race, when Jean-Marie Le Pen beat the Socialists and ended up in a run-off won by Jacques Chirac.

Chirac decided a museum could help heal the rift in French society, but his successor, Nicolas Sarkozy, disagreed, and the museum waited seven years for its formal inauguration by President François Hollande, which finally took place in December last year. If the Front National’s continuing success shows that the museum was no panacea, recent administrative changes have put it on more solid ground, with new funds promised. Immigration is now a topic on the French school curriculum, and the museum trains 2,000 teachers each year. In March, it hosted a debate following the Charlie Hebdo murders.

Could a new museum play a similar role in British cultural life? If either the Migration Museum Project or 19 Princelet Street can raise the funds and realise their ambitions, we will soon know.

• Adopting Britain is at the Southbank Centre, London SE1, until 6 September. Germans in Britain is at General Register House, Edinburgh, until 7 August.