When I was a teenager I went to West Berlin with my local youth orchestra to take part in an Anglo-German cultural exchange. It was 1983 and the wall was up. As we toured the city over 10 days, we would keep butting into this grotesque cold war installation blocking our way, and butting up against my 14-year-old’s defence of socialism.

At that age I reflexively rejected most dominant narratives about race, class and nation. During a period of sus laws and anti-union legislation, I already understood there had to be another version of freedom out there that included me, and I was busy piecing together the fragments of my own worldview. And yet no amount of rationalisation could shake my conclusion that people whom I disagreed with about pretty much everything else were right about the wall.

Clearly, built with the deliberate intention to trap people in a place they might not want to be, the wall was heinous – not just a bad idea, but morally wrong. As such, it was the most obscene symbol of the broader case against the eastern bloc. The fact their governments would not allow residents to travel to the west was prima facie evidence of their lack of freedom: they were understood to be like open prisons.

Not long after the wall came down, this entire logic went into reverse. As country after country shed its Stalinist overlords and went into free-market freefall, the case for their peoples’ right to leave was eclipsed by the fear that they might actually come. In the west their “freedom” was welcomed; their presence was not. While they were demolishing the wall, we were building a fortress. Politics kept them in. For more than a decade, before they gained admission to the European Union, economics would keep them out.

To be black and on the move in the west is to be an object of suspicion

“A map of the world that does not include utopia is not worth even glancing at,” wrote Oscar Wilde. “For it leaves out the one country at which humanity is always landing. And when humanity lands there, it looks out, and, seeing a better country, sets sail.”

West Germans demolish the Berlin Wall in 1989. Photograph: Sipa Press/Rex

The map of my utopian world has no borders. I believe in the free movement of people. As a principle, I think we should all be able to roam the planet and live, love and create where we wish. I could squander the rest of this column parrying caveats and concerns regarding everything from security to wages. But it is the last in a series on utopian thinking: that demands imagining beyond what is possible and practical to what is desirable. So why dream small? Martin Luther King could have climbed the steps of the Lincoln Memorial and called for a 10-point plan for better legislation. But then who would have remembered that?

Benedict Anderson once famously described nation states as “imagined communities”. I’d like to imagine mine without border guards, barbed wire, passport control, walls, fences or barriers. The world would be a better place without them.

Some of this stems from personal history. I am from a travelling people. My parents were born and raised in Barbados, a small island in the Caribbean caught in the crosswinds of colonial ties and postwar labour scarcity. I have 14 aunts and uncles. Along with my parents, nine of them left Barbados for lives in Britain, the US and Canada. I have cousins scattered across the globe. Borders are no friends to diasporas. They privilege form-filling over family.

“Do you plan to work while you are here?” the UK immigration officer asked my grandmother when she came to see us one time. “You have cane here?” she asked wryly – she who worked in the canefields her entire life.

Like my granny, though for different reasons, borders have always been a tense issue for me. With those in uniform struggling to match the colour of my face to the crest on my passport, how could it be otherwise? To be black and on the move in the west is to be an object of suspicion. The documents are supposed to speak for themselves; but somehow there was always more explaining to do. And these personal objections are intimately connected to a more sweeping philosophical and political opposition.

Borders exist, by definition, to separate us from others. The primary two issues then become which “other” that will be, and on what basis we should be separated. As such they are both arbitrary and definite. Arbitrary because they could be anywhere and often move – just look at how Europe’s borders have changed over the past century. Definite, because wherever they are we have to deal with them, and because the process that determines who is allowed to move where and why is exercised with extreme prejudice.

German police escort immigrants who have crossed from Austria to a registration centre, 2015. Photograph: Michael Dalder/Reuters

America’s 1882 Chinese Exclusion Act, the White Australia Policy – a series of measures lasting 70 years – or Britain’s Commonwealth Immigrants Act of 1962 are among the more crude filters. But while the “othering” changes with time – the shift from race to religion as grounds for suspicion over a generation has been breathtaking – the fact of it remains the same. Some people won’t be welcome, not because of what they have done but because of who they are, even if the groups of people in question may change.

This has long been true and long been a problem. Recently, however, it has been compounded by the fact that even as borders have become tougher for people, they have all but been lifted for capital. Money can travel the globe virtually without restriction, in search of regulations that are weaker and labour that is cheaper.

When it does, it often displaces people: sucking investment and resources from one place at the flick of a switch; shutting down factories and shifting them to the other side of the world; or introducing automation that renders some professions obsolete. But those who find their lives turned upside down by the free movement of capital are often prevented from moving country and looking for work. People should have at least the same rights – or more, since humans are more valuable than money.

Sadly, that is not a principle that underpins the system we live in. The rich can buy themselves citizenship in about 20 countries, cash down. Meanwhile, desperate people are turned away at borders all the time. Others are incarcerated for having the audacity to cross borders we have created, to escape wars we have started, environmental chaos we have contributed to, or poverty we have helped create. Others die trying.

It is a fact, rarely stated but generally acknowledged and accepted, that the global poor should not be allowed to travel. That’s most of the world. As such, from the refugee camp in Calais to the rickety vessels on the Mediterranean, from Trump’s wall to the Berlin wall, the border stands as an ultimate point of confrontation in the broader dystopia we have made possible.

Nation states are a relatively new concept; migration is as old as humanity. Borders seek to regulate and restrict that basic human custom for the distinct purpose of excluding some and privileging others. They discriminate between all people with the express intention of then being able to discriminate against some people. They do not simply set boundaries for countries, but are metaphors for the boundaries of how we might think about other human beings. Immigrants are not the problem. Borders are.

• Gary Younge is editor-at-large for the Guardian