Every air passenger is treated with suspicion since 9/11, regardless of appearance. Global traveller Pico Iyer has had to put up with it all his life

I'm sitting in the expansive spaces of Renzo Piano's four-storey airport outside Osaka, sipping an Awake tea from Starbucks and waiting for my bus home. I've chosen to live in Japan for the past 20 years, and I know its rites as I know the way I need tea when feeling displaced, or to head for a righthand window seat as soon as I enter a bus. A small, round-faced Japanese man in his early 30s, accompanied by a tall and somewhat cadaverous man of the same age, approaches me.

"Excuse me," says the small, friendly seeming one; they look like newborn salarymen in their not-quite-perfect suits. "May I see your passport?"

When I look up, surprised, he flashes me a badge showing that he's a plainclothes police officer. Dazed after crossing 16 time zones (from California), I hand him my British passport.

"What are you doing in Japan?"

"I'm writing about it." I pull out my business card with the red embossed logo of Time magazine.

"Time magazine?" says the smiling cop, strangely impressed. "He works for Time magazine," he explains to his lanky and impassive partner. "Very famous magazine," he assures me. "High prestige!"

Then he asks for my address and phone number and where I plan to be for the next 89 days. "If there is some unfortunate incident," he explains, "some terrorist attack" (he's sotto voce now), "then we will know you did it."

Six months later, I fly back to the country I love once more. This time I need to withdraw some yen from an ATM as I stumble out of my trans-Pacific plane, in order to pay for my bus home.

"You're getting some money?" says an attractive young Japanese woman, suddenly appearing beside me with a smile.

"I am. To go back to my apartment."

"You live here?" Few Japanese women have ever come up to me in public, let alone without an introduction, and shown such interest.

"I do."

"May I see your passport?" she asks sweetly, flashing a badge at me, much as the pair of questioners had done two seasons before.

"Just security," she says, anxious not to put me out, as my Japanese neighbours stream, unconcerned, towards the Gakuenmae bus that's about to pull out of its bay.

I tell my friends back in California about these small disruptions and they look much too knowing. It's 9/11, they assure me. Over the past decade, security has tightened around the world, which means that insecurity has increased proportionally. Indeed, in recent years Japan has introduced fingerprinting for all foreign visitors arriving at its airports, and takes photographs of every outsider coming across its borders; a large banner on the wall behind the immigration officers in Osaka – as angry-looking with its red-and-black hand-lettering as a student banner – explains the need for heightened measures in the wake of threats to national order.

But the truth of the matter is that, for those of us with darker skins, and from nations not materially privileged, it was ever thus. When I was 18, I was held in custody in Panama's airport (because of the Indian passport I then carried) and denied formal entry to the nation, while the roguish English friend from high school with whom I was travelling was free to enter with impunity and savour all the dubious pleasures of the Canal Zone. On my way into Hong Kong – a transit lounge of a city if ever there was one, a duty-free zone whose only laws seem to be those of the marketplace – I was hauled into a special cabin for a lengthy interrogation because my face was deemed not to match my (by then British) passport. In Japan I was stripsearched every time I returned to the country, three or four times a year – my lifelong tan moving the authorities to assume that I must be either Saddam Hussein's cousin or an illegal Iranian (or, worst of all, what I really am, a wandering soul with Indian forebears). Once I was sent to a small room in Tokyo reserved for anyone of South Asian ancestry (where bejewelled women in saris loudly complained in exaggerated Oxbridge accents about being taken for common criminals).

Another time, long before my Japanese neighbours had heard of Osama bin Laden, I was even detained on my way out of Osaka – and the British Embassy hastily faxed on a Sunday night – as if any male with brown skin, passable English and a look of shabby quasi-respectability must be doing something wrong if he's crossing a border.

But now, having learned over decades to accept such indignities or injustices, I walk into a chorus of complaints every time I return to California, from my pale-skinned, affluent neighbours. They're patting us down now, my friends object, and they're confiscating our contact-lens fluid. They're forcing us to travel with tiny tubes of toothpaste and moving us to wear loafers when usually we'd prefer lace-ups. They're taking away every bottle of water – but only after bottles of water have been shown to be weapons of mass destruction; they're feeling us up with blue gloves, even here in Santa Barbara, now that they know that underwear can be a lethal weapon.

I listen to their grousing and think that the one thing the 9/11 attacks have achieved, for those of us who spend too much time in airports, is to make suspicion universal; fear and discomfort are equal-opportunity employers now. The world is flat in ways the high-flying global theoreticians don't always acknowledge; these days, even someone from the materially fortunate parts of the world – a man with a ruddy complexion, a woman in a Prada suit – is pulled aside for what is quixotically known as "random screening".

It used to be that the rich corners of the world seemed relatively safe, protected, and the poor ones too dangerous to enter. Now, the logic of the terrorist attacks on New York and Washington has reversed all that. If anything, it's the rich places that feel unsettled. It used to be that officials would alight on people who look like me – from nations of need, in worn jeans, bearing the passports of more prosperous countries – as likely troublemakers; now they realise that even the well-born and well-dressed may not always be well-intentioned.

I understand why my friends feel aggrieved to be treated as if they came from Nigeria or Mexico or India. But I can't really mourn too much that airports, since 9/11, have become places where everyone may be taken to be guilty until proven innocent. The world is all mixed up these days, and America can no longer claim immunity. On 12 September 2001, Le Monde ran its now famous headline: We are all Americans. On 12 September 2011, it might more usefully announce: We are all Indians.

The Terminal Check was originally published in Granta 116: 10 Years Later, available now. Pico Iyer will be in conversation about his work in Granta at Asia House on 5 September. For tickets and information, visit AsiaHouse.org.