It was 01:00 in Tokyo and we were trying to get into an apartment with a pair of stolen keys. We’d got lost trying to find our Airbnb, and were now at an address that looked 90% right. Turns out in the middle of the night that’s close enough. There were keys in the letterbox, where our host said they’d be, so when the security code didn’t work I just worked my surprisingly flexible hands through the slot to grab them, like a modern-day Artful Dodger.

At this point, you’d think we might have stopped to take stock, but we were fuelled by the kind of determination that comes from a 12-hour flight followed by the public-transport pilgrimage from Narita to Shinjuku. But midway through jiggling the keys in the lock, the door whipped open to reveal a nightdress-clad lady and her daughter looking out at us in bemusement. Definitely not our Airbnb then.

Amazingly though, instead of screaming at us, calling the police or both, for the next 20 minutes they tried to help us find the right address. All without us speaking a word of Japanese, or them speaking a word of English. And when we turned up nothing, they formally apologised to us. The gaikokujin, or foreigners, who’d just tried to break into their home.

You may also be interested in:

• Where being nice is the law

• The Swedish word poached by the world

• The country that loves to give

The trope of Japanese apologies is a common one, frequently taken as mere self-effacement at best or, at worst, as unnecessary self-flagellation. And indeed, as a gaijin in Japan, apologising can seem a way of life here. There are dedicated sorry gestures: for example, a hand held up against your forehead is both an apology and a way for little old ladies to plough through crowds, like tiny battleships in a sea of people.

Only 10% of ‘sumimasen’ is an apology. Ninety percent is used to show respect, politeness, honesty

There are also myriad ways to apologise. At least 20, according to this BBC explainer. The lady in our not-quite-apartment used the more formal ‘gomen-nasai’, but the most common is ‘sumimasen’, passed onto me as the most useful word we could learn for the trip. Roughly translated as an apologetic ‘excuse me’, it rings out of doorways, taxis, shops and restaurants, leaving ‘arigatou’ (thank you) by the wayside. This often translates into an assumption that the Japanese apologise to the exclusion of all else.

According to Laurie Inokuma, who holds a degree from Cornell in Japanese language and worked for Japan Airlines for 15 years, however, this isn’t true. ‘Sumimasen’, for instance, isn’t necessarily replacing ‘arigatou’ – it’s encompassing it.

“Only 10% of ‘sumimasen’ is an apology. Ninety percent is used to show respect, politeness, honesty,” she said. “It’s an everyday word. When someone does something for you, getting out of your way in the grocery store, or holding a door, ‘ah, sumimasen’ is the common response.”

Just as easily a ‘thank you’ as a ‘sorry’, ‘sumimasen’ is regularly used to acknowledge the trouble someone has gone to for you. “There's a humility in it; depending on the situation it’s either apologetic or grateful,” Inokuma said.

Erin Niimi Longhurst, the British-Japanese author of Japonisme, which looks at how Japanese traditions can help create a more thoughtful life, agrees. “There’s a culture of apology but also a culture of thankfulness. One of my favourite anecdotes is when my British aunt met a Japanese lady at a conference, then took her along to a family dinner. This woman came in and had beautifully wrapped little gifts that she handed out to us, all from Japan. There were even presents for my much younger brother and sister. She had no idea she was going to be last-minute invited to this dinner, but had brought gifts with her and things to wrap them in, just in case. It was incredible.”

This year’s World Cup is an even grander example of this level of courtesy: when Japan lost its final match, the team made headlines when they stayed behind to clean the entire changing room. They even left a thank-you note.

So, if apologies are just one cog in the larger moving wheel of Japanese politeness, where does the overarching cultural concept come from?

“There is a need for politeness in Japan to get along with living on top of your neighbours – it’s a respect for others,” Inokuma said. In Tokyo, watching great swaths of humanity queuing politely for miles on end to get into Shinjuku Gyoen park or shuffling forwards towards Nakameguro’s riverside during cherry blossom season, this makes sense.

There is a need for politeness in Japan to get along with living on top of your neighbours

Japan has some of the most densely packed cities in the world, with a whopping 93.93% urban population. Tokyo, for instance, has around 6,150 people per sq km, in comparison to London’s 5,729 (bear in mind, that includes Tokyo’s expansive outer suburbs – the bulk of residents are concentrated in the city centre of Greater Tokyo, the world’s most populous metropolitan area, and a further 2.4 million commute in every day). The average living space per person in the city is an elbow-bumping 22 sq m across the country, going down to 19 sq m in Tokyo. We experienced this first hand, staying in apartments throughout our trip that were unanimously spotless, homely – and unbelievably tiny. When there’s a premium on space, it suddenly seems natural to become as considerate of it as possible.

“There is this respect for other people’s space,” Longhurst affirmed. “When you go into a Japanese home you always take your shoes off – a separation of outside and inside. There’s also an attitude of ‘meiwaku’, meaning ‘sorry to bother you’ or ‘sorry to come into your space’.”

But this bone-deep politeness isn’t just a reaction to cramped living quarters. When we got out of the cities into the stillness of the Japanese Alps, people were, if anything, even more polite. We walked into Kamikochi, a mountain valley closed for the season: a strenuous two-hour hike that’s usually a 10-minute drive when the buses are running. It was worth it, but when a worker stopped and offered to drive us back, we nearly cried with relief. The day before on the rural bus up to our Okuhida ryokan (a traditional Japanese inn), I’d left my phone and didn’t realise for hours. The bus driver who found it later personally delivered it, calling Find My iPhone’s ‘lost phone’ number of the ryokan flashing on the screen to get the address to drop it off at.

The whole point of it is to be mindful of where you are in a specific moment, and that comes through in how people interact with each other

In Japonisme, Longhurst explores the relationship between modern Japan’s culture and its traditions, and for her, the nation’s culture of apologising – and the general politeness this is part of – largely comes down to mindfulness. “Many of the Japanese practices are about having this relationship with yourself and the natural world. In things like tea ceremony, a lot of it is about having an awareness of where you are in a specific moment in time. You’ll serve tea, but it’s not just about the tea, it’s about the flower arrangement in the corner that will defer to what season it is, the calligraphy scroll on the wall that will mention what time of year it is. The whole point of it is to be mindful of where you are in a specific moment, and that comes through in how people interact with each other.”

The prevalence of tea houses in Japan’s tourist attractions is testament to this. Wandering the Okochi-Sanso villa in Kyoto, instead of plunging towards the next awe-inspiring sight, we ended with a cup of matcha looking out of a bamboo-obscured window. In comparison to the seething mass of selfie-stick–laden bodies flowing down the Arashiyama district’s popular Bamboo Grove outside, it was a quiet moment of calm. The long exhale to sightseeing’s held breath.

Back in Tokyo, Hidetsugu Ueno, owner of Bar High Five, delved further into this concept, agreeing that mindfulness is part of apologising in Japan – but adding that it goes hand in hand with empathy. “Of course we don't want to apologise if we don't need to. But we can put ourselves in other people’s shoes and feel sorry for them, so we want to say it aloud.” The practice of mindfulness includes being aware of other people within its scope, but, put like this, apologising also stems from a wider emotional ability to understand the feelings of others.

Japan’s crime rate backs this up – the country is renowned for its low crime, boasting some of the lowest homicide rates in the world. Ueno said, “There are crimes here in Japan. We aren't monks. But if we see a wallet on the street, most Japanese will bring it to the police station. We know how much people suffer when they lose their wallet. If you think about it happening to you, you should know how to react. We learn that when we’re small kids at school.”

It’s a chicken-and-egg scenario: is this cultural empathy born of morality, or vice versa? Children in Japanese classrooms have had ‘moral education’ lessons since 1958, teaching the importance of co-operating with others for everyone’s benefit, a concept said to have originated from the samurai.

“This does tie into that historic samurai culture, but a lot of it also comes from wanting to preserve a group dynamic, as well as that idea of doing something for the betterment of others,” Longhurst explained. You only have to remember the Fukushima 50, a group of older workers who stayed behind to help repair the Fukushima nuclear power plant following 2011’s devastating tsunami, to see this in modern-day effect.

In Japan, it’s evident that apologising is a panacea, refined into its own complex language. But this language is also a mirror to Japan’s wider culture. Here ‘sorry’ is the window to a labyrinthine blend of politeness, respect and morality that seems explained partially by the realities of living in a crowded island nation and partially an adherence to the oft-cited rule of treating others as you would want to be treated yourself.

Politeness and respect are just in the Japanese DNA

“The complexities of the culture and the language are intertwined. My husband says politeness and respect are just in the Japanese DNA.” Inokuma mused.

But for me, it was Ueno who summed it up best. “People should be honest, should be kind, should be sincere. People should be that way, no?”

Why We Are What We Are is a BBC Travel series examining the characteristics of a country and investigating whether they are true.

Join more than three million BBC Travel fans by liking us on Facebook, or follow us on Twitter and Instagram.

If you liked this story, sign up for the weekly bbc.com features newsletter called "If You Only Read 6 Things This Week". A handpicked selection of stories from BBC Future, Earth, Culture, Capital and Travel, delivered to your inbox every Friday.