After Japan’s prestigious University of Tokyo fell from its number one spot to number seven in Times Higher Education’s Asia University Rankings earlier this year, I had a chance to travel to Tokyo to interview more than 40 people involved with various parts of the country’s education system.

Students, academics and professionals told me they felt a blow to their national pride from the news of the rankings drop. I found that the THE rankings result underscored the complex problems plaguing the country’s institutions of higher learning.

According to my interviewees, the Japanese education system is seen as a dated institution that is preventing Japan from reaching its full potential as a player, commensurate with its economic size, in international affairs today. The Japanese system served its purpose well during a period of booming economic growth that spanned the 1950s to the 1980s. Now, however, the system is failing to adapt to a new era and prepare workers for the global marketplace (as I have recently argued).

What was startling about my findings was the degree to which interviews confirmed the stereotype of Japanese education as emphasising rote memorisation and fostering a rigid, unquestioning society.

To be sure, Japanese students rank among the best in the Organisation for Economic Co-operation and Development nations for maths and science. But liberal arts and humanities are languishing, even though they are critical to the growing creative industries of the future. According to a survey conducted by Education First, Japan ranks 30th out of 70 in terms of English proficiency.

At its worst, Japan’s education system is seen as producing “mindless drones” incapable of critical thinking and an “obedient public” that is politically aloof.

Challenges include a glut in the number of colleges, a shrinking student population, a lack of a university managerial class, a pervasive problem in English language skills and critical thinking, declining funding, no tradition of fundraising, and a struggle to attract the top researchers and students globally.

Modern Japanese society has placed an importance on hierarchy, social harmony and conformity. These celebrated virtues are great for stability and efficiency but they also make societal disruption extremely difficult and slow. Meanwhile parents tend to blame government officials rather than take matters into their own hands.

A compounding problem is that every stakeholder – from parents to teachers to students to education professionals – is invested in the status quo. The country’s authority figures are products of the system, and making a change means taking on risk and expending additional effort. Without a crisis, real change may therefore take decades (at least 20 years) to achieve, according to many interviewees.

And yet, a sense of urgency is absent because theirs is an objectively safe, comfortable, orderly society that provides a rich cultural environment that can satisfy the most sophisticated tastes. In other words, Japan is a victim of its own success.

A cultural paradox is emerging in youth culture in Japan as well as elsewhere in the developed world. On one hand, many young people are becoming more liberal-minded. That is to say, they are increasingly accepting of foreigners, mixed marriages, gay marriage, divorce, different lifestyles and entrepreneurship. On the other hand, they have also become more conservative in their behaviour, avoiding risk, dating, political change, sexual relationships, sport, career changes, study abroad and travel.

As a result, they are ill-equipped to operate smoothly in the world. Government-supported programmes to encourage a more international outlook in Japanese universities have had little effect.

According to interviewees, young Japanese are increasingly inward-looking, as they can experience a culturally rich, global, extraordinarily safe metropolitan life in Tokyo, while the world outside appears more and more dangerous – with terrorist attacks and political chaos appearing on online news constantly. Young people’s goals have been correspondingly downgraded to “I want enough money to go drinking with my friends after work”.

For their life ambitions, they hope to obtain “a decent job to support a family and retire”.

Japan has become too comfortable, I was told, and young people often insulate themselves within a small group of friends connected by social networks. And yet, paradoxically, Japanese people do care about how their country is viewed abroad, which is why the THE rankings hit a nerve.

Compared to 10 years ago, students are less interested in studying abroad because of the stresses of job-hunting, which begins in early adulthood. Companies interview students in the third year of higher education, which is also the year that Japanese students typically study abroad. Thus many students simply stay home for fear of falling behind in the job search process. The government has encouraged companies to push back the interview timing in order to allow students to avoid this trade-off, but to little avail.

Instead, a palpable pessimism about the future remains the norm. Polls confirm this attitude. People worry about their financial security, and the job market within Japan has become very competitive, especially since the 2008 financial crisis.

Exacerbating the anxiety, younger people have become more aware of how competitive it is out there in the world more broadly. There are indeed some young “changemakers”, but they are a minority and are seen as odd, so there are very few role models.

A recurring theme in my discussions was the necessity for young people to break out of their routines, form broader networks and friendships, access different information sources, and broaden their horizons abroad. During a follow-up trip to Tokyo a few weeks ago, I had a conversation with several university students that brought this point home. Attitudes on issues such as careers and gender equality differed starkly between the students who had spent time abroad and those who had not.

Those who had studied abroad had vastly more open minds.

If the Japanese education system is to better serve society, it will need to find ways of introducing new ideas and possibilities to students. Otherwise, Japan may find itself left behind.

The THE rankings shocked many people in Japan, but they also did something constructive by shattering the bubble of complacency.

Devin Stewart is senior fellow at the Carnegie Council for Ethics in International Affairs in New York City.