An “addiction to exams” is fuelling stress, anxiety and failure in schools across the UK, the former head of the civil service has said.

Gus O’Donnell said the UK was “sleepwalking into a deepening crisis” and called for the current exam system to be overhauled.

Speaking exclusively to the Guardian, Lord O’Donnell said Britain was creating a “troubled generation” and the country would pay the price of cutbacks in mental health services.

The warning comes after research into young people’s mental health in England found 20% of girls aged 17-19 and 10% of boys the same age had self-harmed or tried to kill themselves.

Experts described the figures as deeply worrying and raised questions about the damage that social media, pressure to look good and sexual violence were doing to the mental welfare of young women in England.

O’Donnell said: “We are addicted to exams … but measuring success by exam results is fuelling stress, anxiety and failure among pupils, teachers, and schools.

“It’s clear from what we know that improving kids’ wellbeing is a much better way of improving their lives than improving their scores in these exams.

“We are sleepwalking into a deepening crisis when it comes to the mental health of kids and we need to drastically change the way we assess kids. It’s a fundamental thing about what constitutes a successful life …

“A good question for all of us to think about is: what would the world look like if I hadn’t existed? If you can think: ‘I made the world a better place’ then that is what it’s all about.”

The latest figures show disorders are much more common among boys than girls from the age of two to 10. They then equalise among those aged 11 to 16, before becoming much more widespread among girls (23.9%) than boys (10.3%) aged 17 to 19.

O’Donnell said exams were fuelling the crisis, calling for an urgent change to how success is defined.

Comparing the UK with the Netherlands, he said: “It’s not a trade-off between that and traditional exams. The Netherlands is miles ahead of us on literacy, numeracy and science, and the same is true of Finland, where they start primary school later. We are getting something wrong.”

O’Donnell, who served as Britain’s most senior civil servant under Tony Blair, Gordon Brown and David Cameron, also said he would be in favour of Ofsted measuring wellbeing.

“Ofsted has become scary and very negative … I would like to find a way to make it more positive. Lots of things will affect a child’s wellbeing, it’s not just the school – not just a teacher … but on balance I would have Ofsted measuring that [wellbeing] and we’d need to think very carefully – do pilot work on how to best to do that.”

He also said mental health services were not good enough.

“Current provision is woefully inadequate, there is no question about that. We talk about equality between physical and mental health but we are miles away from it. There are plenty of people with diagnosed mental health issues having to wait far too long … We need more resources in this area,” O’Donnell said.

“We have brought forward a troubled generation and we will pay the price of some of the cutbacks we’ve seen in mental health services and the reduction in benefits. Austerity will leave a mark, I am afraid.”

O’Donnell said he knew people who had been through periods of poor mental health and had been touched by people who had killed themselves, adding: “I’ve seen the pain it causes and wonder: could more have been done to prevent that?”

He also took aim at Brexit, saying it was “sucking the political oxygen out of everything”.

“There is no time for legislation because of what the house is up to all the time. Even if there had been clear long-term policy – and [Theresa] May suggested she was interested in the subject of children’s mental health – nothing would happen,” O’Donnell said.

Discussing the role of civil servants in Britain’s departure from the EU, he said: “They have done a really good job to help politicians get the best deal they could, given the way they decided to manage red lines and trade-offs.

“You cannot help thinking it would have been good to have this debate about what particular Brexit do we want, either before the referendum, or if not then before triggering article 50, and it’s a bit late now.

“That is what has led us into this mess. It’s another example, of the same thing – if you don’t define what constitutes success, you are not quite sure where you are going and you don’t have a road map or strategy, and that is the fundamental problem.”