I felt teachers often judged me because of the way I looked. I didn’t get support for my anger management issues. I wasn’t given chances and often left in an exclusion room.” So says 16-year-old Mehdi, describing his experience in mainstream schools.

Mehdi says that his approach to education has been transformed since he arrived at London East Alternative Provision (Leap), in Tower Hamlets. While acknowledging the reasons for his exclusion from his last school, he says he felt unsupported in mainstream schools and that teachers were more concerned with the majority of less needy pupils.

“Here my mindset has changed. Teachers recognise our mood changes, put us in a safe place and push us in smaller classes, which allows us to learn.”

This school is among those hosting the first cohort of teachers training to specialise in understanding the needs of students such as Mehdi. It is thanks to The Difference, a new social enterprise devoted to improving outcomes for the most vulnerable pupils.

In the last six months exclusions have risen up the political agenda. The recent spate of knife crime, allied to growing evidence of some schools “off-rolling” their most needy pupils, has propelled the problem into the limelight.

A long awaited review from government, published last month, highlighted action needed to tackle this challenging issue, with its most headline-grabbing suggestion that schools be held accountable for the children they exclude.

Less attention was paid to another theme in the report from the former minister Edward Timpson: the urgent need to build expertise and capacity in mainstream and alternative provision schools – often known as pupil referral units – where many excluded pupils complete their education.

For Kiran Gill, founder of The Difference, the current public debate is a welcome recognition of a need her organisation highlighted more than two years ago.

The Difference has now recruited its first teachers, who will start placements in some of the best alternative provision in the country from September. Their aim is to support pupils at risk of exclusion by becoming experts in mental health and the emotional and psychological trauma that often underpin challenging behaviour. The teachers may go back into mainstream schools trained as specialists, or stay in alternative provision.

Exclusions have been rising since 2013, with male pupils from disadvantaged backgrounds, certain ethnic groups and those with special most likely to be excluded. Only 1% of excluded pupils get the five good GCSEs they need to access the workforce, and a study of UK prisoners found 42% had been permanently excluded from school.

The first round of expressions of interest for The Difference was oversubscribed, with 200 applicants from primary and secondary schools.

One of the successful applicants, David Adum-Yeboah, is giving up a faculty headship in a mainstream school to work at Leap, because he felt he didn’t have the skills to deal with challenging students such as Mehdi.

“I have worked in schools where I wasn’t exposed to many children from difficult backgrounds. It was only when I went to an inner-city school that I realised how unskilled I was about challenge and defiance,” he says. “I want to be able to spot the issues early on and put corrective actions in place, so kids like Mehdi can stay in mainstream. The brilliant work that happens here could happen in a mainstream school.”

Heads and practitioners already involved with the scheme acknowledge that zero exclusions may be unrealistic, but maintain that high expectations and inclusion are not mutually exclusive.

The Difference has links with the Commission for Ethical School Leadership, which is striving to find a common set of ethical values to guide headteacher decision-making.

Astrid Schon, deputy head of Leap, which achieves results and pupil destinations significantly above the national average for excluded pupils, says most of its pupils have a family history of trauma, such as domestic violence, drugs, and bereavement. “Children who behave in an extreme way must have had trauma or conflict, and alarm bells should go off straight away,” she says.

“Their behaviour is a form of communication, yet often these issues have not been dealt with at an earlier stage when the children really need a bespoke approach, which we aim to give.

“We get to know them so well. We know before they set foot in the classroom if something is off and can immediately work with them to de-escalate a difficult situation.”

Another Tower Hamlets head, Jemima Reilly from Morpeth school, believes many school leaders are looking for alternatives to the “no excuses” behaviour policies that penalise children for minor infringements of school policy and have been lauded in recent years.

Her school will be sending a member of staff on a subsidiary strand of the Difference work: specialist development work on inclusion. Reilly says she is looking forward to new ways of working being brought back into her school even if it challenges existing ways of working.

“Accountability drives schools down the progress route and we can get behind on social and emotional health issues, especially as there is piecemeal mental health support for schools,” she says. “This is really a way of addressing that. It is very easy to get dragged down the path of detentions, then more detentions, without thinking about the causes.

“There must be an alternative narrative to the zero-tolerance approach and it should be incorporated in teacher training from the beginning,” she says. “We are de-skilling teachers if we take away pupils with bad behaviour. Acknowledging they may have had earlier traumatic experiences can depersonalise tense situations and be helpful to all teachers.”

Ultimately, the scheme aims to create what Gill calls a “coalition of leaders who think differently and can change perceptions and practice. “There is not enough understanding of what works in the alternative provision sector,” she says. “Equally, mainstream teachers can bring expertise to their placements and may stay in alternative provision and raise the status of working in AP as an aspirational career route.”

Adum-Yeboah doesn’t rule out staying in the AP sector: “I want to be a head in the future but I also want to be a head who can recognise trauma and early triggers,” he says.

“This scheme is the antidote to those systems where kids are thrown out and made the forgotten population.”