I was only 27 years old, but it felt as if my entire teaching future had already been mapped out for me. As the deputy head of English in a prestigious secondary school, my meetings with the head seemed to revolve around my career advancement prospects: where did I see myself in five, 10, 20 years' time? Head of English? A member of the senior management team? I didn't want any of those. Not yet anyway. So what did I want? I wasn't sure.

I resolved to take some time off, but I didn't want to just travel the world with a backpack and a guidebook, barely scratching the surface of each culture I dipped into. Instead, I wanted a journey with purpose. The idea, when it came, was fully formed, sparked by a surprising statistic I read one morning in the staffroom: nearly half of all England's newly qualified teachers were leaving the profession within their first five years.

I wanted to know why.

So I took to the road to find out: moving into my old and rusting VW campervan; signing up to a supply-teacher agency. I then spent a year travelling through the 10 areas of the country that were deemed as having the most challenging schools, one month in each, and teaching in those schools. By witnessing the frontline, I could find an answer to why so many teachers were fleeing the profession, and I could journey through my own country – and experience it as I never had before.

Beginning in Nottingham, where I had done my degree and PGCE, I was booked in to a secondary school to cover an English teacher on a long-term absence due to stress. In this school, over the last academic year, almost a quarter of the staff had resigned. While I was here, one young man threatened to break my nose; another stabbed his friend in the hand with an unfolded paper-clip, drawing blood; a girl spent an entire lesson hopping about outside my room, bellowing obscenities through the window at me. An 11-year-old had to be removed from a lesson for shouting at his classmate, an orphaned Somali refugee, "At least I've got a family to go home to". One day, when a violent fight broke out in my classroom, I felt horribly aware that if I tried to break the fight up I could be reported and perhaps even sued. I had no choice but to stand back, shout at them to stop, and be ignored.

The next two months followed in this fashion as I worked in Manchester and Birmingham. The nights grew increasingly cold, and I cursed my idea of weathering them inside an unheated van, sleeping on A-road laybys because I often could not afford a campsite. I washed each morning with the chilled water that spat from the van's ineffectual tap, and shaved quickly and haphazardly over the tiny sink. My working day was dominated by confrontations with aggressive, disaffected or miserable teenagers.

On one particularly memorable day, I was in a school in the West Midlands – a small and specialist school for pupils with emotional, behavioural and social difficulties, many of whose students had been excluded from mainstream schools. These were children with a proliferation of asbos; children on terrifyingly high dosages of Ritalin; some with criminal records and, already in their short lives, histories of violence.

During one lesson, I taught Caroline, one of the few girls at this school: a tiny 11-year-old with huge, pretty eyes and an endearingly babyish appearance.

"Shall we have a go at this work, Caroline?" I asked her.

She turned and stared at me. "Fuck off, you fucking southern cunt," she said. "Fuck off back down south. No one wants you here. We all fucking hate you."

There was a calm and committed malevolence in her voice and, for one brief but terrifying moment, I thought I might cry. Instead, I decided to be honest. "Caroline, you've really hurt my feelings there. Those are very nasty things to say."

"Fuck your fucking feelings," she said, and marched out.

I spent the next month working for a small tuition centre in the Peak District, where I tutored a boy who had been permanently excluded from his school for drug-dealing, and then followed that with two months in Sheffield and West Yorkshire.

It was in the latter, out on the fringes of the great Leeds-Bradford conurbation, that I taught at a secondary school and met Ralph, a 13-year-old boy who took an instant dislike to me. Things came to a head one morning when Ralph walked out of my classroom. When I followed him out into the corridor, Ralph turned, screaming that he would break my jaw if I didn't turn around and go back inside the classroom. When I didn't, he launched.

Time slowed. I still remember that scene now: the view over my fingertips as Ralph pushed forward and raised his fist. Shamefully, I backed away and slipped into the classroom to the sound of his shouts: "Fucking posh cunt".

I sat down at my desk and noticed I was shaking. I felt something deep and necessary to my confidence had been broken for ever. A line exists between teacher and student, a line that cannot be crossed in either direction, a very physical line. And Ralph had just shattered it. I've been threatened by a student more times than I can count, but this was the first time I truly believed a student would go through with a threat. I left the school soon after.

I stumbled through the rest of my year, but it was never quite the same again. I taught in tough schools across London, the West Country, Liverpool and Middlesbrough, enduring along the way the threat of violence I was becoming increasingly attuned to. In London, the concept of knife-crime was ever-present as members of various senior management teams entered my classroom to wave squeaking and popping security wands over the students to check if they were carrying knives.

In one school in Liverpool, one boy, Saeed, faced down his bully, Alex, in my classroom by producing a knife and waving it in front of his enemy's face. I froze along with the rest of the class and with Alex as Saeed slowly raised his other hand, extended the forefinger, and lightly placed it on the tip of the blade. He gave a slight pull, and the knife bent, twanging back into place when released. It was plastic.

The class dissolved into laughter, Saeed was escorted from the room, Alex slapped the table he stood next to and hooted, "Fuck me! Fuck me!". I went back to my van that night and got so drunk on cheap red wine that I was sick.

When the year ended, I returned to my home county of Cornwall and took a summer job working in a village pub, living in my van in a field, and reflecting on my weird year. I would tell the locals about my journey, and they would ask how I had managed to last a whole year.

And, when I reflected on it, I would remember the good things as well as the bad. Even in the failing schools, there had still been individual students who were trying so hard, who were brilliant, in fact, and each of them had given me a little morale boost each day that pushed me onwards.

I had also seen in these challenging areas some wonderful schools, which, beset as they were with their difficult intake, would still thrive against the odds – three of the schools I had worked at in particularly difficult areas had achieved outstanding status in their most recent Ofsted inspections. What was it, then, that set these schools apart?

It had felt to me in these schools that a teacher really could make a difference. Supported by good and hands-on senior management teams – rather than by shadowy headteachers who rarely enter their classrooms – the teachers knew that any sanctions they implemented would be backed up, which empowered them, in turn, to support and encourage their students to achieve to the best of their capabilities. In one school, when a pupil shouted an expletive at me, the head put him on a temporary exclusion for swearing at a teacher. This kind of thing is what counts.

My aim had been to find out why so many teachers were leaving. And I think I did find my answer – a score of them, in fact, and a few ideas about what can be done to make things better. In order to stop teachers leaving, it's useless throwing money at them (no teacher teaches for the money), or implementing structural innovations such as academies or free schools. Instead, changes need to be made at a much more fundamental, frontline level, which involves supporting teachers and assisting them to support their students to learn and achieve to their maximum capability – which is, after all, what teachers train to do.

Such changes are not dramatic or expensive. They merely require a slight shift in the cultural attitude. With more power to stop violence in the classrooms, with more freedom to exclude those students who cannot cope with mainstream education, with smaller class-sizes, with an enhanced communication between teachers and parents, with protection for teachers against the overwhelming fear of litigation, with the encouragement of a zero-tolerance approach to such unacceptable misbehaviour as violence and psychological abuse (such as cyber-bullying), and with the removal of the league table culture – where schools are unfairly ranked by a cold system of results-based numbers – perhaps teachers would be encouraged to remain in their profession and continue to provide the service so invaluable to this country's future.

I still teach, and at times still love it, but I don't know for how much longer. For, until such changes are implemented, it is this teacher's opinion that the professional exodus will continue.

• Charlie Carroll is a pseudonym. All names have been changed. On The Edge by Charlie Carroll is published by Monday Books, price £8.99. To order a copy for £7.19, with free UK p&p, go to theguardian.com/bookshop or call 0330 333 6846