The Labour party has suffered some flesh wounds in the past few days. Tom Watson’s resignation was unexpected, and I write as someone who spoke to him on the very morning he stood down and who had absolutely no idea.

The next morning, as I stood on a rainy street in Stechford, many messages came in to my phone to ask me if I was going to either stand to be deputy leader or if I was going to leave the Labour party.

That they were directly contradictory was amusing. The reality was that these messages were a distraction because I was a bit busy: no matter how much campaigning AI we develop, the doors in front of me simply do not knock themselves. So I was planning on doing neither.

I’ve left the Labour party once before, over Iraq, so I am no stranger to the feeling you get when you head for the door. I don’t have that feeling at the moment, although not all is rosy. I am maddened by some of the players, I am maddened by how the selection process for candidates is being handled to create new players.

I am left at times isolated by those I love and respect fleeing the party nest. The problems of prejudice are shameful and belittle our proud history of progressive values, and, if I am honest, it is bloody freezing on the doorstep, so packing it all in and getting back to my house is also tempting. But then I remember what I am currently fighting against and I know I must carry on.

Every day in Yardley I meet people who are living in grotty hotels because there are no homes for them, or single parents nearly £400 worse off a month because of universal credit. I meet people with terminal illnesses who have been refused their benefits.

Every Friday afternoon I have to make arrangements for where my son will go because his school cannot afford to stay open past 1. This isn’t an accident – someone did this to my people, and to me.

I have felt the force of what governments can do. I remember my elder son being in the first cohort of kids who got a free nursery place, I remember the palliative care my mother got at home as I watched her die. I remember the children’s centre where I took my babies when I needed a community to help me cope. My sons both have savings accounts for their future which were given to them with money donated to them at birth. I remember what it felt like to feel that things were getting better.

I might have given the Labour party the last six years of my life but it has given me and my family a foundation. I am not just loyal to the Labour party; I literally owe it. I am in the red.

Michael Roby leaves Birmingham magistrates court after pleading guilty to a public order offence at Jess Phillips’s constituency office a day after a parliamentary Brexit debate. Photograph: Matthew Cooper/PA

The rise of the politics of hatred and division will not be beaten if I walk away. Last week I met a man who had been convicted of a public order offence after he came to my office and tried to kick the door in while he shouted that I was a fascist. This man had been told by divisive forces that people like me hated people like him, so he hated me in return.

I don’t hate him, I never did. People who claimed to represent his kind lied to him and it harmed him. We sat down and talked together, we chatted about Brexit together, laughed together and reminisced about the streets we both grew up on. The politics of hope is harder to spread than the politics of hate. Those currently in power have decided on the latter.

The Labour party is not perfect but I have seen in my own life how it is the greatest vehicle for positive hopeful social change. It is the only vehicle, I think, that has even an outside chance of fighting the divisive decade we are living in.

That means speaking up when Labour gets it wrong too, but any woman in our party will tell you that fighting for progress from within and increasing your numbers in the ranks is the only way to do it. Progress within and without our movement was never given easily, it always had to be won, with resistance and persistence.

When I talk with my sons in years to come I want to be able to say that I stayed on board to stop the hatred, division and regression. This brilliant vehicle for social change may need some repairs as it heads down the road; I will have my jump leads ready and with others I’ll try to keep it going.

The Labour party is bigger than any one of us and if it didn’t exist we would have to invent it. The struggle is real.

Jess Phillips is Labour candidate for Birmingham Yardley