Dudley central mosque doesn’t look like a building with the potential to help shape an election. Sitting in the shadow of the impressive Dudley castle, it is a crumbling former school with inadequate parking: depressing and dilapidated, the only hint to its true purpose are the two white domes stuck on its facade.

The Muslim population in here have historically been staunch Labour supporters: the party has seven Muslim councillors on the local authority. But after the community lost a 15-year battle to open a new mosque all that is up for grabs.

In 2003, the Dudley Muslim Association (DMA) announced plans for the mosque, as well as a community centre. The idea was to regenerate a rubbish-strewn, derelict piece of land, with hopes of creating a space for 800 worshippers, as well as providing sport and leisure facilities for the whole community.

But what followed could never have been predicted – and to this day the wounds have not healed. Racist tensions had previously existed in the town – Enoch Powell was Tory MP of neighbouring Wolverhampton South when he made his infamous Rivers of Blood speech – but they had been buried deep. The community lived together in something like harmony.

However, the mosque issue became a tinderbox. Far-right groups, including the EDL, hijacked the campaign to stir up racial hatred and began to descend on the town in their hundreds taking part in mass rallies.

There followed a high court battle. The land was originally given to the DMA under a lease agreement. However, the deadline for developing the site lapsed and in November 2015 following a two week-long court of appeal hearing the DMA was ordered to hand the land back to the Labour council – sounding the death knell for the mosque.

For many of Dudley’s Muslims, the episode is still viewed as a painful one – and one where the far right was able to succeed and cause deep divisions in a place it called home.

Sitting in the mosque’s cold and draughty community centre, mosque chairman Mohammed Aurangzeb looks back at that time with much regret.

As he talks, two policemen come in carrying a letter. Bus drivers are complaining that parents dropping children off at mosque for Quran-reciting lessons are causing an obstruction.

“Do you see what we are up against?” says Aurangzeb. “I was 10 years old when I came to this country, I grew up here and I love this town but we don’t have a proper place to rest our head in prayer. This was our dream – we were building something not just for the future of our children but the whole community but it was never allowed to happen. The politicians bowed down to the extremists.”

For Mohammed and his son-in-law, Shaz Saleem, the current secretary of the mosque and head of the local taxi association, which has more than 150 mainly Muslim members, the blame lies firmly at the door of the Labour party. Both are planning to vote Conservative.

Saleem, previously a lifelong Labour supporter, stood as a Labour council candidate three times but failed to get selected. He has now changed his loyalty and is hoping to become a Conservative councillor.

He said: “We were made false promises by the Labour party and they left us feeling humiliated. All we wanted was a mosque to pray in but what happened was an absolute shambles. We were told one thing to our face and then they stabbed us in the back. We have been left so disillusioned.”

Both parties are aware of the importance of the BAME vote in the West Midlands town and have courted the congregation at the mosque.

According to the 2011 census, the whole of Dudley has a population of 312,925 people and of those 4.1% describe themselves as Muslim. This number is higher now: according to the organisation Operation Black Vote (OBV) there are 9,545 BAME voters just in the Dudley North constituency. They could be enough to decide the outcome here.

Simon Woolley, OBV’s chairman, says Dudley North is one of the crucial seats where the outcome will help decide who has the keys to Downing Street. “And in this, like many other marginal seats, around 100, the black, Asian and minority ethnic vote will play a decisive role,” he said. “Never before in British politics has the BAME vote been so powerful.”

And the candidates know it. On 17 November the Tory hopeful, Marco Longhi, was pictured walking arm in arm with mosque elders during the Prophet Mohammed’s birthday. Saleem says Labour candidate Melanie Dudley was invited to the procession but did not attend. However, Dudley did contact the mosque on Friday to deliver leaflets. Her request was refused.

It would seem like a win-win situation for the Conservatives – but there are some who say Labour still have a chance of victory.

Khurshid Ahmed, the former chair of the DMA, who was at the forefront of the mosque battle, says despite the hurt and betrayal felt by the Muslim community they would not abandon the Labour party.

He said: “For a long time the community did become bewildered and disillusioned because they were solidly let down and were devastated. This was a disadvantaged community that was lacking the capacity to articulate its own needs and aspirations and local politicians became scared and succumbed to the far right.

“But those politicians were from both Labour and the Conservatives, and were equally hostile to the mosque idea and to blame for it never being built. The community has an appetite for change but that doesn’t mean bringing in a Tory MP – most will still vote Labour – and there is hope that everyone will learn lessons and things may improve.”

Over the years the town’s political landscape has been fairly unpredictable. The council was Labour but with a majority of just one and changed hands once again after the local elections in May. The political balance stood at 36 Labour and 36 Conservative. The mayor of Dudley, Conservative councillor David Stanley, had the casting vote and put his party in prime position.

During the EU referendum the constituency, in which the average age is 50, voted by 71.4% to leave, one of the highest margins in the country. In 2014 eight Ukip councillors were elected and the National Front and BNP, already a minor presence on the town’s streets, had started to become more visible. Some say that was a precursor to the referendum result.

This makes Dudley North one of the most marginal constituencies in the UK; its incumbent Labour MP Ian Austin, who backed Theresa May’s EU withdrawal deal despite voting remain, scraped to victory with just 22 votes in 2017.

But there have already been some shocks. In February this year Austin, MP for 14 years, abruptly announced he was standing down, telling the BBC that Jeremy Corbyn was “completely unfit” to be prime minister and that Labour voters should back Boris Johnson in next month’s election.

Then, with only minutes before nominations closed, Brexit party candidate Rupert Lowe announced he would no longer be running. He urged voters to support Longhi to not split the pro-leave vote.

Historically, Dudley was a prosperous place with thousands employed in its steel industry. But all that has gone and homelessness and poverty are top of the agenda. Figures released earlier this year show that almost half of children in the area are growing up in poverty. The End Child Poverty coalition, which released the figures, said more than a third (34%) of Dudley children were enduring impoverishment, ranking the borough among the 10 worst in the West Midlands.

Facebook Twitter Pinterest Stuart Homer and Loraine Stockton in their cafe, Gather. Photograph: Christopher Thomond/The Guardian

But there is hope. Time and again when the people of Dudley were asked what was positive about their town, after what became a fairly uncomfortable long pause, they would point towards Gather – an unassuming cafe in a rather drab shopping precinct.

Once inside it becomes immediately obvious why this place is winning the hearts of locals. It is a social enterprise and vital lifeline doubling up as a food bank, craft centre and general go-to hub for those on the margins of society.

Five years ago its owners Stuart Homer and Lorraine Stockton were contacted by a family in crisis two days before Christmas. The father was a student, the mother had chronic depression and they had been sanctioned for not going to a benefits appointment. With not enough food to last them over the holiday period and their electricity meter about to run out they could not afford to buy their three children any gifts.

As part of their charity work helping unpaid carers Homer and Stockton galvanised a local supermarket to donate food and others donated presents and the incident became a stark reminder that there were many others who would fall through the cracks that winter. Those that they would not be able to reach.

“It was a light bulb moment,” says Homer. “We went past the point of the sticking plaster strategy and decided that we needed to provide a space in Dudley that was open to all.”

Now both work in the cafe with volunteers and run a number of schemes to help the impoverished. A blackboard in the shop is full of names. Stockton tells me she has pages and pages yet to go up. This board – Pay it Forward – is a scheme where locals can pay for someone else to eat.

“Nobody takes the mickey out of the system, it’s people who are genuinely in need and we have got busier as the years have gone on,” says Stockton.

Both Stockton and Homer are aware that they are propping up a failing system, one that does not provide for those increasing numbers in the area who are in desperate need. Neither will reveal their political affiliations but both are open about the fact that there is an urgent need for change.

“There has been real movement on a grassroots level and people want things to improve because there has been so much negativity about the town over the past 20 years. We wanted to show people you can achieve a lot with some creativity. There is still so much in Dudley to be proud of but people need to be reminded of that,” adds Homer.