They were unlikely scenes for a quiet corner of the English countryside more used to fetes and flower festivals as two extremes of the British political spectrum invaded a tiny Derbyshire village yesterday.

Down a lane lined with redbrick houses, fields and conifer–edged bungalows, about 1,500 protestors spent several hours scuffling with police, hurling the occasional missile and yelling anti-Nazi slogans as dozens of officers, with dogs, vans and horses, held them back. There were several arrests.

Over the thick hedge was the 10th annual gathering of the British National Party – the Red, White and Blue festival – underway this weekend on private farmland owned by a supporter between the villages of Denby and Codnor in the Amber Valley.

Inside the BNP site security men and women, all in black with trousers tucked into army boots, were stopping people leaving the site "for their own safety". But a temporary blockade of about 60 protesters stopping party members reaching the site from the other end of the lane was quickly broken up by police, and the promised "kettling", or herding, of the BNP by the Unite Against Fascism activists never happened.

"We have the full support of residents," said BNP official Simon Darby. "We are pleased by the turnout, it's up on last year, but with us winning the European election and a couple of county council seats we knew we'd see a few more here. But we are not here for trouble, it's a nice family festival."

But the reaction from locals was mixed. At one end of the Codnor Denby Lane the church of St James held hourly prayers for "peace and tolerance" and allowed the protesters to use their toilets, while other residents brought cups of tea to police and reporters. But shopkeepers were furious at the invasion, not of the BNP but of the protesters – a mix of anti-racist groups, union and Socialist Worker party members, or the "Trots, anarchists and troublemakers", as one woman put it.

Eric Madeley closed his watch and clock repairers yesterday, despite it being his busiest day. "In a selfish way the BNP are not bothering me but these protesters are. They are just so intimidating, especially those with scarves round their faces. It's very worrying.

"There is a lot of support for the BNP around here, you can tell when you talk to people. But we don't have any bother, we are quite ethnically mixed – the Chinese running the chip shop, Asians have the shop down the road and there's the Indian restaurant. There's no animosity towards them. At the end of the day the BNP is entitled to its view, not that I'd vote for them, but there is a lot of uncertainty in the country."

The chemist was open for business. "The BNP are a political party so they are entitled to their conference, same as anyone," said pharmacist Umesh Solanki. "People will be feeling resentful about immigration in a time of recession, same as any country. A small lawful protest is also fine but the numbers here are too high."

Local Labour MP Judy Mallaber said such a political gathering should not be being held in the village. "For three years we have argued this site is completely unsuitable and it causes extreme stress for the people who live and farm right next to it and it means people who wish to exercise their right to protest have to do so in narrow streets, which causes more upset to residents and shopkeepers. The BNP are seeking support, but they just show their contempt by going ahead without consideration for the effect on our community."

John Kimberley, of the Amber Valley campaign against racism and fascism, said: "We just wanted to be close enough for the people inside the BNP camp to hear us, to demoralise them and to make sure that they know that no matter how they might try to present themselves, we recognise they are fascists."

The noise of the police helicopter made that impossible and inside the BNP site many of the festival-goers were mostly unaware of the skirmishes outside. Builder David Weaver, 47, had brought his family up from Cornwall for the event. Sitting on the grass next to a faux cemetery of crosses commemorating those the BNP claimed were "white victims of black racism", he was having a picnic with his wife Penny and daughters Holly, 14, Beth, 12, and 10-year-old Jenny. "It's the first time we've come here. I've been a Labour voter but I've joined the BNP because I think those people outside are shortsighted, they have no vision of what Britain will be like in 20 years time," he said. "Where are all the feminist groups coming out against the burqa? I have three daughters and I tell them I'm fighting for them, so they have a future that isn't taken over by ranting Islamic clerics. I'm not racist, I don't hate anyone, but immigration has gone too far."

Inside the speaker's marquee the BNP leader, Nick Griffin, was on the podium delivering a speech attacking those who called the party racist. Outside, next to a stall selling Enoch Powell T-Shirts and golliwog mugs, Robert Walker, 47, and his 16-year-old son Daniel, from Huddersfield, were running a train set display.

"A British craft," he said, bemoaning that much of it was made in China. Walker complained that he had not been able to find a toy white conductor for the set.

Joe and Judith Osbourne are retired farmers whose fields edge the BNP site.

"Last year there was goose-stepping down this lane at 2am and shouts of 'Heil Hitler'," said Mrs Osbourne.

"We were horrified when this first started," said Mr Osbourne. "We see them as racist and divisive. They say they have the support of the local people, but when they offered free entry to the festival for residents only two people took them up on it so I don't know what support they think they have. Even the pub has a sign saying 'No BNP'. It is very upsetting to see them all in the next field."

But as the flags of St George and the Union Jacks fluttered across the BNP campsite, the party's treasurer John Walker said they were already planning next year's gathering: "This time we will get licensing for live music and make it a livelier event."