Back in the tail end of 2009, Nigel Farage stepped aside from his leadership of the United Kingdom Independence Party to concentrate on challenging the speaker of the House of Commons, John Bercow, in the parliamentary elections of 2010. In a characteristically forthright statement, Farage said that Bercow "represented the worst" of a legislature that had "broken the trust" of the British people. In due course Bercow, a somewhat maverick Tory, was returned to parliament and the speaker's throne, but not before Farage himself had been spectacularly unseated. It was during an election stunt, while he was flying his light aircraft high over the Angleterre profonde of Northants. The banner trailing the plane, and bearing the legend "Vote for Your Country, Vote for Ukip", created a little too much drag, and the habitually ebullient Farage fell to Earth.

Photographs from the crash site showed Farage disentangling himself from the wreckage and, with what was either astonishing insouciance or simply shock, walking away, his typically rubious face only a touch drained of colour. This Biggles-like tale would add lustre to any politician's image, yet for Farage and the party he subsequently returned to helm, the most telling detail is not that he escaped unscathed, but that it was blazoning Ukip's unashamed nationalism that had caused him to fall.

In the wake of Ukip's triumphant outing in the Eastleigh byelection last week – pushing the Conservatives into a humiliating third place – it's worth asking ourselves if Farage's party, now indisputably in the ascendant, can maintain its semi-benign cast, or, if the runner-up does indeed begin to run over the Tories, the mask will slip to reveal a more sinister visage. After all, Farage has for some years now been the MEP for the Southeast of England, which means that he sits in a democratically elected assembly the legitimacy of which he utterly denies. A benign view of this would be that he and the rest of Ukip are simply plucky small-nation nationalists standing up against an oppressive suzerainty; a darker perspective would be that some Ukip supporters have a more deep-seated antagonism to our current constitutional settlement, one they share with a quiescent sector of our society, that, when the time is right, will blossom not into a lovely English rose but a poisonous xenophobia.

In the spring of last year my then 10-year-old son and I set off from our home in Stockwell, South London, for our annual long-distance walk. On this occasion our destination was the manor house in Worcestershire belonging to friends. Our route took us past the tube station where a middle-aged white woman has a greengrocery stall. I often see her chatting happily with her customers – many of whom belong to ethnic minorities. During the 2010 general election I had conducted a vox pop for the Evening Standard in the area, and asked her how she would be voting: "Oh, BNP, I s'pose." When I had remonstrated with her, pointing out that, despite their disavowals, Nick Griffin's party remained racist and fascist to the core, she came back at me with the habitual plaint of the London white working class: "I'm not racial – but …" The "buts" in her case were those so carefully analysed by Daniel Trilling in Bloody Nasty People, his recent study of the rise of the British National Party. She was not opposed to black and brown Britons per se; what she objected to were recent immigrants of any colour scrounging social benefits, including healthcare and council housing. She despaired of any of the mainstream parties curtailing mass immigration, and looked to the BNP as the only credible promoter of what she saw as the rights of indigenous, white working people.

For a while after this conversation – although I had managed to avoid it becoming acrimonious – I bought my bananas elsewhere; but by the time we set off on our walk, the BNP – having failed to capitalise on those "buts" – had been trounced at the ballot box and was in disarray. And besides, while I regarded the Cockney coster's position as dangerously misguided, I couldn't gainsay her perception that successive governments had failed to curtail mass immigration, and, more importantly, failed to explain to her section of the electorate either why such immigration was desirable, or indeed if it really had had the negative socio-economic impact she believed. By patronising her in my mind, I was able to resume patronising her stall.

On our April 2012 walk my son and I tracked out from the inner-London neighbourhoods where multiculturalism is a demographic fact, and into the outer suburbs where a predominantly white proletariat and petit-bourgeoisie has proved – relatively speaking – receptive to the far right. We were heading west, through Hounslow, but it was in Bromley, an equivalent area of southeast London, that Matthew Collins received his political education in the 1980s. Collins, who ended up informing on his fascist comrades in the National Front and the BNP for the antifascist magazine Searchlight, describes with emetic detail this school of hard knocks – aimed at the black, the brown, and especially the red – in his memoir, Hate. Collins grew up in the tough estates between Lewisham and Catford, but his tale spins violent filaments across the city, tying together the far right's traditional face-stomping grounds in the East End with such leafier realms as Eltham, the suburb where, in 1993, the black teenager Stephen Lawrence was stabbed to death.

In the past decade some householders in these outwardly innocuous inter- and postwar cul-de-sacs have planted flagpoles beside the pampas grass or the rockery and raised the flag of St George. It was a phenomenon the late JG Ballard descanted on in his final novel, Kingdom Come. I remember going to see him shortly before its publication at his home in Shepperton, on the western outskirts of London, and him pointing out to me the row of red-and-white pennants limping along his own somnolent road. For Ballard, this was a faux English nationalism; one signed up to by those whose mass-mediatised ennui was ever on the point of exploding into violent anomie. Trilling and Collins, by contrast, offer a different view of the flag-hoisters, and in particular of the extreme rightwing English Defence League. Espousing the pirated language of identity politics, the EDL poses as oppositional only to politically extreme Islamism, while welcoming supporters of all ethnicities and sexual orientations. But really, it's by their wolfish penchant for street-fighting that we should know them: the EDL phenomenon is, Trilling and Collins suggest, the dressing up of old bigotries in an off-the-peg liberal fleece.

But a flag can mean different things to different people. I noticed plenty more St Georges as my son and I headed west to Marlow on the Thames, and then struck northwest over the Chiltern Hills to Oxford. However, out here in the sticks, there were just as many Union flags in front gardens. To the far right the "Union Jack" is tainted by association with what they refer to as "state"; by which they mean both the discredited parliamentary system (riddled with Jewish influence, emasculated by effete liberal cosmopolitanism), and its violence-monopolising organs – the police, MI5 – which have been charged with containing their own savagery. The Union flag complex of Northern Ireland's Loyalists has so often been their casus belli – and perhaps never more so than during the recent riots attendant on the newly restricted flying of the flag from Belfast's city hall. Collins writes about the linkages forged between the Ulster Defence Association and the mainland far right in the early 1990s. For the emergent BNP, and especially the terroristic Combat 18 (the "18" was a cipher of Adolf Hitler's initials), the Loyalist paramilitaries possessed the elan born of taking the fight to state.

In the past it was often observed that the proliferation of Union flags in Ulster was alienating to other Britons – but the same might well be said about the new crop of flags sprouting in the gardens of Middle England. It's difficult not to conclude that their growth has coincided with the move out from the urban centres of ethnic minority Britons, and it is in these black and brown faces that the red, white and blue is being waved. There's this, and there's also the flag-caparisoned business of war: the prosecution of the "War on Terror" since 2001 has both manufactured a readymade British Muslim enemy within, and imposed a plangent cognitive dissonance on Tory patriots, whose own unquestioning loyalty to the core institutions of state – the monarchy, the armed forces – has been tested to its limits by what amount to a series of failed and quixotic ventures mounted nominally in support of liberal values – women's education, human rights, democracy – but actually to sustain US hegemony.

Under such circumstances symbolism becomes the only recourse: the comfort blanket of the flag. Our walk took us towards RAF Brize Norton, the biggest air force base in Britain and the home of the lumbering transports dispatched to points east. The EDL has been known to pitch up at Wootton Bassett (now styled "Royal" as an anodyne for confused patriots), to take part in the gloomy ceremonials attending the repatriation of British war dead to RAF Lyneham, but Brize Norton maintains a lower profile as a bastion of the Anglo-American alliance, and a purveyor of privatised services to the jets of the Murdoch family.

Reaching nearby Witney, the town at the centre of David Cameron's parliamentary constituency, we found a mutant, cloned centre, all chain stores and charity shops. Once at the heart of a carpet-making industry (the wool was treated with the urine of drinkers in the local pubs), Witney is now palpably down on its luck: another dormitory town for keyboard-rifflers, its local credentials a function of subsidised agriculture and Barbour purveying by appointment to the likes of Jeremy Clarkson and the Chipping Norton set.

Here seems the right place to consider the core thesis of Trilling's Bloody Nasty People. The author correctly identifies the willingness on the part of the last Labour government to implicitly play the race card, as a major factor in the rise of the BNP. Throughout the last decade the party steadily gained council seats in its heartlands of the Northwest (Blackburn and Burnley), and outer East London (Barking and Dagenham), up until the point of their electoral zenith, 2009, when Griffin was elected to Brussels as an MEP for the Northwest, and his deputy Andrew Brons for Yorkshire and Humber. Labour had triangulated on immigration – and so, of course, did many Tories. Indeed, the willingness of mainstream politicians to subsume all incomers to the catch-all "asylum seekers", instead of attempting to first understand – and then explain – the complex interplay of history and economics that had led to mass immigration, left the stage free for anyone prepared to argue simply for its cessation, and even for repatriation.

Come the hour, comes the man. But Griffin, a long-time advocate of "racial purity", whose stated political aim had always been to return Britain to its ethnic composition (whatever that might be) before the docking of the Windrush, was always an unlikely messiah. Despite learning from the French National Front how to massage his innate fascism into a marketable nationalism, he is no barnstorming Jean-Marie Le Pen; indeed, it was arguably his weaselly performance on a carefully stage-managed edition of the BBC's Question Time in October 2009 that marked the start of the BNP's decline.

Trilling's is an indispensable guide to the tortuous convolutions of contemporary far-right British politics – the fissions and fusions, the phlegm and the fists – but his approach to the innately shape-shifting character of British fascism is too fixed to be altogether useful. For Trilling – as for the turncoat Collins – the most important thing to know about the far right is how to counter it. Fair enough. But by concentrating on a traditional socialist analysis – whereby all cultural conflicts are translatable into economic ones – he risks the very reductionism of his opponents, who take the diametrical view.

There's this, and there's also his focus on the BNP activists themselves. Political activists of all stripes are usually a wacky bunch, and never more so than in a system like Britain's, where power is effected via the quiescence of the electorate as much as its convictions. For Trilling and Collins, fascism is a sarcoma on the body politic that cannot be allowed to metastasise – what's required is the radiotherapy of reason; but this ignores the possibility that fascism is an unavoidable sequel to the existing British constitutional settlement; that exactly like cancer itself, the British forms of fascism are diseases of senescence – if a regime lasts long enough, they're bound to appear.

From Witney we headed in heavy rain northwest into the Cotswolds. A day's walk away is the village of Temple Guiting in Gloucestershire, where, in the summer of 1962, John Tyndall, the unapologetically fascist founder of the BNP convened a meeting of international neo-Nazis, including the leader of the American Nazi Party, George Lincoln Rockwell. The Cotswold Declaration that emerged from this bizarre assembly sought to establish a World Union of National Socialists.

They were in the right neck of the woods: our route took us along the valley of the oddly homonymous River Windrush to the village of Swinbrook, site of the family home of the fascist David Bertram Ogilvy Freeman-Mitford, 2nd Baron Redesdale. Stopping at the local gastro-pub for ginger beers and sea salt and balsamic vinegar crisps, we luxuriated in the cleared skies, the evening sun on dry-stone walls, pristine greensward and bleating sheep – all the accoutrements of an echt and immemorial England. The thematics continued inside the pub, which was given over as a shrine to the Mitford sisters. Enlarged photos hung above the rag-rolled wainscoting, and a bookcase sported their published works. Further blow-ups featured Edwardian and interwar shooting parties – presumably some of the guns in the latter belonging to the Right Club, of which Redesdale was a leading and erratic light.

I was tempted to ask the diners whether they were sensible to the ideological queasiness of this bucolical confection, but I didn't want to put them off their bruschetta. Instead, the boy and I strolled on to the church graveyard, where lay the remains of the Redesdales and a brace of their Nazi offspring – Diana Mosley (nee Mitford), and Unity Valkyrie Mitford.

When I say fascism is endemic to the British body politic, I use the term as loosely as it was contemporaneously understood. Martin Pugh's revisionist account of interwar British fascism, Hurrah for the Blackshirts! (a reference to the Daily Mail's infamous January 1934 banner headline), recontextualises the far right, locating its apex further up the social scale and shifting its locus away from the urban and metropolitan to the rural and provincial. This is not to say that Pugh neglects the proletarian aspects of the British Union of Fascists – on the contrary – but the extent and influence of aristocratic fascists at this time cannot be overstated. As Pugh drily observes, not only were there all the usual appeasement suspects, but Churchill's own opposition to fascism was "non-ideological".

The fascist groupuscules that emerged after the first world war were initially of an extreme reactionary and antisemitic cast, but they also had taproots deep in the English countryside. Oswald Mosley, whose fascist conversion was godfathered by Il Duce (and his party financed by him), paid much attention to farmers, and just as his attacks on international finance capital would strike a chord with today's anti-globalisation protesters, so his emphasis on protectionism and economic autarky would play well in the shires. Even the antisemitism of the BUF – as detailed by Pugh – emerges only in the late 1930s as a tactical measure and as a byproduct of anti-communism, rather than being intrinsic to Mosley's own political vision. This is not to suggest that Mosley wasn't an antisemite, only that his antisemitism – like the racism of the British right today – was an ambient noise rather than a fanfare.

Trilling, keen to excite our vigilance, points out that Mosley's BUF, even during its heyday in the mid-1930s, never enjoyed the electoral success of Griffin's BNP. But this may well have been an accident of timing: the BUF was unable to effectively contest the snap election of 1935. By contrast, it's worth recognising that, even at its peak, the BNP has never been able to mount the sort of mass rallies that Mosley pulled off, let alone take the battle to the streets in the way that the BUF did in the East End of London in the run-up to the second world war.

No, looking back to the interwar period, and mapping the events of the crisis year of 1931 on to our own recent past, a sense of profound deja vu emerges: the same widespread impression of national destiny being blown off course by powerful currents of transnational capital; the general conviction that the political class is bankrupt; the perennial anxieties about imperialist "scuttling' (for the Indian National Congress and the League of Nations, read al-Qaida and Brussels bureaucracy); and naturally, the same search for syphilitic spirochetes poisoning the ineffable virility of the British character.

My son and I had two more beautiful days tramping through the heart of England, and then we found ourselves among friends; friends whose own family history has been shaped by fascism – although not the usual, socially acceptable victimology. My host's father grew up among adults who greeted each other in the street by saying "PJ" ("Perish Judah"); but it was only when, aged 14, he was removed from his class at Dartmouth Naval Academy by military police, that he became aware of the full extent of his parents' fascism.

It was 1940, and they had been imprisoned along with the Mosleys and 747 others. (Although, as Pugh points out, the internments under Regulation 18B were both arbitrary – all the most aristocratic fascists remained at liberty – and motivated more by professional closure on the part of MI5, than any perception of the BUF acting as an effective fifth column for the Nazis.) My friend is about as far removed from British fascism as it's possible to imagine, and he finds the anti-EU posturing to the right of the Tory Party – his natural home – anathema.

It would probably provide little succour to Stuart Lawrence, Stephen's brother, to try thinking of the racism directed at him by the Metropolitan Police as simply another manifestation of a shape-shifting and endemic British fascism. As reported in the Evening Standard on 22 January, Stuart Lawrence received racially motivated hate mail two days after his lawyer, Imran Khan, had filed a letter of complaint to the Met on his behalf. The letter detailed 25 separate stop-and-searches conducted by the police on Lawrence – a teacher, who has not been so much as arrested, let alone convicted of any crime.

Just as the BUF waned in the late 1930s, so the BNP has been in decline for the past couple of years, despite the political climate seeming more propitious than ever for far right germination. But then British fascism, I would contend, has never been simply about skinheads sporting swastikas: there remains a sector of our society that still believes parliamentary democracy to be a sham; still thinks that black and brown people are inferior (while Jews are worrisomely and magically superior); remains powerfully xenophobic and looks to a nationalist renaissance; and of course, still reads the Daily Mail.

It may not be the case that Ukip occupies the same Cotswolds of the psyche as these indigenous fascists, but their territory certainly borders it, and just as the Tories may fear their own diaspora, Nigel Farage has to worry about immigration from the far right.

• This article was amended on 13 March 2013. The original referred to Dartford Naval Academy rather than Dartmouth Naval Academy. This has been corrected.