Some days in Britain, in London - when it's raining and four out of seven tube lines are closed because of suicides on the tracks, the all-day breakfast is chilled close to absolute zero like some quantum physics experiment, you accidentally read part of the Sun, and at work you find that your new boss is the daughter of a baronet your old boss went to Balliol with, and it's only 10am - then you start to make that list, the reasons why you stay. Mine has the usual stuff, the pubs, clubs, postcard villages are the small things, and then a bunch of quirky things, the News Quiz, BBC Radio 4's decades-running satirical panel show, a sharper version of Good News Week. On the News Quiz the comics are sharp, the commentators are funny, and the jokes are on the money. I love it like a mistress and like such it will always, always slap me in the face. For as soon as there is any mention of the A-word - Australians - I know what's coming.

“Australian scientists discovered this week” the show's host, Sandi Toksvig, trills in her impeccable public (ie private) school accent. “That's amazing! I had no idea there were Australian scientists!” I have been listening to this show for 15 years, off and on. I have heard that joke at least five times.

Nor is it unique to the Beeb. The “gormless Australian” stereotype runs through great swaths of British culture, from the BBC to standup comedy, to the print media. Even the Guardian is, shock horror, not immune to it. The attitude is something of a default setting in the UK. Visiting a small seaside town in Cornwall, I was introduced to three friends of friends, and garnered three convict jokes in the space of an hour or two. Everyone everywhere makes jokes about everyone else, but the “gormless Australian” cliche has a deeper intent because it is usually joined to a more serious charge: that the country is unspeakably sexist and racist, and that such racism licenses an almost continuous stream of mockery. After a recent visit down under, the Daily Show’s John Oliver dubbed us “the most casually racist country I've ever been in”, though he has presumably spent at least some time in Mississippi or West Virginia. With these remarks comes an almost palpable pleasure, akin to making jokes about the Germans. Look what they did! How can they have any possible comeback? Just as every Basil-Fawlty style goosestep is a tiny Holocaust memorial, every joke about dumb soap operas or sheilas is a raised fist, sticking it to the man, by proxy.

It's possible to over-react to it – still feeling the colonial relationship a bit too keenly – and, though it can be wearying, it is difficult to take exception to much of it. This is, after all, a country with great swaths of very ugly racism running through it - and many of the people pointing it out to the Brits are Australians themselves, such as John Pilger and Germaine Greer. Faced with the British assertion that Australia is a sexist place, you could point to things like our vastly better childcare system or the boys' club of the British establishment – but it all wilts before the spectacle of a country assailing its prime minister with the grossest misogyny, a process well out of control. And many of the young Aussies who hit the UK for a two-year work visa tour of duty are a bit rambunctious - perhaps more so than they are at home - and Australian brands such as Foster’s play up to the image to sell product. Even the brigade of Aussies who seem to run the UK's legal firms and arts organisations can on occasion be seen Jager-bombing and mooning antipodean-curious Finns and Spaniards at the SheBu Walkie (the Shepherd's Bush Walkabout pub). It's no worse, indeed a lot better, than Newcastle upon Tyne on a Saturday night, but what to do? The attitude could hardly be called racism itself, though wounded types will sometimes try that on. That is simply an over-reaction inviting a fresh round of derision. Brits know that, and they enjoy watching the dilemma.

Indeed the phenomenon would barely be worth remarking on at all, had it anything much to do with Australia. But most of these jokes come squarely from the left-liberal camp, and they have little to do with the continent-state in the southern hemisphere. What they're really talking about are uncouth, assertive, rambunctious people closer to home, the British white working class. Australia is no classless society, as every Australian knows - save, perhaps, the Australian - but there is nothing like the complex matrix of class manners, assumptions and behaviour which structures every aspect of British life. Every gesture, every interaction hangs within it. Language, accent, space and taste are all sliced and diced by expectations of what could and should be and should not be done.

For a century, until the 1970s, the broad working class, and the middle upper classes were locked in conflict over who would control social and economic life. Yet the shared idea was that it would be, in some form, collective, the right arguing for crown and country, the left arguing for socialism. Prosperity, after the Labour victory of 1945, was seen not in American terms, as houses and cars, but as better council flats, secure working-class jobs, holidays at Butlin’s and the like.

Margaret Thatcher's 1979 victory swept all that away, but not in the way she imagined. Thatcher hoped that privatisation and union busting would reintroduce “Victorian values”, a people who would be dynamic and entrepreneurial at work, continent and conservative at home. Didn't work out that way. A lot of people got rich, most didn't, but they all developed a taste for tabloids, celebrity, rave music, cheap holidays, bad TV and party drugs. The result horrified rightwing moralists, but it gave no cheer to the liberal left either. The working class had taken a deal - they would achieve a sort of cultural dominance in exchange for relinquishing a claim on real economic power. It was impossible for many on the liberal left to not feel that their elite culture that had once been dominant - that of the BBC, the quality press, high culture, “authentic” Britishness, anti-Americanism - had been sidelined by a ravening street party.

One of the best pictures of this was contained in Martin Amis's London Fields, in which the exuberant, thuggish, porn-addicted crim Keith Talent, stands as a self-parodic rendering of the secret fear of the liberal middle-classes who, pre-Thatcher, imagined they would inherit cultural power, only to find that the new society was indifferent to their values. London Fields expressed the idea that some sort of catastrophe had occurred in British life (Lionel Asbo meanwhile, suggests some such has occurred in Martin Amis's). But with the distancing of fiction unsuitable for everyday use, another device is required. The right have no qualms about sneering - check out any given issue of their house magazines; the most racist article in Australia in any given week is usually the Taki column in the local edition of the Spectator - but the liberal-left intelligentsia need a device to pillory a mass whose tastes and politics have so disappointed them. Thus the fear and hatred of a rambunctious, consumerist, perhaps superficial and certainly undeferential mass is transferred from proles to Aussies. Many find Australia sexist because they encounter here standard suburban gender relations they are shielded from in the UK; and they ignore how backward the UK is with regard to things such as subsidised childcare, senior public service gender ratios, and the like.

But it is above all the racism charge that legitimises it. British concern about the Aboriginal health gap or mandatory detention does not, amazingly enough, run that deep. Were it so there would be more of an encounter with recent British racism - such as the lethal concentration camps the British ran in Kenya during the 1950s, around the same time as the coronation whose 60th anniversary is being lavishly recalled, as I write. There would also be an acknowledgement of the different forms racism takes in different societies. Settler societies have a certain type of visible racism; bound societies such as Britain confine racism to seething pockets within the class matrix. John Oliver saw Australian racism in Sydney's west, with white yoof talking about “Lebos” in a variety of ways. Had he done the same investigation in south London he might have found less overt name-calling but the same or greater mix of suspicion enmity and conflict. Raised in Bedford (black population 3%), Cambridge-educated, it might be as foreign an expedition for him as his Australian one. Cosseted by class barriers, left-liberal Brits often experience less of their own society than they do of others, through a media determined to set the world to rights.

One doesn't hold out much hope that the convict gags will cease. But nor will the whining-continues-after-the-engines-are-shut-off-on-BA-flights whingeing pom jokes. Nevertheless, it is not impossible that the British liberal establishment will one day reflect upon the deeper purpose of knee-jerk anti-Australianism, those interests it serves, and those it hides.