In the past seven days two stories have shown how little we know about race in modern Britain. First, the one that dominated the headlines: what Roy Hodgson said to the England team at half-time in their World Cup qualifier, and what he meant by using the word "monkey" to refer to a black player. Everyone, it seemed, had an opinion. Comment columnists and sports writers weighed in. Radio stations ran phone-ins. Arguments raged.

There's no doubt that, given the history of monkey chants at football grounds during the 70s and 80s, Hodgson was at the very least stupid and incredibly insensitive to use such a word in reference to one of his own team. It's right that he apologised. But what concerns me about this story is not whether Hodgson's a racist but the way it skews our understanding of how racism impacts on modern society. Media reporting is now all about what TV and sports personalities say, rather than what ordinary people do out there in the real world. And the discussions are always centred on whether the alleged target was justified in being "offended".

So the stories we hear are: should Carol Thatcher have called a black tennis player a "golliwog"; should England football captain John Terry have called an opponent a "fucking black cunt"; should Ron Atkinson have called a footballer a "fucking lazy thick nigger"? How should they be punished? Should they be sacked? And in too many cases the abusers quickly become martyrs, the new victims of a nation gripped by political correctness – you just can't say anything nowadays, can you?

And liberal commentators have caught on too. When talking about such issues as sexism, or gay rights, or disability, or antisemitism, their regular refrain is: "Imagine if they'd been talking about black people, just think of the fuss that would have been made."

And added to this, controversial comments by black personalities are seized upon as proof that there's an equal and opposite racism coming from the other side: Rio Ferdinand agreeing that Ashley Cole was a "choc-ice" – outrageous; Diane Abbott tweeting that "White people love playing 'divide & rule'" – the woman who campaigns against racism turns out to be just as racist herself.

In the endless coverage of all this, let's not pretend we're actually talking of racism; these are celebrity gaffe stories for the search-engine age. Though the issues are, for black people, irritating and possibly offensive, they are as relevant to racism as a cough is to tuberculosis. The fact you were once upset by a tweet doesn't mean you understand what racism's about.

The second race story last week addressed the real issues. An undercover BBC London investigation into lettings agents showed the massive extent to which black people are being denied homes – without them even knowing it. One reporter posed as a landlord and asked the agents not to show his rental property to African-Caribbean people. The agents – even the Asian ones – readily agreed. To test the agents' willingness, two other reporters, one black, one white, posed as prospective tenants. As the agents had promised, the black hopeful was told the property had gone; the white prospect was invited for a viewing.

Ten agents were found to be discriminating in this way; even more shocking, one of them said on (hidden) camera: "Ninety-nine per cent of my landlords don't want Afro-Caribbeans or any troublesome people." If true, and there's no reason to doubt his claim, it throws open all sorts of questions about how widespread discrimination is in modern Britain. This investigation, after all, took place in a city often held up as being at ease with difference. If it's happening in London, it can happen, and probably does, anywhere in Britain. And to cap it all, the black "rejects" would have no idea they were being discriminated against; they wouldn't have suspected a thing, unless this happened over and over again – in which case they'd risk being labelled paranoid for blaming it on their skin colour.

This is the true story of racism in the UK: how it is still so casual, and how it excludes and disenfranchises thousands. It's a story, though, which attracted minimal media attention. A news article in the Guardian and the Telegraph, but no interest from any other newspaper, and no battalions of columnists giving their opinions.

No celebrities involved; only black people affected. No story. As they say, if a tree falls in the forest and no one hears it, does it make a sound? If blatant racism occurs, and no one reports it, does it exist?

How things have changed. In 1988 a similar investigation took place by undercover BBC reporters in the city of Bristol. The documentary Black and White, broadcast over five successive weekdays, exposed discrimination in jobs, housing and the leisure industry and shocked the nation. Stunned viewers asked themselves how this could be happening in a country they believed to be so tolerant, and where discrimination was supposed to be illegal.

The documentary had a massive impact, opening people's minds to the fact that racism wasn't simply about National Front marches, skinhead thugs, abuse and violence. Geoff Small, the black undercover reporter, recalls: "For weeks afterwards, white people would approach me in the street and offer their sympathy and sincere apologies for what I'd gone through." Now, however, we look back on the 70s and 80s with a sneer. We believe we're so much more sophisticated now. Alf Garnett and the Black and White Minstrel show have gone; we've heard all about police racism through Stephen Lawrence; we see black faces presenting TV shows; many minorities have been successful in business; and we've even had black faces in the cabinet. For Christ's sake, even the President of the United States is black!

These steps, important though they are, are mainly about individual achievement; and the problem is that now, in this allegedly postracial era, the rose-tinted glasses of our diversity training are in fact blinding us to the reality for most black people – which has changed little in the intervening years. In case we need reminding, jobless rates are twice as high for black people as for whites, unemployment is a staggering 56% for young black men; black pupils are less likely than their white peers to get five good GCSEs and are routinely marked down by their teachers; they are three times more likely to be permanently excluded than the overall school population; stop and search rates are over five times higher for black people; and black people are five times more likely to be jailed.

But the story Britain likes to tell itself is that discrimination has gone away – that now we're used to black and brown people living among us, the barriers have come down. It was great that we celebrated Jessica Ennis and Mo Farah at London 2012, but the common conclusion, that this proved our nation was at ease with diversity, was pure delusion. As is the dominant view that the rise in interracial relationships and "beautiful" mixed-race babies shows that racism is on the way out.

The fact it persists undiminished is, it seems, too much for our society to take. And to keep the fantasy in place, we look for excuses: if there are more black jobless it's because of their poor attitude to work; if they perform badly at school it's their laziness and culture; if they're overrepresented in the criminal justice system then they're just naturally more aggressive. If black people could only sort out these self-inflicted problems themselves, everything would be OK. After all, doesn't every business say it welcomes job applicants from all backgrounds? The doors are open, why aren't you coming in?

Needless to say, this proves problematic for genuine anti-racists. It's the problem doctors used to have in "proving" that smoking causes cancer. You can't show it in individual cases, but when you track the increase in smoking and compare it to the increase in cancer rates, it becomes obvious. As for inequality, the general statistics show the racial disparities but individual cases are almost impossible to prove, allowing the deniers to claim that no barriers exist.

We certainly need the facts and figures, but what history shows is that most of us only react to this kind of thing when we see it with our own eyes. Would the abuse allegations at Abu Ghraib prison in Iraq ever have been believed were it not for the photographic proof? Statistics, unfortunately, can be manipulated and argued over endlessly. But seeing individuals act in a certain way is far more damning. What if, for instance, the BBC team had gone undercover with recruitment agents, or in the teachers' common room, or the judges' golf club? And what about the police? Ten years ago another undercover BBC investigation caused a storm after exposing the raw racism among trainee officers, most of whom used terms of racial abuse on a daily basis.

To investigate all these would, of course, be a significant undertaking; but we need to change the terms of the debate. If we are to achieve genuine equality, which most people say they want, then we need to show how racism can operate, indeed thrive, without a sign telling the world it exists. Without visible proof, we look set for years more discrimination, and years more denial.