« Inbound maintenance! | Main | What have the Romans done for us ...? »

False Positives and the Database State

There is, in the UK (as elsewhere) a prevailing climate of paranoia about adults interacting with children.

In an attempt to be seen to Do Something, in the wake of a particularly gruesome multiple murder, the British government established a new agency, the Independent Safeguarding Authority, "to help prevent unsuitable people from working with children and vulnerable adults." Working with the Criminal Records Bureau, the ISA "will assess every person who wants to work or volunteer with vulnerable people. Potential employees and volunteers will need to apply to register with the ISA." For a fee of £64 you apply to the ISA for a background check. They then certify that you're not an evil paedophile and a threat to society, and issue you with a piece of paper that says you're allowed to interact with children in a specific role. Want multiple roles — driving kids to school in your taxi, and teaching them karate in the evening? — get multiple certificates.

Authors need to get a certificate before they can visit schools to deliver readings. MPs need a background check, it seems, before they can visit schools. (Usually the employer is responsible for getting the certificate; hilarity ensues when it transpires that MPs aren't actually employed by Parliament ...)

As you can imagine, the authors are upset. As Philip Pullman puts it, "It seems to be fuelled by the same combination of prurience, sexual fear and cold political calculation," the author of the bestselling His Dark Materials trilogy said today. "When you go into a school as an author or an illustrator you talk to a class at a time or else to the whole school. How on earth — how on earth — how in the world is anybody going to rape or assault a child in those circumstances? It's preposterous."

He's completely right, in my opinion. But the situation is worse than he imagines. I'm not going to apply for a CRB check &mdash ever. And not because I'm a criminal. (My sum total of negative interaction with the law over the past 44 years has amounted to two speeding tickets, most recently six years ago.)

Nor am I outraged at the privacy thing. (I'm used to the idea that we live in a panopticon.)

What I'm worried about is the problem of false positives.

Even the simplest of databases have been found to contain error rates of 10%. (The HMRC database in this study contains merely first, second and surname, title, sex, data of birth, address and National Insurance number — nevertheless 10% of the records contain errors.) Other agencies are even more prone to mistakes. For example: my wife recently discovered that our GP's medical records showed her as having been born outside the UK rather than in an NHS hospital in Manchester. We don't know why that error's in the system, and we've got the birth certificate and witnesses to prove that it is an error, but imagine the fun that might ensue if the control freaks in Whitehall decided to enforce record sharing between the NHS and the Immigration Agency ...! (Hopefully they're not that stupid, but who can tell?)

The point is, if 10% of government database records contain an error, than the probability of a sweep of databases coming up with an error rises as you consult more sources. And there are a whole bundle of wonderful ways for errors to show up. If your name and date of birth are the same as someone with heavy criminal record, a CRB check could label you as a bad guy. If your social security number is one digit transposition away from $BAD_GUY, see above. If the previous owner of your house was a child abuser, see above. If your street address is one letter/digit away from a street address occupied by a criminal and some bored clerk mis-typed it, you can end up being conflated with somebody else. And the more sources the CRB checks, the higher the probability of a false positive result — that is, of them obtaining a positive result (subject is a criminal) when in fact the subject is a negative.

This is not a hypothetical worry. As of last November, the CRB had falsely identified more than 12,000 people as criminals, according to the Home Office. (Raw parliamentary answer here.) These are the disputes that were upheld, that is, ones where the falsely mis-identified were able to convince the CRB that their record was incorrect. These are false positives which have been conclusively identified as such. While the identified false positive rate is around 0.1%, the true figure is certainly much higher: because there will be a proportion of individuals identified as false positives who are in the unfortunate position of lacking the documentation to prove their innocence.

I expect the ISA will be returning many false positives, because they're looking in multiple places for evidence of misbehaviour, and the more places they look in, the more likely they are to stumble across corrupt database records that are superficially incriminating. The harder they look for evidence of misdeeds, the likelier they are to find them (even if no such misdeeds took place).

I'm not going near that thing with a barge-pole. The nature of the precautionary bureaucracy we're establishing in the UK is such that flags raised by the ISA will almost inevitably be propagated elsewhere through the police and social security system, sooner or later. I'm probably as safe as ISA background check applicant can be, because I've got a unique name, no criminal record (beyond the aforementioned speeding tickets), and the previous owners of everywhere I've lived in the past 20 years have been pillars of respectability. However, even an 0.1% chance of being branded as Evil™ is too damn high, because the personal cost if you fail an ISA check is potentially enormous going forward. I assume that in the near future, failing an ISA check will itself be something that people are required to disclose on job applications — not to mention ending up in current police intelligence databases. To put it in perspective, that 0.1% probability of being on the receiving end of a false positive is of the same order as the risk of being seriously injured in a road traffic accident at some time in one's life.

So I won't be doing any readings in schools, or work with youth groups, in the forseeable future. Sorry — but it's too dangerous.

| Permalink