With the publication this week of my draft Bill I return to the subject of Data Communications.

No-one will be more aware than readers of Computer Weekly that communications technologies and services are changing remarkably fast nowadays, with the advent of mobile telephones, email, Facebook and Twitter.

In the past people might write each other letters and ring each other up. But nowadays, in our fast-changing, increasingly dangerous and uncertain world, people use the internet to communicate in all sorts of ways. Regrettably these developments have not been lost on criminals, notably terrorists and paedophiles.

If the police and intelligence agencies are to continue to do the difficult and dangerous work we ask of them we must give them the proper tools to do the job. This means not just batons. It means full access to all communications data about everyone in the UK: who speaks to whom, what websites each of you have visited, where your phone was at all times and so forth.

We shall require this data from any entity operating a telecommunications system.

People ask me what we mean by telecommunications system nowadays; the simple answer is a system (including the apparatus comprised in it) that exists for the purpose of facilitating the transmission of communications by any means involving the use of electrical or electro-magnetic energy. Facebook is included; Boy Scout semaphore is not.

This is a necessary and proportionate step if we are to continue to keep the streets of Britain safe.

Of course no-one is more concerned with privacy and with human rights than I. Any concerns people raise can easily be dismissed as the self-righteous outpourings of conspiracy theorists who literally want terrorists and paedophiles to walk the streets of Britain freely.

People complained about our previous plans to create a giant government database of retained communications data. This new plan is entirely different, with distributed databases operated for government by service providers, equipped with high-speed connections to my own desk in 70 Whitehall. As I have said, this is merely the upgrading of a capability we have always had and are in danger of losing.

Let me give an idea of the scale and scope of the problem. Last year there were 540,000 communications interceptions. These related only to the most serious types of crimes.

People ask me whether this will work with https secure sockets. Constrained as I am when talking about matters of intelligence, I merely say yes and smile. We do not discuss Sigint for obvious security reasons. But of course we have https cracked; CESG pretty much invented the whole encryption game.

Routine storage of all https comms data has the additional benefit that the criminal world's (and indeed everyone else's) banking and other secure online transactions become an open book to us. This data will be carefully retained, mined, and used only in the most serious circumstances such as criminal conspiracy.

People have sought to express concerns about costs.

To this I say: this is the British government you are talking about. We have commissioned and deployed some of the largest systems in the world. No-one does major IT projects like us.

Secondly I shall not purchase some great supercomputer to do this. The telephone and internet service providers will merely buy their own supercomputers, and I shall course cover their reasonable costs on presentation of credible invoices.

Our projections show this investment in British business will bring a seven-fold return, making a significant contribution to HMG's Growth Agenda.

We have developed strong relationships with the world-leading Chinese manufacturers of such “black boxes” by virtue of our far-sighted “revolving door” staff outplacement strategy. Thanks to this new special relationship, they have signalled a readiness to offer the UK (in the light of our prevailing economic circumstances) some really significant savings on their equipment. I have seen in demonstrated. It performs most impressively. Similar systems been extensively tested on some of the world's largest and most recalcitrant populations.

Those with nothing to hide will have nothing to fear from this simple upgrade of our communications interception capabilities which strike, in my view, a sensible balance between liberty and security. There will be an elaborate paraphernalia of oversight procedures, reporting through the independent Information Commissioner to me. The independent Interception of Communications Commissioner (IOCC) will continue his admirable series of annual reports to ensure full transparency of the process. I might point out, as a measure of how well the existing system works, that not once in 27 years of interception oversight has the IOCC ever had cause to report an incident of abuse.

Unless we act immediately to implement this eminently proportionate and reasonable legislation lives will be lost. Crimes enabled by email and the internet will go undetected and unpunished. The vulnerable will not be protected. Terrorists and criminals will not be caught and prosecuted.

No responsible government could allow such a situation to develop unaddressed.

Sir Bonar Neville-Kingdom is Data-sharing Czar in Her Majesty's Government. You can follow his daily thoughts on Twitter at @sirbonar.

Photo credit: paulclarke.com