If ever there was an open goal for big data, healthcare should be it.

Read this NHS patient record database project pushed back A scheme that would collect the health records of nearly every patient in England has been delayed for six months Read More

By gathering information from doctors, patients, drug companies, insurers, and charities, and putting the big data machinery to work on analysing it, we should be able to get better insights into a range of conditions and then come up with better ways to treat them.

The NHS, one of the largest public health organisations in the world, understands this well. In a few months' time, it is due to begin extracting and sharing confidential data from NHS patients' medical records with other healthcare bodies, as well as private companies, under the banner of "improving patient care".

The scheme, called Care.data, will see patients' data shared by their GPs with the HSCIC (Health and Social Care Information Centre). The HSCIC can then share those details, along with patients' hospital records, with businesses, as long as those companies compensate them financially. Whether this constitutes 'selling patients details' is an ongoing semantic battle between Care.data's supporters and opponents.

The data does not come with a patients' name attached, per se — according to HSCIC, Care.data will not release "person-identifiable data", and stats will be provided in "aggregated or pseudonymous forms only". In future, however, it is likely to include identifiers that allow organisations to track an (unidentified) individual moving through the healthcare system – to link person A who went to their doctor with complaint B with the person that later turns up in A&E with a related complaint — or let person-identifiable data be released with their consent.

The data will be put to a variety of uses, HSCIC says – showing patients which healthcare bodies have the best performance in certain fields, improving NHS' predictive modelling, identifying healthcare trends, or measuring which treatments work best in which circumstances. Such an enormous repository of information should theoretically help identify patterns in disease and treatment that may not have come to light before, and therefore help improve the health of the nation.

Yet, not everyone's in favour of the move. According to those opposed to the new initiative, Care.data could give police a shortcut to access individuals' medical records and, because it is pseudonymised (rather than anonymised) it could still be reverse engineered to link medical histories to the individuals they come from. Others object to the half-way house of allowing private companies to access the (pseudonymised) data at a cost — either that they are accessing the data at all, that they are paying do so, or they are not paying enough for the privilege.

While I am convinced there is merit to sharing the NHS' data stores, I have already opted out of the scheme.

My opting out of Care.data will mean little to the NHS. Two years ago, my medical notes were seemingly lost – fact that no NHS organisation noticed until I went hunting for them. Decades of notes got lost in the system, and none of the half-hearted attempts, made over several months and at my repeated request, to rediscover them have yet resulted in them being sent to my current GP.

'Government IT project' has long been another name for 'disastrous IT car crash'. Despite a few successes, the National Programme for IT went billions over budget and years over deadline, with the most catastrophic part of the project being the plan to provide a basic digital patient record for English and Welsh citizens, known as a Summary Care Record .

The NHS is trapped between its paper past and its digital future, a rock and a hard place it can't get out of. UK patients' notes have traditionally been kept on paper, which has the advantage of being lossless and less leaky than its digital counterpart. However, as my own and many others' experience shows, the NHS' addiction of paper has all the downsides of the format with none of its benefits. Before the NHS could ever hope to make the best use of the scads of data it has, it must render it all into digital form, securely, and easily accessible. The electronic patient record fiasco showed how difficult such a task would be, and that involved a small section of the data the NHS holds on its patients.

Despite repeated promises that government really has learned lessons from the tech foul-ups of its past, the progress of Universal Credit and a delay to the implementation of Care.data, announced today , suggest otherwise.

Initially scheduled to begin operation in April, the database that will underpin Care.data will now go live in the autumn. The delay was ostensibly introduced because NHS England was persuaded by the complaints of the British Medical Association and others that not enough had been done to educate the public about the scheme and its implications.

Care.data and the furore that surrounds it is almost a direct replay of what happened with Summary Care Records: privacy concerns, a roundly-ignored NHS leafleting campaign, public confusion about opting out, implementation delays.

NHS England will now spend the extra time before Care.data's launch promoting the scheme to the public. "We have been told very clearly that patients need more time to learn about the benefits of sharing information and their right to object to their information being shared," NHS England national director for patients and information Tim Kelsey said in a statement.

The NHS would do well to educate users about the risks too: the benefits are obvious to most people, as are the downsides. The NHS, like most government bodies, has a history of data breaches and the fines that go with them . While leaving a USB stick with sensitive patient information on the train is unforgiveable, the potential for harm to individuals from a breach of such a large, centralised database is mind-boggling. Reverse engineering a link between an individual and a pseudonymised medical history could be of benefit to everyone from security agencies to insurance companies, and therefore hugely damaging to all of us.

And what of our right to change our minds? This too should be considered. Under the current scheme, if, having shared your data, you subsequently decide you'd prefer not to be included in the database, your record can't be removed. Data-sharing should not be such a binary proposition: the data defines an individual, it should be up to them to decide whether, when or where it's included in Care.data or not.

These problems can be circumvented, but they must be dealt with, publically and soberly, if the NHS really does want to win public confidence. The NHS should approach selling the scheme to the public as if was opt-in, not opt-out, then work to convince us to join it. Tell us how sharing our data can help, but tell us what risk too.

Let us decide if that balance is worth it. If it's found wanting, the NHS must go back to the drawing board and retool the scheme until it is. It's just too important to get wrong.