More and more personal, private information is being used and stored online than ever before, and at the same time, attacks on that information are increasing in frequency and sophistication. Phishing is a growth industry—it's very profitable to trick people into handing over names, passwords, credit card numbers, and so on, so that their finances can be pillaged. Important activities like banking and filing tax returns are being performed, and these need strong proof of identity. On the other hand, there's no reason why a storefront like, say, iTunes, needs to know your identity; it only needs to know that the money being handed over is yours to hand over.

Ultimately, we want to be able to securely make transactions without giving third parties the ability to masquerade as us; we want to be able to visit websites and make purchases without those sites being able to track us or combine different pieces of information to draw a more complete picture of us; we want to be able to be able to disclose some information about ourselves, but not everything. The U-Prove framework, released as a CTP today by Microsoft, aims to solve these problems.

With current systems, there's no good way to prove your identity to those sites that need verification, and conversely, there's also no good way to restrict what you inadvertently reveal to those sites that don't need your identity. To use a credit card on iTunes, I have to hand over so much information that Apple, if it was a bad actor, could masquerade as me. I can't just give Apple some electronic money; instead, I have to give them my name, address, and credit card number. In practice, the real problem with me handing over so much info to iTunes isn't that Apple might pretend to be me—with billions in the bank the company doesn't really need to charge things to my credit card, after all—but that hackers (both external and internal) might take this stored data and use it for their own nefarious purposes.

And these are not hypothetical concerns. An estimated 222 million records were exposed in 2009 in a wide range of data breaches. Many occurred in banking and government systems (systems which arguably do need to retain a significant amount of personal data), but more than half were disclosures by businesses, many of whom only retain private information because current technology gives them no choice. If convenience features like automatic "click once"-style payments are to be offered by these companies, they have to retain information about my name, address, credit cards, and so on.

The U-Prove system is designed to be a solution to this problem. It was put together by respected cryptography researcher Dr Stefan Brands. He created a company to develop and market U-Prove, Credentica, which was bought by Microsoft in March 2008. With U-Prove, identity information can be used securely, and private data can be safely shared to those parties that need it, without leaking more information than is required.

U-Prove allows the creation of secure ID tokens, which are pieces of data that incorporate whatever information I need for a given task—but no more—along with cryptographic protection to ensure that they can't be forged, reused, traced back to me, or linked to other tokens that I have issued.

In a world with U-Prove, many existing identity management problems would go away. If my credit card company and online music service both supported U-Prove, I could create a token that allowed a single limited electronic money transfer from my card to the music company, without disclosing my name, address, or date of birth, and without that token being usable to make further purchases. Similarly, I might want to buy a computer game from an online store, the same situation as before, but this time with a twist: the computer game is rated 18+. So to make the purchase, I have to reveal my age, as well as the money transfer, to the online store. U-Prove lets me do this, but still doesn't require me to reveal my name, address, or any other irrelevant detail.

An hour-long presentation by Dr Brands describes how U-Prove works and how it achieves what it does (with even more detail available in his freely downloadable book). It builds on existing public key cryptography concepts, but adds to them the important ability to hide data. Normal public key cryptography is something of an all-or-nothing affair—to prove that a particular piece of data was encrypted by a particular person, you need to know the data. U-Prove allows that proof to take place without revealing all the data.

All of this is useful, but it suffers a substantial chicken-and-egg problem. Users don't have U-Prove, and have no incentive to get it, because service providers don't support it anyway. Without identity providers (governments, banks, credit card companies), users, and service providers all agreeing to use the system, it's unfortunately pointless. Systems similar to U-Prove have been tried before, but typically they've demanded money from both users and service providers alike to integrate the software. Outside narrow niches, this is an unappealing prospect—they're paying to integrate a system that no one uses anyway.

On top of this, Microsoft has dabbled, unsuccessfully, in this area before. In 2001, the company announced its Hailstorm platform. Hailstorm was a set of Web services, integrating with the company's Passport authentication system (now known as Windows Live ID) that allowed users to authenticate with third-party online applications and provide them with varying access to personal data stored within Passport. Hailstorm was, however, a centralized system, with key portions owned and operated by Microsoft. The software community balked at giving Redmond this much control—massive centralization is not an acceptable approach to the identity problem—and Hailstorm, along with third-party usage of Passport, was killed off.

It is for these reasons that Microsoft has released its U-Prove SDK using the open source BSD license. Source code is available in both C# and Java, and the technology is covered by Microsoft's Open Specification Promise. This is a irrevocable promise by Microsoft that the company will not assert any claims against anyone using the technology that relate to any patents covering the technology. By releasing the technology under a permissive license, and by making a legally binding agreement that patents covering the technology will not be used in legal action, the company hopes that there will be no barriers to using the system for both service and identity providers.

In addition to this core technology, Microsoft is integrating U-Prove into a range of its own identity products: Windows CardSpace 2, Active Directory Federation Services 2, and Windows Identity Foundation. These technologies, codenamed "Geneva," provide identity management facilities to end-users, administrators, and developers (respectively), allowing each of those groups to integrate U-Prove into their existing systems. As well as pure software systems, U-Prove can also be integrated with smartcard systems to provide multi-factor authentication and various other features.

Unfortunately, the reality is that U-Prove adoption still faces an uphill struggle. The desirability for end-users is clear; it permits both secure and private use of online services, and should offer substantial protection against many of the online threats currently faced. It will substantially mitigate the damage caused by the kind of commonplace security breaches, and allow a much higher degree of privacy than is currently possible. Identity providers—governments, banks, credit card companies—might also like the system. Credit card fraud costs credit companies and banks money, after all—a secure system for online payments would undoubtedly be preferred.

Online service providers, however, are a slightly different story. U-Prove allows meaningful privacy. Obviously, if I'm buying something from Amazon and need it to delivered, I need to give Amazon a delivery address. This kind of information disclosure is unavoidable. But if I'm just downloading an MP3 from the company, they don't need to know where I live (perhaps they need to know my country, but certainly not my full address). If Amazon supported U-Prove, I could make both these transactions, and Amazon would not even be able to know that it was me both times—that's the level of privacy it provides. Except that Amazon probably wants to know. Amazon, as with many other online service providers (king of these being Google), makes extensive use of shared data to make suggestions, both to me and other shoppers. If I take away Amazon's ability to link purchase data, to determine trends and patterns, Amazon becomes that much worse off.

This ability to mine data is increasingly important. It's why so many brick-and-mortar stores offer loyalty cards these days—loyalty cards let them track my habits even when I use (otherwise untraceable) cash. A reliable, effective system that allows private purchasing would diminish that data mining capability, to the detriment of the vendors.

This is a pity. The technology is clever, and the capabilities it offers would make the online world a great deal safer. But outside of certain closed systems—large corporations wanting to govern access to their own internal systems, for example—it's hard to see U-Prove ever taking off.