In a recent announcement, Xmarks revealed that it can no longer afford to continue operating its popular cross-browser bookmark synchronization service. Although Xmarks attracted a significant audience of users, the company was never able to build a sustainable business around the software. Xmarks cofounder Todd Agulnick says that the lights are scheduled to go out in 90 days unless the company can come up with a successful freemium strategy or a buyer emerges.

This is a disappointing development for those of us who rely on Xmarks' excellent browser add-ons and reliable synchronization service. A number of popular Web browsers have their own built-in synchronization offerings, but none appear to be truly conducive to cross-browser synchronization. This inspired me to take a close look at the bookmark sync APIs in Firefox and Chrome in order to evaluate their conduciveness to third-party interoperability and determine if the potential exists for an Xmarks successor to emerge.

What I found are a lot of challenges and not a lot of easy solutions. The native synchronization services for Chrome and Firefox are compelling and have a lot of potential, but interoperability doesn't yet appear to be an immediate priority for either one at this stage.

Chrome

Chrome's synchronization mechanism is a bit unusual. Google opted to use real-time communication to "push" synchronization events, causing changes to be reflected almost immediately in all connected browser instances. Synchronization event notifications are transmitted in an XMPP-based protocol that runs on top of the Google Talk infrastructure. The actual synchronization data is encoded in a protobuf format so that it will be less bandwidth-intensive when it is transmitted over the wire.

For those who are unfamiliar, protobuf (Protocol Buffers) is an open binary-serialization format that was developed by Google. To use protobuf in an application, developers create "proto" documents that basically use a specialized interface description language to describe the structure of the data that will be transmitted over the wire. The proto documents are then run through a compiler that can generate code stubs in a number of programming languages.

If you look at the sync directory in the Chromium source code, you can find all of the proto files in the protocol subdirectory. If you want to see how the XMPP-based event notification protocol works, you might want to take a look at the sync listener tool included in the source code, which can log into Google's XMPP server and echo sync notifications at the command line.

The use of an open protobuf format and standards-based XMPP means that it should theoretically be possible for a third-party to create a client application that is interoperable with Chrome's synchronization service. Of course, doing so in practice would still be a technically challenging undertaking.

The mechanics are complicated and there is no high-level documentation that describes Chrome's synchronization architecture. The comments in the source code are instructive and relatively thorough, but they tend to provide a bottom-up view that is—as you would expect of code comments—aimed more at articulating information that is relevant to Chrome contributors.

One of the downsides of Chrome's sophisticated synchronization protocol is that it's largely overkill for developers who want to make simple clients, like mobile applications that provide convenient read-only access to the user's bookmarks in the cloud. The user's bookmark data is exposed through Google Docs, but it's not accessible through the GData APIs. There were experimental REST APIs available at one point—which led to the development of some useful third-party client applications like SyncdMarks—but it has since been shut down. Google is expected to offer REST APIs for bookmarks again at some point in the future, but it's not clear when it will happen.

It's a bit frustrating that Google's own bookmark synchronization service isn't even integrated with Android. I'd settle for having a native Android application that gives me read-only access to my bookmarks, but such a thing doesn't seem to exist yet and likely won't be created until there are developer-friendly REST APIs for accessing Chrome bookmarks. The mobile Google Docs Web interface does provide browser-based access to bookmarks, but it's a poor substitute for a native application, especially because it seems to flatten the folder tree.

Firefox

Mozilla began exploring synchronization several years ago when the Weave project emerged from Mozilla Labs. Weave was a bit flaky during the early stages of development, but it matured considerably over the past few years and has become quite robust.

Now called Firefox Sync, it is available to Firefox 3.6 users in the form of a browser add-on that communicates with Mozilla's own hosted synchronization server. In light of its suitability for widespread adoption, Mozilla intends to include it as a standard built-in feature in Firefox 4, the next major version of the Web browser.

The Firefox synchronization service has a documented REST API that operates on a simple polling principle and appears on the surface to pose no major challenges for third-party implementors... until you see the encryption mechanism.

One of the distinguishing characteristics of Mozilla's approach to synchronization is the organization's extreme focus on privacy. Firefox Sync uses an encryption scheme that is intended to ensure that even the operator of the synchronization service can't sneak a peek at the user's data. It's designed to be flexible, however, so that features like secure bookmark sharing between users can be implemented at a later date.

The downside of the complex encryption scheme is that it makes it nontrivial to create a compatible client implementation and also practically precludes third parties from offering convenient Web interfaces for browsing and managing bookmarks (more on that later). Matters are made worse by a Mozilla-specific idiosyncrasy in the implementation that adds extra hurdles for developers to jump through to create independent implementations.

The specific details about how Firefox Sync encryption actually works are pretty dry and beyond the scope of this article, but Mozilla's Anant Narayanan has published a very clear and accessible overview that is worth a look if such things interest you. It's not all that intimidating if you know what you are doing and have good encryption libraries, but there are warts. You have to parse by hand a wonky NSS-specific ASN.1 format, for example. That makes it seem like support for third-party interoperability was an afterthought rather than a guiding principle of Weave's design.

The emphasis on security over ease of development is an understandable trade-off that many users will likely appreciate, but it's a bit frustrating from the perspective of a third-party developer. Interoperating with the Firefox Sync system isn't prodigiously difficult, but the barrier to entry is high enough to deter most casual developers. It's just not worth the effort in most cases.

An experimental Python client with partial Firefox Sync support was developed by Mozilla's Michael Hanson. It offers a compelling demonstration that independent implementations are viable, but it also shows that there are a lot of hoops to jump through. Somebody has also started working on a Java library for communicating with the service, but it's still at a very early stage of development and doesn't do much yet.

One of the most intriguing third-party Firefox Sync client implementations is a highly experimental extension for Chrome that is largely developed in JavaScript. It's far from being production-ready, but it's a promising sign that the third-party developer community could eventually make Firefox Sync a viable solution for cross-browser bookmark synchronization.

It's likely that there won't be a ton of third-party momentum around Mozilla's synchronization service, however, until there is a more complete set of production-ready libraries that abstract away the complexities. Even after that has been achieved, the strictness of the security model will still impose some limitations on what third-party developers can do.

One of the issues I raised earlier in the article is that Mozilla doesn't really want Web applications to access the user's bookmarks. This is because a Web application that interacts directly with the bookmarks would have to have access to the user's key, so that it could decrypt and extract the relevant data. Mozilla is quite insistent that decryption should only occur on the client-side, on the user's own computer, so that third-party Web services never have the ability to see the user's bookmarks.

This policy unfortunately rules out a lot of potentially useful third-party applications. For example, it forbids developers from making a Web application that periodically pulls the user's latest bookmarks from the Firefox Sync server and propagates them into a Delicious account. Narayanan is experimenting with ways to facilitate developing a browser-based bookmark management interface that will work anywhere without eroding the service's privacy standards, but it's somewhat esoteric. Much like the experimental Chrome extension, it involves implementing all of the decryption mechanisms in pure JavaScript so it can run in the browser rather than on the server.

Although the Firefox Sync service poses some challenges and annoyances for third-party developers, there are also some aspects that are advantageous for interoperability. Mozilla supplies an open source reference implementation of the Weave server software, for example. This will make it easier for third-party developers to create compatible server implementations and it will also simplify debugging for client developers because it will allow them to test against an observable local server instance. The availability of an open source sync server implementation is also really great for users, because it will allow them to optionally self-host their own bookmark storage.

Conclusion

When I started working on this article, my goal was to unravel the bookmark sync APIs so that I could start making my own client applications and possibly roll my own custom cross-browser solution to fill the need left by Xmarks. What I discovered is that such an undertaking isn't really within the realm of weekend projects at this point. The APIs just aren't there yet. It's important to remember that both the Chrome and Firefox synchronization services are still under development and will likely be more supportive of interoperability in the future as they mature.