Designing, implementing, and maintaining APIs for the Web is more than a challenge; for many companies, it is an imperative. This series takes the reader on a journey from determining the business case for APIs to a design methodology, meeting implementation challenges, and taking the long view on maintaining public APIs on the Web over time. Along the way there are interviews with influential individuals and even a suggested reading list on APIs and related topics.

This InfoQ article is part of the series “Web APIs From Start to Finish”. You can subscribe to receive notifications via RSS.

Roy T. Fielding is a Senior Principal Scientist at Adobe and a major force in the world of networked software. While a graduate student at University of California at Irvine (UCI), Fielding worked on a class project to create a maintenance robot for the Web called MOMSpider. Along the way, he created the libwww-perl library and derived some of the underlying principles behind the architecture of the WWW; what Fielding originally called the HTTP Object Model. A few years later, when working on his Ph.D. dissertation, he renamed his model to Representational State Transfer or REST "to evoke an image of how a well-designed Web application behaves".

Fielding’s contributions to open standards are extensive. His name appears on more than a dozen RFC specifications including HTTP, URI Templates, and others. Fielding is also one of the editors for the W3C’s Do Not Track standards effort. As a founding member of the Apache HTTP Server project, he helped create the world's most popular Web server software, wrote the Apache License, incorporated the Apache Software Foundation, and served as its first chairman.

Recently Roy took some time while traveling between standards meetings to answer a series of questions on a topic that often starts debates; Versioning on the Web. He also talked about why hypermedia is a requirement in his REST style, the process of designing network software that can adapt over time, and the challenge of thinking at the scale of decades.

InfoQ: Back in August of 2013, you gave a talk for the Adobe Evolve conference and, in that talk, you offered advice on how to approach "versioning" APIs on the web. It was a single word: "DON’T". What kind of reaction have you seen from that guidance?

Roy: I think everyone in attendance had a positive reaction, since most are our customers and familiar with the design rationale behind the Adobe Experience Manager products. Of course, I wasn’t reading the slides for that audience; I was explaining the rationale behind the conclusions seen in them. The Internet reaction to the published slides was a little more mixed, with some folks misunderstanding what I meant by versioning and others misunderstanding the point about changing the hostname/branding. By versioning, I meant sticking client-visible interface numbers inside various names so that the client labels every interaction as belonging to a given version of that API. Unfortunately, versioning interface names only manages change for the API owner’s sake. That is a myopic view of interface design: one where the owner’s desire for control ignores the customer’s need for continuity.

InfoQ: So, what happens when you version an API?

Roy: Either, (a) the version is eventually changed and all of the components written to the prior version need to be restarted, redeployed, or abandoned because they cannot adapt to the benefits of that newer system, or (b) the version is never changed and is just a permanent lead weight making every API call less efficient. A lot of developers throw up their hands in disgust at this point and claim that I just don’t understand their problem. Their systems are important. They are going to change. New features are going to be provided. Data is going to be rearranged. They need some way to control how old clients can coexist with new ones. Naturally, that is where I have to explain why "hypermedia as the engine of application state" is a REST constraint. Not an option. Not an ideal. Hypermedia is a constraint. As in, you either do it or you aren’t doing REST. You can’t have evolvability if clients have their controls baked into their design at deployment. Controls have to be learned on the fly. That’s what hypermedia enables. But that alone is still not enough for evolvability. Hypermedia allows application controls to be supplied on demand, but we also need to be able to adapt the clients' understanding of representations (understanding of media types and their expected processing). That is where code-on-demand shines.

InfoQ: So, one of the reasons Hypermedia is a requirement in the REST style is to deal with change over time, right?

Roy: Anticipating change is one of the central themes of REST. It makes sense that experienced developers are going to think about all of the ways that their API might change in the future, and to think that versioning the interface is paving the way for those changes. That led to a never-ending debate about where and how to version the API. The techniques that developers learn from managing in-house software, where they might reasonably believe they have control over deployment of both clients and servers, simply don’t apply to network-based software intended to cross organizational boundaries. This is precisely the problem that REST is trying to solve: how to evolve a system gracefully without the need to break or replace already deployed components. Hence, my slides try to restore focus to where it belongs: evolvability. In other words, don’t build an API to be RESTful — build it to have the properties you want. REST is useful because it induces certain properties that are known to benefit multi-org systems, like evolvability. Evolvability means that the system doesn’t have to be restarted or redeployed in order to adapt to change.

InfoQ: Does that mean as long as I use the REST style I am free and clear of versioning issues?

Roy: No. It is always possible for some unexpected reason to come along that requires a completely different API, especially when the semantics of the interface change or security issues require the abandonment of previously deployed software. My point was that there is no need to anticipate such world-breaking changes with a version ID. We have the hostname for that. What you are creating is not a new version of the API, but a new system with a new brand. On the Web, we call that a new website. Websites don’t come with version numbers attached because they never need to. Neither should a RESTful API. A RESTful API (done right) is just a website for clients with a limited vocabulary.

InfoQ: One of the things you talk about when referring to your architectural style REST is that it was designed to support "software engineering on the scale of decades." What does "scale of decades" mean, in tangible terms?

Roy: REST was originally created to solve my problem: how do I improve HTTP without breaking the Web. It was an important problem to solve when I started rewriting the HTTP standard in 1994-95. I was a post-Masters Ph.D. student in software engineering, trying not to screw up what was clearly becoming the printing press of our age, which means I had to define a system that could withstand decades of change produced by people spread all over the world. How many software systems built in 1994 still work today? I meant it literally: decades of use while the system continued to evolve, in independent and orthogonal directions, without ever needing to be shut down or redeployed. Two decades, so far.

InfoQ: You yourself have acknowledged that this is a level of engineering in which most architects, designers, and developers don’t operate. So why talk about this level of engineering scale?

Roy: I talk about it because the initial reaction to using REST for machine-to-machine interaction is almost always of the form "we don’t see a reason to bother with hypermedia — it just slows down the interactions, as opposed to the client knowing directly what to send." The rationale behind decoupling for evolvability is simply not apparent to developers who think they are working towards a modest goal, like "works next week" or "we’ll fix it in the next release". If developers can conceive of their systems being used for a much longer time, then we can escape their own preconceptions about how it will need to change over time. We can then work back from decades to years (how long until you don’t know your users?) or even months (how long until you’ve lost control over client deployment?).

InfoQ: The HTTP application-level protocol is often cited as an example of successful engineering at the scale of decades. Yet, HTTP has gone through more than one version and the early versions of HTTP got a number of things wrong including the Host header problem, absolute time caching directives, and others. Does that run counter to your "DON’T" guidance for Web APIs?

Roy: No, HTTP doesn’t version the interface names — there are no numbers on the methods or URIs. That doesn’t mean other aspects of the communication aren’t versioned. We do want change, since otherwise we would not be able to improve over time, and part of change is being able to declare in what language the data is spoken. We just don’t want breaking change. Hence, versioning is used in ways that are informative rather than contractual. BTW, it is more accurate to say that HTTP got almost everything right, but that the world simply changed around (and because of) it. Host would have been a stupid idea in 1992 because nobody needed multiple domains per IP until the Web made being on the Internet a business imperative. Persistent connections would have been a terrible idea up until Mosaic added embedded images to HTML. And absolute times for expiration made more sense when people hosting mirrors looked at those fields, not caches, and the norm was to expire in weeks rather than seconds.

InfoQ: Then what lessons can we draw from the fact that HTTP and even HTML have changed over time?

Roy: What we learned from HTTP and HTML was the need to define how the protocol/language can be expected to change over time, and what recipients ought to do when they receive a change they do not yet understand. HTTP was able to improve over time because we required new syntax to be ignorable and semantics to be changed only when accompanied by a version that indicates such understanding.

InfoQ: It seems Web developers struggle to handle change more today than in the past. Are we running into new problems? just seeing more of the same?

Roy: I think there are just more opportunities to struggle now than there were in the past. It has become so easy for people to create systems with astonishing reach, whereas it used to take years to get a company just to deploy a server outside its own network. It’s a good problem to have, most of the time. Software developers have always struggled with temporal thinking.

InfoQ: Finally, in addition to "DON’T", what advice you would pass on to Web API designers, architects, and developers to help them deal with the problem of change over time?

Roy: Heh, I didn’t say DON’T change over time — just don’t use deliberately breaking names in an API. I find it impossible to give out generic advice, since almost anything I could say would have to be specific to the context and type of system being built. REST is still my advice on how to build an application for the Web in a fashion that is known to work well over time and known to create more Web as a result (more addressable resources).

About the Interviewee

Roy T. Fielding is a Senior Principal Scientist at Adobe and a major force in the world of networked software. While a graduate student at University of California at Irvine (UCI), Fielding worked on a class project to create a maintenance robot for the Web called MOMSpider. Along the way, he created the libwww-perl library and derived some of the underlying principles behind the architecture of the WWW; what Fielding originally called the HTTP Object Model. A few years later, when working on his Ph.D. dissertation, he renamed his model to Representational State Transfer or REST "to evoke an image of how a well-designed Web application behaves".

Designing, implementing, and maintaining APIs for the Web is more than a challenge; for many companies, it is an imperative. This series takes the reader on a journey from determining the business case for APIs to a design methodology, meeting implementation challenges, and taking the long view on maintaining public APIs on the Web over time. Along the way there are interviews with influential individuals and even a suggested reading list on APIs and related topics.

This InfoQ article is part of the series “Web APIs From Start to Finish”. You can subscribe to receive notifications via RSS.