INFORMATIONAL

Network Working Group M. Rose Request for Comments: 3117 Dover Beach Consulting, Inc. Category: Informational November 2001 On the Design of Application Protocols Status of this Memo This memo provides information for the Internet community. It does not specify an Internet standard of any kind. Distribution of this memo is unlimited. Copyright Notice Copyright (C) The Internet Society (2001). All Rights Reserved. Abstract This memo describes the design principles for the Blocks eXtensible eXchange Protocol (BXXP). BXXP is a generic application protocol framework for connection-oriented, asynchronous interactions. The framework permits simultaneous and independent exchanges within the context of a single application user-identity, supporting both textual and binary messages. Rose Informational [Page 1]

RFC 3117 On the Design of Application Protocols November 2001 1 . A Problem 19 Years in the Making 1] is close to being the perfect application protocol: it solves a large, important problem in a minimalist way. It's simple enough for an entry-level implementation to fit on one or two screens of code, and flexible enough to form the basis of very powerful product offerings in a robust and competitive market. Modulo a few oddities (e.g., SAML), the design is well conceived and the resulting specification is well-written and largely self-contained. There is very little about good application protocol design that you can't learn by reading the SMTP specification. Unfortunately, there's one little problem: SMTP was originally published in 1981 and since that time, a lot of application protocols have been designed for the Internet, but there hasn't been a lot of reuse going on. You might expect this if the application protocols were all radically different, but this isn't the case: most are surprisingly similar in their functional behavior, even though the actual details vary considerably. In late 1998, as Carl Malamud and I were sitting down to review the Blocks architecture, we realized that we needed to have a protocol for exchanging Blocks. The conventional wisdom is that when you need an application protocol, there are four ways to proceed: 1. find an existing exchange protocol that (more or less) does what you want; 2. define an exchange model on top of the world-wide web infrastructure that (more or less) does what you want; 3. define an exchange model on top of the electronic mail infrastructure that (more or less) does what you want; or, 4. define a new protocol from scratch that does exactly what you want. An engineer can make reasoned arguments about the merits of each of the these approaches. Here's the process we followed... The most appealing option is to find an existing protocol and use that. (In other words, we'd rather "buy" than "make".) So, we did a survey of many existing application protocols and found that none of them were a good match for the semantics of the protocol we needed. For example, most application protocols are oriented toward client/server behavior, and emphasize the client pulling data from the server; in contrast with Blocks, a client usually pulls data from Rose Informational [Page 3]

RFC 3117 On the Design of Application Protocols November 2001 2] or SMTP into what we wanted, but by the time we did all that, the base protocol and our protocol would have more differences than similarities. In other words, the cost of modifying an off-the-shelf implementation becomes comparable with starting from scratch. Another approach is to use HTTP [3] as the exchange protocol and define the rules for data exchange over that. For example, IPP [4] (the Internet Printing Protocol) uses this approach. The basic idea is that HTTP defines the rules for exchanging data and then you define the data's syntax and semantics. Because you inherit the entire HTTP infrastructure (e.g., HTTP's authentication mechanisms, caching proxies, and so on), there's less for you to have to invent (and code!). Or, conversely, you might view the HTTP infrastructure as too helpful. As an added bonus, if you decide that your protocol runs over port 80, you may be able to sneak your traffic past older firewalls, at the cost of port 80 saturation. HTTP has many strengths: it's ubiquitous, it's familiar, and there are a lot of tools available for developing HTTP-based systems. Another good thing about HTTP is that it uses MIME [5] for encoding data. Unfortunately for us, even with HTTP 1.1 [6], there still wasn't a good fit. As a consequence of the highly-desirable goal of maintaining compatibility with the original HTTP, HTTP's framing mechanism isn't flexible enough to support server-side asynchronous behavior and its authentication model isn't similar to other Internet applications. Mapping IPP onto HTTP 1.1 illustrates the former issue. For example, the IPP server is supposed to signal its client when a job completes. Since the HTTP client must originate all requests and since the decision to close a persistent connection in HTTP is unilateral, the best that the IPP specification can do is specify this functionality in a non-deterministic fashion. Further, the IPP mapping onto HTTP shows that even subtle shifts in behavior have unintended consequences. For example, requests in IPP are typically much larger than those seen by many HTTP server implementations -- resulting in oddities in many HTTP servers (e.g., requests are sometimes silently truncated). The lesson is that HTTP's framing mechanism is very rigid with respect to its view of the request/response model. Rose Informational [Page 4]

RFC 3117 On the Design of Application Protocols November 2001 Rose Informational [Page 5]

RFC 3117 On the Design of Application Protocols November 2001 2 . You can Solve Any Problem... 7]). Another branch in the taxonomy, connectionless, consists of those that don't want the delay or overhead of establishing and maintaining a reliable stream. For example, most DNS [8] traffic is characterized by a single request and response, both of which fit within a single IP datagram. In this case, it makes sense to implement a basic reliability service above the transport layer in the application protocol itself. Second, we're only going to consider message-oriented application protocols. A "message" -- in our lexicon -- is simply structured data exchanged between loosely-coupled systems. Another branch in the taxonomy, tightly-coupled systems, uses remote procedure calls as the exchange paradigm. Unlike the connection-oriented/connectionless dichotomy, the issue of loosely- or tightly-coupled systems is similar to a continuous spectrum. Fortunately, the edges are fairly sharp. For example, NFS [9] is a tightly-coupled system using RPCs. When running in a properly-configured LAN, a remote disk accessible via NFS is virtually indistinguishable from a local disk. To achieve this, tightly-coupled systems are highly concerned with issues of latency. Hence, most (but not all) tightly-coupled systems use connection-less RPC mechanisms; further, most tend to be implemented as operating system functions rather than user-level programs. (In some environments, the tightly-coupled systems are implemented as single-purpose servers, on hardware specifically optimized for that one function.) Finally, we're going to consider the needs of application protocols that exchange messages asynchronously. The classic client/server model is that the client sends a request and the server sends a Rose Informational [Page 6]

RFC 3117 On the Design of Application Protocols November 2001 Rose Informational [Page 7]

RFC 3117 On the Design of Application Protocols November 2001 3 . Protocol Mechanisms 3.1 Framing Rose Informational [Page 8]

RFC 3117 On the Design of Application Protocols November 2001 10] and network bugs. In addition, if the messages being exchanged are small (say less than 32k), then the overhead of establishing a connection for each message contributes significant latency during data exchange. 3.2 Encoding 11]. The 822-style divides a message into textual headers and an unstructured body. Each header consists of a name and a value and is terminated with a CR-LF pair. An additional CR-LF separates the headers from the body. It is this structure that HTTP uses to indicate the length of the body for framing purposes. More formally, HTTP uses MIME, an application of the 822-style to encode both the data itself (the body) and information about the data (the headers). That is, although HTTP is commonly viewed as a retrieval mechanism for HTML [12], it is really a retrieval mechanism for objects encoded using MIME, most of which are either HTML pages or referenced objects such as GIFs. 3.3 Reporting Rose Informational [Page 9]

RFC 3117 On the Design of Application Protocols November 2001 13] provides a convenient mechanism for extending the number of values that can occur in the second and third positions. Virtually all of the application protocols we've discussed thus far use the three-digit reply codes, although there is less coordination between the designers of different application protocols than most would care to admit. (A variation on the theory of reply codes is employed by IMAP [14] which provides the same information using a different syntax.) In addition to conveying a reply code, most application protocols also send a textual diagnostic suitable for human, not machine, consumption. (More accurately, the textual diagnostic is suitable for people who can read a widely used variant of the English language.) Since reply codes reflect both positive and negative outcomes, there have been some innovative uses made for the text accompanying positive responses, e.g., prayer wheels [39]. Regardless, some of the more modern application protocols include a language localization parameter for the diagnostic text. Finally, since the introduction of reply codes in 1981, two unresolved criticisms have been raised: o a reply code is used both to signal the outcome of an operation and a change in the application protocol's state; and, o a reply code doesn't specify whether the associated textual diagnostic is destined for the end-user, administrator, or programmer. 3.4 Asynchrony 15] in SMTP or persistent connections in HTTP 1.1. Pipelining allows a client to make multiple requests of a server, but requires the requests to be processed serially. (Note that a protocol needs to explicitly provide support for pipelining, since, without explicit guidance, many implementors Rose Informational [Page 10]

RFC 3117 On the Design of Application Protocols November 2001 Rose Informational [Page 11]

RFC 3117 On the Design of Application Protocols November 2001 3.5 Authentication 16] that authenticates the messages it exchanges.) A large part of the problem is that different security mechanisms optimize for strength, scalability, or ease of deployment. So, a few years ago, SASL [17] (the Simple Authentication and Security Layer) was developed to provide a framework for authenticating protocol peers. SASL let's you describe how an authentication mechanism works, e.g., an OTP [18] (One-Time Password) exchange. It's then up to each protocol designer to specify how SASL exchanges are generically conveyed by the protocol. For example, [19] explains how SASL works with SMTP. A notable exception to the SASL bandwagon is HTTP, which defines its own authentication mechanisms [20]. There is little reason why SASL couldn't be introduced to HTTP, although to avoid certain race- conditions, the persistent connection mechanism of HTTP 1.1 must be used. SASL has an interesting feature in that in addition to explicit protocol exchanges to authenticate identity, it can also use implicit information provided from the layer below. For example, if the connection is running over IPsec [21], then the credentials of each peer are known and verified when the TCP connection is established. Finally, as its name implies, SASL can do more than authentication -- depending on which SASL mechanism is in use, message integrity or privacy services may also be provided. 3.6 Privacy 22], is available to all application protocols, e.g., SMTP and ACAP [23] (the Application Configuration Access Protocol). Rose Informational [Page 12]

RFC 3117 On the Design of Application Protocols November 2001 24] and [25]) listens on a single port for plaintext traffic, and, once a connection is established, the use of TLS on that connection is negotiable. Finally, note that both SASL and TLS are about "transport security" not "object security". What this means is that they focus on providing security properties for the actual communication, they don't provide any security properties for the data exchanged independent of the communication. 3.7 Let's Recap 26] uses octet-stuffing, but IMAP uses octet-counting, and so on. Rose Informational [Page 13]

RFC 3117 On the Design of Application Protocols November 2001 4 . Protocol Properties 4.1 Scalability 27]. Further, suggestions are also being considered for modification of TCP implementations to reduce concurrent learning, e.g., [28]. In terms of the server, each incoming connection must be dispatched and (probably) authenticated against the same resources. Consequently, server overhead increases based on the number of connections established, rather than the number of remote users. The same issues of fairness arise: it's much harder for servers to allocate resources on a per-user basis, when a user can cause an arbitrary number of connections to pound on the server. Another important aspect of scalability to consider is the relative numbers of clients and servers. (This is true even in the peer-to- peer model, where a peer can act both in the client and server role.) Typically, there are many more client peers than server peers. In this case, functional requirements should be shifted from the servers onto the clients. The reason is that a server is likely to be interacting with multiple clients and this functional shift makes it easier to scale. Rose Informational [Page 14]

RFC 3117 On the Design of Application Protocols November 2001 29] can't function without being built around the notion of extensions. 4.5 Robustness Rose Informational [Page 16]

RFC 3117 On the Design of Application Protocols November 2001 5 . The BXXP Framework 5.1 Framing and Encoding Rose Informational [Page 17]

RFC 3117 On the Design of Application Protocols November 2001 30]. (Actually, a "Content-Type: text/xml" with binary transfer encoding). We chose XML as the default because it provides a simple mechanism for nested, textual representations. (Alas, the 822-style encoding doesn't easily support nesting.) By design, XML's nature isn't optimized for compact representations. That's okay because we're focusing on loosely-coupled systems and besides there are efficient XML parsers available. Further, there's a fair amount of anecdotal experience -- and we'll stress the word "anecdotal" -- that if you have any kind of compression (either at the link-layer or during encryption), then XML encodings squeeze down nicely. Even so, use of XML is probably the most controversial part of BXXP. After all, there are more efficient representations around. We agree, but the real issue isn't efficiency, it's ease of use: there are a lot of people who grok the XML thing and there are a lot of XML tools out there. The pain of recreating this social infrastructure far outweighs any benefits of devising a new representation. So, if the "make" option is too expensive, is there something else we can "buy" besides XML? Well, there's ASN.1/BER (just kidding). Rose Informational [Page 18]

RFC 3117 On the Design of Application Protocols November 2001 31], which does use ASN.1, the same issues arose. In the end, the working group agreed that the use of ASN.1 for SNMP was axiomatic, but not because anyone thought that ASN.1 was the most efficient, or the easiest to explain, or even well liked. ASN.1 was given axiomatic status because the working group decided it was not going to spend the next three years explaining an alternative encoding scheme to the developer community. So -- and we apologize for appealing to dogma -- use of XML as the favored encoding scheme in BXXP is axiomatic. 5.2 Reporting 5.3 Asynchrony 32], but none appear to have achieved orbit.) So, here's what we did: frames are exchanged in the context of a "channel". Each channel has an associated "profile" that defines the syntax and semantics of the messages exchanged over a channel. Channels provide both an extensibility mechanism for BXXP and the basis for parallelism. Remember the last parameter in the command line of a BXXP frame? The "part of the application" that gets the message is identified by a channel number. A profile is defined according to a "Profile Registration" template. The template defines how the profile is identified (using a URI [33]), what kind of messages get exchanged, along with the syntax and semantics of those messages. When you create a channel, you identify a profile and maybe piggyback your first message. If the channel is successfully created, you get back a positive response; otherwise, you get back a negative response explaining why. Perhaps the easiest way to see how channels provide an extensibility mechanism is to consider what happens when a session is established. Each BXXP peer immediately sends a greeting on channel zero Rose Informational [Page 19]

RFC 3117 On the Design of Application Protocols November 2001 Rose Informational [Page 20]

RFC 3117 On the Design of Application Protocols November 2001 5.4 Authentication 34]) to be done on a per-user granularity. 5.5 Privacy 5.6 Things We Left Out Rose Informational [Page 21]

RFC 3117 On the Design of Application Protocols November 2001 5.7 From Framework to Protocol 35] and read it. Next, define your own profile. Finally, get one of the open source SDKs (in C, Java, or Tcl) and start coding. The BEEP specification defines several profiles itself: a channel management profile, a family of profiles for SASL, and a transport security profile. In addition, there's a second specification [36] that explains how a BEEP session maps onto a single TCP connection. For a complete example of an application protocol defined using BEEP, look at reliable syslog [37]. This document exemplifies the formula: application protocol = BEEP + 1 or more profiles + authorization policies + provisioning rules (e.g., use of SRV RRs [38]) Rose Informational [Page 22]

RFC 3117 On the Design of Application Protocols November 2001 6 . BXXP is now BEEP 7 . Security Considerations 35]'s Section 8 for a discussion of BEEP-related security issues. Rose Informational [Page 23]

RFC 3117 On the Design of Application Protocols November 2001