Note that since this PEP was first created, PEP 3333 was created as a more incremental update that permitted use of WSGI on Python 3.2+. However, an alternative specification that furthers the Python 3 goals of a cleaner separation of binary and text data may still be valuable.

Further exploration of the concepts covered in this PEP has been deferred for lack of a current champion interested in promoting the goals of the PEP and collecting and incorporating feedback, and with sufficient available time to do so effectively.

Python 2.6 is the first Python version that supported an alias for bytes and the b"foo" literal syntax. This is why it is the minimum version supported by Web3.

Whatever Python 3 version fixed http://bugs.python.org/issue4006 so os.environ['foo'] returns surrogates (ala PEP 383 ) when the value of 'foo' cannot be decoded using the current locale instead of failing with a KeyError is the true minimum Python 3 version. In particular, however, Python 3.0 is not supported.

Applications and servers which are written to this specification are meant to work properly under Python 2.6.X, Python 2.7.X and Python 3.1+. Neither an application nor a server that implements the Web3 specification can be easily written which will work under Python 2 versions earlier than 2.6 nor Python 3 versions earlier than 3.1.

Python currently boasts a wide variety of web application frameworks which use the WSGI 1.0 protocol. However, due to changes in the language, the WSGI 1.0 protocol is not compatible with Python 3. This specification describes a standardized WSGI-like protocol that lets Python 2.6, 2.7 and 3.1+ applications communicate with web servers. Web3 is clearly a WSGI derivative; it only uses a different name than "WSGI" in order to indicate that it is not in any way backwards compatible.

This protocol and specification is influenced heavily by the Web Services Gateway Interface (WSGI) 1.0 standard described in PEP 333 . The high-level rationale for having any standard that allows Python-based web servers and applications to interoperate is outlined in PEP 333 . This document essentially uses PEP 333 as a template, and changes its wording in various places for the purpose of forming a different standard.

Here a middleware that changes the HTTP_HOST key if an X-Host header exists and adds a comment to all html responses:

The presence of middleware in general is transparent to both the "server/gateway" and the "application/framework" sides of the interface, and should require no special support. A user who desires to incorporate middleware into an application simply provides the middleware component to the server, as if it were an application, and configures the middleware component to invoke the application, as if the middleware component were a server. Of course, the "application" that the middleware wraps may in fact be another middleware component wrapping another application, and so on, creating what is referred to as a "middleware stack".

A single object may play the role of a server with respect to some application(s), while also acting as an application with respect to some server(s). Such "middleware" components can perform such functions as:

The server or gateway invokes the application callable once for each request it receives from an HTTP client, that is directed at the application. To illustrate, here is a simple CGI gateway, implemented as a function taking an application object. Note that this simple example has limited error handling, because by default an uncaught exception will be dumped to sys.stderr and logged by the web server.

Alternately, an application callable may return a callable instead of the tuple if the server supports asynchronous execution. See information concerning web3.async for more information.

Although we refer to it as an "application" object, this should not be construed to mean that application developers will use Web3 as a web programming API. It is assumed that application developers will continue to use existing, high-level framework services to develop their applications. Web3 is a tool for framework and server developers, and is not intended to directly support application developers.)

The application object is simply a callable object that accepts one argument. The term "object" should not be misconstrued as requiring an actual object instance: a function, method, or instance with a __call__ method are all acceptable for use as an application object. Application objects must be able to be invoked more than once, as virtually all servers/gateways (other than CGI) will make such repeated requests. If this cannot be guaranteed by the implementation of the actual application, it has to be wrapped in a function that creates a new instance on each call.

Throughout this specification, we will use the term "application callable" to mean "a function, a method, or an instance with a __call__ method". It is up to the server, gateway, or application implementing the application callable to choose the appropriate implementation technique for their needs. Conversely, a server, gateway, or application that is invoking a callable must not have any dependency on what kind of callable was provided to it. Application callables are only to be called, not introspected upon.

In addition to "pure" servers/gateways and applications/frameworks, it is also possible to create "middleware" components that implement both sides of this specification. Such components act as an application to their containing server, and as a server to a contained application, and can be used to provide extended APIs, content transformation, navigation, and other useful functions.

The Web3 interface has two sides: the "server" or "gateway" side, and the "application" or "framework" side. The server side invokes a callable object that is provided by the application side. The specifics of how that object is provided are up to the server or gateway. It is assumed that some servers or gateways will require an application's deployer to write a short script to create an instance of the server or gateway, and supply it with the application object. Other servers and gateways may use configuration files or other mechanisms to specify where an application object should be imported from, or otherwise obtained.

The application callable must accept one positional argument. For the sake of illustration, we have named it environ , but it is not required to have this name. A server or gateway must invoke the application object using a positional (not keyword) argument. (E.g. by calling body, status, headers = application(environ) as shown above.)

The environ parameter is a dictionary object, containing CGI-style environment variables. This object must be a builtin Python dictionary (not a subclass, UserDict or other dictionary emulation), and the application is allowed to modify the dictionary in any way it desires. The dictionary must also include certain Web3-required variables (described in a later section), and may also include server-specific extension variables, named according to a convention that will be described below.

When called by the server, the application object must return a tuple yielding three elements: status , headers and body , or, if supported by an async server, an argumentless callable which either returns None or a tuple of those three elements.

The status element is a status in bytes of the form b'999 Message here' .

headers is a Python list of (header_name, header_value) pairs describing the HTTP response header. The headers structure must be a literal Python list; it must yield two-tuples. Both header_name and header_value must be bytes values.

The body is an iterable yielding zero or more bytes instances. This can be accomplished in a variety of ways, such as by returning a list containing bytes instances as body , or by returning a generator function as body that yields bytes instances, or by the body being an instance of a class which is iterable. Regardless of how it is accomplished, the application object must always return a body iterable yielding zero or more bytes instances.

The server or gateway must transmit the yielded bytes to the client in an unbuffered fashion, completing the transmission of each set of bytes before requesting another one. (In other words, applications should perform their own buffering. See the Buffering and Streaming section below for more on how application output must be handled.)

The server or gateway should treat the yielded bytes as binary byte sequences: in particular, it should ensure that line endings are not altered. The application is responsible for ensuring that the string(s) to be written are in a format suitable for the client. (The server or gateway may apply HTTP transfer encodings, or perform other transformations for the purpose of implementing HTTP features such as byte-range transmission. See Other HTTP Features, below, for more details.)

If the body iterable returned by the application has a close() method, the server or gateway must call that method upon completion of the current request, whether the request was completed normally, or terminated early due to an error. This is to support resource release by the application amd is intended to complement PEP 325's generator support, and other common iterables with close() methods.

Finally, servers and gateways must not directly use any other attributes of the body iterable returned by the application.

environ Variables The environ dictionary is required to contain various CGI environment variables, as defined by the Common Gateway Interface specification . The following CGI variables must be present. Each key is a native string. Each value is a bytes instance. Note In Python 3.1+, a "native string" is a str type decoded using the surrogateescape error handler, as done by os.environ.__getitem__ . In Python 2.6 and 2.7, a "native string" is a str types representing a set of bytes. REQUEST_METHOD The HTTP request method, such as "GET" or "POST" . SCRIPT_NAME The initial portion of the request URL's "path" that corresponds to the application object, so that the application knows its virtual "location". This may be the empty bytes instance if the application corresponds to the "root" of the server. SCRIPT_NAME will be a bytes instance representing a sequence of URL-encoded segments separated by the slash character ( / ). It is assumed that %2F characters will be decoded into literal slash characters within PATH_INFO , as per CGI. PATH_INFO The remainder of the request URL's "path", designating the virtual "location" of the request's target within the application. This may be a bytes instance if the request URL targets the application root and does not have a trailing slash. PATH_INFO will be a bytes instance representing a sequence of URL-encoded segments separated by the slash character ( / ). It is assumed that %2F characters will be decoded into literal slash characters within PATH_INFO , as per CGI. QUERY_STRING The portion of the request URL (in bytes) that follows the "?" , if any, or the empty bytes instance. SERVER_NAME , SERVER_PORT When combined with SCRIPT_NAME and PATH_INFO (or their raw equivalents)`, these variables can be used to complete the URL. Note, however, that HTTP_HOST , if present, should be used in preference to SERVER_NAME for reconstructing the request URL. See the URL Reconstruction section below for more detail. SERVER_PORT should be a bytes instance, not an integer. SERVER_PROTOCOL The version of the protocol the client used to send the request. Typically this will be something like "HTTP/1.0" or "HTTP/1.1" and may be used by the application to determine how to treat any HTTP request headers. (This variable should probably be called REQUEST_PROTOCOL , since it denotes the protocol used in the request, and is not necessarily the protocol that will be used in the server's response. However, for compatibility with CGI we have to keep the existing name.) The following CGI values may present be in the Web3 environment. Each key is a native string. Each value is a bytes instances. CONTENT_TYPE The contents of any Content-Type fields in the HTTP request. CONTENT_LENGTH The contents of any Content-Length fields in the HTTP request. HTTP_ Variables Variables corresponding to the client-supplied HTTP request headers (i.e., variables whose names begin with "HTTP_" ). The presence or absence of these variables should correspond with the presence or absence of the appropriate HTTP header in the request. A server or gateway should attempt to provide as many other CGI variables as are applicable, each with a string for its key and a bytes instance for its value. In addition, if SSL is in use, the server or gateway should also provide as many of the Apache SSL environment variables as are applicable, such as HTTPS=on and SSL_PROTOCOL . Note, however, that an application that uses any CGI variables other than the ones listed above are necessarily non-portable to web servers that do not support the relevant extensions. (For example, web servers that do not publish files will not be able to provide a meaningful DOCUMENT_ROOT or PATH_TRANSLATED .) A Web3-compliant server or gateway should document what variables it provides, along with their definitions as appropriate. Applications should check for the presence of any variables they require, and have a fallback plan in the event such a variable is absent. Note that CGI variable values must be bytes instances, if they are present at all. It is a violation of this specification for a CGI variable's value to be of any type other than bytes . On Python 2, this means they will be of type str . On Python 3, this means they will be of type bytes . They keys of all CGI and non-CGI variables in the environ, however, must be "native strings" (on both Python 2 and Python 3, they will be of type str ). In addition to the CGI-defined variables, the environ dictionary may also contain arbitrary operating-system "environment variables", and must contain the following Web3-defined variables. Variable Value web3.version The tuple (1, 0) , representing Web3 version 1.0. web3.url_scheme A bytes value representing the "scheme" portion of the URL at which the application is being invoked. Normally, this will have the value b"http" or b"https" , as appropriate. web3.input An input stream (file-like object) from which bytes constituting the HTTP request body can be read. (The server or gateway may perform reads on-demand as requested by the application, or it may pre- read the client's request body and buffer it in-memory or on disk, or use any other technique for providing such an input stream, according to its preference.) web3.errors An output stream (file-like object) to which error output text can be written, for the purpose of recording program or other errors in a standardized and possibly centralized location. This should be a "text mode" stream; i.e., applications should use "

" as a line ending, and assume that it will be converted to the correct line ending by the server/gateway. Applications may not send bytes to the 'write' method of this stream; they may only send text. For many servers, web3.errors will be the server's main error log. Alternatively, this may be sys.stderr , or a log file of some sort. The server's documentation should include an explanation of how to configure this or where to find the recorded output. A server or gateway may supply different error streams to different applications, if this is desired. web3.multithread This value should evaluate true if the application object may be simultaneously invoked by another thread in the same process, and should evaluate false otherwise. web3.multiprocess This value should evaluate true if an equivalent application object may be simultaneously invoked by another process, and should evaluate false otherwise. web3.run_once This value should evaluate true if the server or gateway expects (but does not guarantee!) that the application will only be invoked this one time during the life of its containing process. Normally, this will only be true for a gateway based on CGI (or something similar). web3.script_name The non-URL-decoded SCRIPT_NAME value. Through a historical inequity, by virtue of the CGI specification, SCRIPT_NAME is present within the environment as an already URL-decoded string. This is the original URL-encoded value derived from the request URI. If the server cannot provide this value, it must omit it from the environ. web3.path_info The non-URL-decoded PATH_INFO value. Through a historical inequity, by virtue of the CGI specification, PATH_INFO is present within the environment as an already URL-decoded string. This is the original URL-encoded value derived from the request URI. If the server cannot provide this value, it must omit it from the environ. web3.async This is True if the webserver supports async invocation. In that case an application is allowed to return a callable instead of a tuple with the response. The exact semantics are not specified by this specification. Finally, the environ dictionary may also contain server-defined variables. These variables should have names which are native strings, composed of only lower-case letters, numbers, dots, and underscores, and should be prefixed with a name that is unique to the defining server or gateway. For example, mod_web3 might define variables with names like mod_web3.some_variable . Input Stream The input stream ( web3.input ) provided by the server must support the following methods: Method Notes read(size) 1,4 readline([size]) 1,2,4 readlines([size]) 1,3,4 __iter__() 4 The semantics of each method are as documented in the Python Library Reference, except for these notes as listed in the table above: The server is not required to read past the client's specified Content-Length , and is allowed to simulate an end-of-file condition if the application attempts to read past that point. The application should not attempt to read more data than is specified by the CONTENT_LENGTH variable. The implementation must support the optional size argument to readline() . The application is free to not supply a size argument to readlines() , and the server or gateway is free to ignore the value of any supplied size argument. The read , readline and __iter__ methods must return a bytes instance. The readlines method must return a sequence which contains instances of bytes. The methods listed in the table above must be supported by all servers conforming to this specification. Applications conforming to this specification must not use any other methods or attributes of the input object. In particular, applications must not attempt to close this stream, even if it possesses a close() method. The input stream should silently ignore attempts to read more than the content length of the request. If no content length is specified the stream must be a dummy stream that does not return anything. Error Stream The error stream ( web3.errors ) provided by the server must support the following methods: Method Stream Notes flush() errors 1 write(str) errors 2 writelines(seq) errors 2 The semantics of each method are as documented in the Python Library Reference, except for these notes as listed in the table above: Since the errors stream may not be rewound, servers and gateways are free to forward write operations immediately, without buffering. In this case, the flush() method may be a no-op. Portable applications, however, cannot assume that output is unbuffered or that flush() is a no-op. They must call flush() if they need to ensure that output has in fact been written. (For example, to minimize intermingling of data from multiple processes writing to the same error log.) The write() method must accept a string argument, but needn't necessarily accept a bytes argument. The writelines() method must accept a sequence argument that consists entirely of strings, but needn't necessarily accept any bytes instance as a member of the sequence. The methods listed in the table above must be supported by all servers conforming to this specification. Applications conforming to this specification must not use any other methods or attributes of the errors object. In particular, applications must not attempt to close this stream, even if it possesses a close() method.

Values Returned by A Web3 Application Web3 applications return a tuple in the form ( status , headers , body ). If the server supports asynchronous applications ( web3.async ), the response may be a callable object (which accepts no arguments). The status value is assumed by a gateway or server to be an HTTP "status" bytes instance like b'200 OK' or b'404 Not Found' . That is, it is a string consisting of a Status-Code and a Reason-Phrase, in that order and separated by a single space, with no surrounding whitespace or other characters. (See RFC 2616, Section 6.1.1 for more information.) The string must not contain control characters, and must not be terminated with a carriage return, linefeed, or combination thereof. The headers value is assumed by a gateway or server to be a literal Python list of (header_name, header_value) tuples. Each header_name must be a bytes instance representing a valid HTTP header field-name (as defined by RFC 2616, Section 4.2), without a trailing colon or other punctuation. Each header_value must be a bytes instance and must not include any control characters, including carriage returns or linefeeds, either embedded or at the end. (These requirements are to minimize the complexity of any parsing that must be performed by servers, gateways, and intermediate response processors that need to inspect or modify response headers.) In general, the server or gateway is responsible for ensuring that correct headers are sent to the client: if the application omits a header required by HTTP (or other relevant specifications that are in effect), the server or gateway must add it. For example, the HTTP Date: and Server: headers would normally be supplied by the server or gateway. The gateway must however not override values with the same name if they are emitted by the application. (A reminder for server/gateway authors: HTTP header names are case-insensitive, so be sure to take that into consideration when examining application-supplied headers!) Applications and middleware are forbidden from using HTTP/1.1 "hop-by-hop" features or headers, any equivalent features in HTTP/1.0, or any headers that would affect the persistence of the client's connection to the web server. These features are the exclusive province of the actual web server, and a server or gateway should consider it a fatal error for an application to attempt sending them, and raise an error if they are supplied as return values from an application in the headers structure. (For more specifics on "hop-by-hop" features and headers, please see the Other HTTP Features section below.)

Dealing with Compatibility Across Python Versions Creating Web3 code that runs under both Python 2.6/2.7 and Python 3.1+ requires some care on the part of the developer. In general, the Web3 specification assumes a certain level of equivalence between the Python 2 str type and the Python 3 bytes type. For example, under Python 2, the values present in the Web3 environ will be instances of the str type; in Python 3, these will be instances of the bytes type. The Python 3 bytes type does not possess all the methods of the Python 2 str type, and some methods which it does possess behave differently than the Python 2 str type. Effectively, to ensure that Web3 middleware and applications work across Python versions, developers must do these things: Do not assume comparison equivalence between text values and bytes values. If you do so, your code may work under Python 2, but it will not work properly under Python 3. For example, don't write somebytes == 'abc' . This will sometimes be true on Python 2 but it will never be true on Python 3, because a sequence of bytes never compares equal to a string under Python 3. Instead, always compare a bytes value with a bytes value, e.g. "somebytes == b'abc'". Code which does this is compatible with and works the same in Python 2.6, 2.7, and 3.1. The b in front of 'abc' signals to Python 3 that the value is a literal bytes instance; under Python 2 it's a forward compatibility placebo. Don't use the __contains__ method (directly or indirectly) of items that are meant to be byteslike without ensuring that its argument is also a bytes instance. If you do so, your code may work under Python 2, but it will not work properly under Python 3. For example, 'abc' in somebytes' will raise a TypeError under Python 3, but it will return True under Python 2.6 and 2.7. However, b'abc' in somebytes will work the same on both versions. In Python 3.2, this restriction may be partially removed, as it's rumored that bytes types may obtain a __mod__ implementation. __getitem__ should not be used. Don't try to use the format method or the __mod__ method of instances of bytes (directly or indirectly). In Python 2, the str type which we treat equivalently to Python 3's bytes supports these method but actual Python 3's bytes instances don't support these methods. If you use these methods, your code will work under Python 2, but not under Python 3. Do not try to concatenate a bytes value with a string value. This may work under Python 2, but it will not work under Python 3. For example, doing 'abc' + somebytes will work under Python 2, but it will result in a TypeError under Python 3. Instead, always make sure you're concatenating two items of the same type, e.g. b'abc' + somebytes . Web3 expects byte values in other places, such as in all the values returned by an application. In short, to ensure compatibility of Web3 application code between Python 2 and Python 3, in Python 2, treat CGI and server variable values in the environment as if they had the Python 3 bytes API even though they actually have a more capable API. Likewise for all stringlike values returned by a Web3 application.

Buffering and Streaming Generally speaking, applications will achieve the best throughput by buffering their (modestly-sized) output and sending it all at once. This is a common approach in existing frameworks: the output is buffered in a StringIO or similar object, then transmitted all at once, along with the response headers. The corresponding approach in Web3 is for the application to simply return a single-element body iterable (such as a list) containing the response body as a single string. This is the recommended approach for the vast majority of application functions, that render HTML pages whose text easily fits in memory. For large files, however, or for specialized uses of HTTP streaming (such as multipart "server push"), an application may need to provide output in smaller blocks (e.g. to avoid loading a large file into memory). It's also sometimes the case that part of a response may be time-consuming to produce, but it would be useful to send ahead the portion of the response that precedes it. In these cases, applications will usually return a body iterator (often a generator-iterator) that produces the output in a block-by-block fashion. These blocks may be broken to coincide with multipart boundaries (for "server push"), or just before time-consuming tasks (such as reading another block of an on-disk file). Web3 servers, gateways, and middleware must not delay the transmission of any block; they must either fully transmit the block to the client, or guarantee that they will continue transmission even while the application is producing its next block. A server/gateway or middleware may provide this guarantee in one of three ways: Send the entire block to the operating system (and request that any O/S buffers be flushed) before returning control to the application, OR Use a different thread to ensure that the block continues to be transmitted while the application produces the next block. (Middleware only) send the entire block to its parent gateway/server. By providing this guarantee, Web3 allows applications to ensure that transmission will not become stalled at an arbitrary point in their output data. This is critical for proper functioning of e.g. multipart "server push" streaming, where data between multipart boundaries should be transmitted in full to the client.

Unicode Issues HTTP does not directly support Unicode, and neither does this interface. All encoding/decoding must be handled by the application; all values passed to or from the server must be of the Python 3 type bytes or instances of the Python 2 type str , not Python 2 unicode or Python 3 str objects. All "bytes instances" referred to in this specification must: On Python 2, be of type str .

. On Python 3, be of type bytes . All "bytes instances" must not : On Python 2, be of type unicode .

. On Python 3, be of type str . The result of using a textlike object where a byteslike object is required is undefined. Values returned from a Web3 app as a status or as response headers must follow RFC 2616 with respect to encoding. That is, the bytes returned must contain a character stream of ISO-8859-1 characters, or the character stream should use RFC 2047 MIME encoding. On Python platforms which do not have a native bytes-like type (e.g. IronPython, etc.), but instead which generally use textlike strings to represent bytes data, the definition of "bytes instance" can be changed: their "bytes instances" must be native strings that contain only code points representable in ISO-8859-1 encoding ( \u0000 through \u00FF , inclusive). It is a fatal error for an application on such a platform to supply strings containing any other Unicode character or code point. Similarly, servers and gateways on those platforms must not supply strings to an application containing any other Unicode characters.

HTTP 1.1 Expect/Continue Servers and gateways that implement HTTP 1.1 must provide transparent support for HTTP 1.1's "expect/continue" mechanism. This may be done in any of several ways: Respond to requests containing an Expect: 100-continue request with an immediate "100 Continue" response, and proceed normally. Proceed with the request normally, but provide the application with a web3.input stream that will send the "100 Continue" response if/when the application first attempts to read from the input stream. The read request must then remain blocked until the client responds. Wait until the client decides that the server does not support expect/continue, and sends the request body on its own. (This is suboptimal, and is not recommended.) Note that these behavior restrictions do not apply for HTTP 1.0 requests, or for requests that are not directed to an application object. For more information on HTTP 1.1 Expect/Continue, see RFC 2616, sections 8.2.3 and 10.1.1.

Other HTTP Features In general, servers and gateways should "play dumb" and allow the application complete control over its output. They should only make changes that do not alter the effective semantics of the application's response. It is always possible for the application developer to add middleware components to supply additional features, so server/gateway developers should be conservative in their implementation. In a sense, a server should consider itself to be like an HTTP "gateway server", with the application being an HTTP "origin server". (See RFC 2616, section 1.3, for the definition of these terms.) However, because Web3 servers and applications do not communicate via HTTP, what RFC 2616 calls "hop-by-hop" headers do not apply to Web3 internal communications. Web3 applications must not generate any "hop-by-hop" headers , attempt to use HTTP features that would require them to generate such headers, or rely on the content of any incoming "hop-by-hop" headers in the environ dictionary. Web3 servers must handle any supported inbound "hop-by-hop" headers on their own, such as by decoding any inbound Transfer-Encoding , including chunked encoding if applicable. Applying these principles to a variety of HTTP features, it should be clear that a server may handle cache validation via the If-None-Match and If-Modified-Since request headers and the Last-Modified and ETag response headers. However, it is not required to do this, and the application should perform its own cache validation if it wants to support that feature, since the server/gateway is not required to do such validation. Similarly, a server may re-encode or transport-encode an application's response, but the application should use a suitable content encoding on its own, and must not apply a transport encoding. A server may transmit byte ranges of the application's response if requested by the client, and the application doesn't natively support byte ranges. Again, however, the application should perform this function on its own if desired. Note that these restrictions on applications do not necessarily mean that every application must reimplement every HTTP feature; many HTTP features can be partially or fully implemented by middleware components, thus freeing both server and application authors from implementing the same features over and over again.