Interactions in Lisp This paper appears in

Lisp Evolution and Standardization,

Proceedings of the First International Workshop,

C. Queinnec and J. Chailloux, editors,

AFCET, IOS, February 22-23, Paris France, 1988. The substance of the original text appears in normal fonting, though a small number of out-and-out typos were corrected. Oddities of spelling that were the custom of the time (either generally, or just for me) were left alone. Formatting of any headings and/or tables may have been adjusted slightly for HTML. Any new text that has been added appears bracketed and in color green; such text is intended to help clarify the historical context, since considerable time passed between the time this paper was published and the time I converted it to HTML.

Special thanks to Stephanie Peteranecz for bringing the text of this paper back online after its original source had been lost.

--Kent Pitman, 18-Feb-2001. Annotated original document follows.

Click here for an index of other titles by Kent Pitman.

Interactions in Lisp Kent M. Pitman

Symbolics, Inc.

11 Cambridge Center

Cambridge, MA 02142 USA [This address is, not surprisingly, obsolete. If you need to reach me, send me e-mail.

--KMP 18-Feb-2001]

Abstract

This short paper discusses various aspects of Lisp's interaction with the outside world, including file systems, error systems, and window systems. Observations are made about the current state of Common Lisp and its interactions with the "outside." Suggestions are made about possible future directions for Lisp evolution.

Introduction

It should be obvious to any student of Common LISP: The Language (CLtL) [Steele 84] that Common Lisp is not so much a language as it is a constraint on a language. The fact that a myriad of technical details are left unspecified means that every actual implementation of present day Common Lisp must have significant implementation-dependent aspects. This implies that there can be no implementation will run in any other implementation.

Of the many issues left vague by CLtL, some are related to data representation or algorithms, which may need to vary between implementations. The site-effect behavior of operators like DELETE and REMPROP is not fully specified, for example. Also, various issues of sharing between structures created by the QUOTE special form and the backquote facility are left up to the implementation.

Although significant, these issues do not preempt the programmer's ability to use the associated facilities because the parts that are left vague are not necessary to the useful employment of those facilities. The lack of specification is an important use of information hiding which for the most part adds strength, rather than weakness, to the Common Lisp language.

In contrast, however, some things left vague in CLtL cannot be justified as desirable information hiding. These are things which programmers frequently do need to know, but which were not possible for the designers of Common Lisp to agree upon prior to the publication of CLtL. They include interfaces to the file system, where a standard exists but needs repair; the error system, where a standard is ready to be proposed; and the window system, where considerable design work remains to be done before a standard can be considered.

Vendors have tried independently to fill the gaps left by CLtL, and many of their solutions have been quite creative. But rarely have they been compatible with one another. In the end, developers of portable application programs are always left to answer the same important question:

"Do I settle for the least common denominator for the sake of portability, or do I use the best of what each system provides at the cost of maintaining implementation-dependent code?"

The File System

A Standard in Need of Repair

Many users are surprised to find how ill defined the CLtL file system interface really is. This is probably because most implementations make good faith effort to provide a useful interpretation of the fairly vague terminology which the pathname chapter contains. As such, the programmer often doesn't realize how much he is taking for granted until he tries to port his application to another implementation, particularly one provided by a different vendor on a different operating system.

Common Lisp is not entirely to blame here. It would have been very desirable to say that all file systems have hierarchical directories; that all filenames have at least 32 characters; that all filenames have a name, a type, and a version; or that all file systems use upper (or lower) case as canonical in filenames. But file systems are separate from Lisp in most implementations; they come with the operating system and they have fixed, unmodifiable constraints that make the specification of a portable standard very complicated.

In order to present the illusion of added structure, Common Lisp could have simulated types or version numbers for file systems which didn't provide them primitively, but in cases where Lisp needed to cooperate with other programs that did not analogous simulation, many problems would probably have arisen.

The Lisp Machine community had dealt successfully with "generic file systems" for years, but only by providing a much larger set of primitives than the Common Lisp designers were willing to accept. Although the Common Lisp pathname system was based on the Lisp Machines, in fact that it was only a subset proved to be a problem.

Given the degree of variation in file systems, there were really only two ways in which the Common Lisp file system interface could have been done better:

The designers of Common Lisp could have decided that the problem was too complicated for a serious interface to be provided and could have provided a radically simpler interface. For example, they might have defined that functions like LOAD and OPEN takes only strings and might have asserted that such strings are inherently implementation-dependent and must be dealt with using sharpsign conditionals. (Present day reality isn't far from this.)

The designers of Common Lisp could have opted to provide primitives and functionally more like what is provided on Lisp Machines. However, there were many who believed that Common Lisp was already too large; it is unlikely that any additional complexity could have gotten past the committee.

Many of the problems with the Common Lisp file system interface were not anticipated by the designers. This is probably because implementors were initially concerned primarily with their own interfaces. They read the manual and implemented that they believed was a true interpretation of what it said. Only when users began to port programs from implementation to implementation did the degree of divergence become apparent.

The Condition System

Ready for Standardization

Traditionally, Lisp and most other languages have treated program errors as things which should simply never happen. If you got an error, your program halted; there was nothing more to be done. It is this hard-line of errors which is to a large extent responsible for the dim view that the public has traditionally taken towards computers. The traditional stereotype of a computer which, given a large problem that contains only a minor detail in error, simply says, "It does not compute!" (rather than being more fault tolerant) is not far from the truth.

About all that CLtL says about errors is that they are "handled in an implementation-dependent way." It suggests that each implementation have an interactive debugger although it offers no further detail. It also suggests that implementors provide a means for trapping errors, but again it gives no detailed advice.

This leaves writers of large, portable programs in a quandary because most of the functions in CLtL are willing under some circumstances to signal an error. In a few cases, there is no defined way to test in advance for those circumstances in order to avoid them. (Various floating-point arithmetic and file system errors are common examples.)

Fortunately, the state of the art in handling Lisp error conditions has improved dramatically in the past five years, and Common Lisp is now in a position to adopt a condition system which will substantially improve the condition handling capabilities of user programs. The system being developed [Pitman 87] is a modification of a design originated by Weinreb and others at Symbolics in 1983.

In this system, an error is signaled when a program wants to admit to the "outside world" that it is struck and does not know how to usefully proceed. No further computation can occur without external intervention.

Programs may at any time establish restart points to which control may be transferred in the event of trouble. At the point where an error occurs, there is an effective branch in the road with typically several restart points to which control might be transferred. Other programs can handle the condition by providing advice about which branch to take. As soon as a selection is made, the process of handling is complete and the original program continues.

The error condition is represented by a structured object which holds information about the situation. The signaling protocol is standardized in a way that allows other programs to conveniently receive information about a condition being signaled and the restarts available in that context. Those programs can then provide advice about how to proceed. If no program offers advice about how to proceed, handling is attempted interactively using the debugger.

Some important features of this condition system are:

The representation of conditions as objects is standardized.

A hierarchy of condition types is provided. Simple type discrimination is a common way to decide whether and how to proceed from various errors.

Information about how the condition object is displayed is separated from the structure of the object, so handlers need not parse error message strings.

In some older Lisp dialects, certain kinds of errors could not be signaled unless the signaler was prepared to handle a particular style of restarting. In this system, the process of signaling and the process of establishing restarts are usefully decoupled.

Using the condition system requires no special understanding of Lisp internals. The program interface is surprisingly portable.

The system has been generalized to allow the notion of non-error (non-fatal) conditions, which do not require user intervention even if unhandled.

Graphics and Windows

Standard Needed, But None is Sight

More and more programmers have at their disposal a bit-mapped screen, a state-of-the-art pointing device, and perhaps even a full window system. In spite of this, the set of display operations available in Common Lisp is extremely limited. The only operations defined are those which are fully compatible with a screen printing terminal interface. The programmer cannot even reliably clear the screen using only Common Lisp primitives, much less read a cursor position, draw a box, or accept mouse input.

There are competing theories about how windows and graphics should be introduced into Common Lisp.

Common Lisp could provide a standard Lisp-level interface to a low level console device control protocol such as X . This is the approach taken by those involved in the CLX effort. It would assure a strong base on which to build portable programs which use windows and graphics. It would also simplify the task of the documentation writer because only a single interface style would generally need to be documented. There are two main disadvantages to this approach. The first is that the X protocol is too low level. Even its strong proponents agree that at least one and perhaps several layers of abstraction need to be built upon it before a level is reached which is practical for day-to- day programming.

The second disadvantage is that it causes an application to have the same look on all systems, rather than allowing its interface to adapt to the interface style of the host system. This tends to thwart the desire of hardware or operating system vendors to provide a "uniform look" for all products that run in their environment.

Alternatively, Common Lisp could provide a very abstract interface to the window system. This is the approach taken by; the CommonWindows effort. This approach is likely to appeal most to users because it provides the Lisp system with the greatest latitude in producing an interface that is culturally compatible with other programs running on that same machine.

The key disadvantages are that there is less opportunity for the application designer to do any fine tuning of the interface, and that documentation for portable applications may be considerably more complicated since the interface could have a very different "feel" in each environment.

Given the tremendous market pressure to find a solution to the window system problem in the near term, and given the Common Lisp tradition for compromise, it seems likely that the ultimate result will be a hybrid one.

Probably layers such as the following will continue to be added from low level to high level as they can be agreed upon:

Low Level Device Control Protocol

Interactive Stream Protocol

Output Graphics Protocol

Window System or User Interface Management System

At the same time, certain high level interfaces will probably be provided directly. For example, one could easily imagine the Common Lisp designers moving to adopt facilities for high-level interfaces to common idiomatic tasks, such as simple menus, multiple- choice menus, box drawing, and table formatting.

Those who could not make do with these high level interfaces would presumably fall back on whatever layer of the emerging layered display protocol had been standardized upon the time the standard went to press, or sacrifice compatibility with the portable standard by suing a proprietary or "de facto" standard.

Even once the display protocols are worked out, there is a long list of other issues related to windows which have been swept under the rug by the current Common Lisp design but which show up often enough in everyday programming to deserve explicit treatment in some future standard. Among these are the question of newline confirmation of input, the echoing behavior of interactive READ-CHAR , input editing, character sets, fonts, multi-processing (including issues of memory sharing), locks, and asynchronous interrupts.

Conclusion

Common Lisp has traditionally attempted to stay out of the business of specifying "programming environments." Yet the constant pressure from users to provide an environment in which complex applications can be delivered portably is somewhat in conflict with this desire.

The new Common Lisp condition system leaves many decisions still open to the implementation, but moves a step closer to changing Common Lisp from a programming language to a complete programming environment. The various window systems proposals which are being considered still continue this trend.

Those who think Common Lisp is already too big should brace themselves for things yet to come.

Acknowledgements

Conversations with Mike McMahon (Symbolics) and Bob Scheifler (MIT) were quite helpful in fleshing out my understanding of the current state of affairs in the area of window systems. Sonya Keene, Bob Laddaga, and Mike McMahon read drafts of this paper and provided many useful comments.

References

Original printed text document

Copyright 1988, Kent M. Pitman. All Rights Reserved. HTML hypertext version of document

Copyright 2001, Kent M. Pitman. All rights reserved.

The following limited, non-exclusive, revokable licenses are granted: Browsing of this document (that is, transmission and display of a temporary copy of this document for the ordinary purpose of direct viewing by a human being in the usual manner that hypertext browsers permit such viewing) is expressly permitted, provided that no recopying, redistribution, redisplay, or retransmission is made of any such copy. Bookmarking of this document (that is, recording only the document's title and Uniform Resource Locator, or URL, but not its content, for the purpose of remembering an association between the document's title and the URL, and/or for the purpose of making a subsequent request for a fresh copy of the content named by that URL) is also expressly permitted. All other uses require negotiated permission.

Click here for an index of other titles by Kent Pitman.