ATLAST

Autodesk Threaded Language Application System Toolkit

by John Walker



Atlast is an attempt to make software component technology and open architecture applications commonplace in the mainstream software market. It is both a software component which can be readily integrated into existing applications, providing them a ready-made macro language and facilities for user extension and customisation and, at the same time, it is a foundation upon which new applications can be built in an open, component-oriented manner.

Atlast was developed at Autodesk, Inc. Autodesk returned the rights to me in 1991, and I subsequently placed the program in the public domain. Autodesk's connection with this program is purely historical: it is neither endorsed, used, nor supported by Autodesk, Inc.

Atlast is based upon the FORTH-83 language, but has been extended in many ways and modified to better serve its mission as an embedded toolkit for open, programmable applications. Atlast is implemented in a single file, written in portable C; it has been ported to many different machines and operating systems, including MS-DOS, OS/2, the Macintosh, and a wide variety of Unix machines. Atlast includes native support for floating point, C-like strings, Unix-compatible file access, and a wide variety of facilities for embedding within applications. Integers are 32 bits (64 bits in the 64-bit version of Atlast) and identifiers can be up to 127 characters; extensive stack and heap pointer checking is available to aid in debugging. Atlast may be configured at compilation time to include only the facilities needed by a given application, thus saving memory and increasing execution speed (when error checking is disabled).

The following extract from the Atlast manual describes the rationale for the development of Atlast and its intended scope of applications.

Virtually every industry analyst agrees that open architecture is essential to the success of applications. And yet, even today, we write program after program that is closed—that its users cannot program—that admits of no extensions without our adding to its source code. If we believe intellectually, from a sound understanding of the economic incentives in the marketplace, that open systems are better, and have seen this belief confirmed repeatedly in the marketplace, then the only question that remains is why? Why not make every program an open program?

Well, because it's hard! Writing a closed program has traditionally been much less work at every stage of the development cycle: easier to design, less code to write, simpler documentation, and far fewer considerations in the test phase. In addition, closed products are believed to be less demanding of support, although I'll argue later that this assumption may be incorrect.

The painful path to programmability

Most programs start out as nonprogrammable, closed applications, then painfully claw their way to programmability through the introduction of a limited script or macro facility, succeeded by an increasingly comprehensive interpretive macro language which grows like topsy and without a coherent design as user demands upon it grow. Finally, perhaps, the program is outfitted with bindings to existing languages such as C.

An alternative to this is adopting a standard language as the macro language for a product. After our initial foray into the awful menu macro language that still burdens us, AutoCAD took this approach, integrating David Betz’ XLISP, a simple Lisp interpreter which was subsequently extended by Autodesk to add floating point, many additional Common Lisp functions, and, eventually, access to the AutoCAD database.

This approach has many attractions. First, choosing a standard language allows users to avail themselves of existing books and training resources to learn its basics. The developer of a dedicated macro language must create all this material from scratch. Second, an interpretive language, where all programs are represented in ASCII code, is inherently portable across computers and operating systems. Once the interpreter is gotten to work on a new system, all the programs it supports are pretty much guaranteed to work. Third, most existing languages have evolved to the point that most of the rough edges have been taken off their design. Extending an existing language along the lines laid down by its designers is much less likely to result in an incomprehensible disaster than growing an ad-hoc macro language feature by neat-o feature.

Unfortunately, interpreters are slow, slow, slow. A simple calculation of the number of instructions of overhead per instruction that furthers the execution of the program quickly demonstrates that no interpreter is suitable for serious computation. As long as the interpreter is deployed in the role of a macro language, this may not be a substantial consideration. Most early AutoLISP® programs, for example, spent most of their time submitting commands to AutoCAD with the (command) function. The execution time of the program was overwhelmingly dominated by the time AutoCAD took to perform the commands, not the time AutoLISP spent constructing and submitting them. However, as soon as applications tried to do substantial computation, for example the parametric object calculations in AutoCAD AEC, the overhead of AutoLISP became a crushing burden, verging on intolerable. The obvious alternative was to provide a compiled language. But that, too, has its problems.

Introducing Atlast

Atlast™ is a toolkit that makes applications programmable. Deliberately designed to be easy to integrate both into existing programs and newly-developed ones, Atlast provides any program that incorporates it most of the benefits of programmability with very little explicit effort on the part of the developer. Indeed, once you begin to “think Atlast” as part of the design cycle, you'll probably find that the way you design and build programs changes substantially. I'm coming to think of Atlast as the “monster that feeds on programs,” because including it in a program tends to shrink the amount of special-purpose code that would otherwise have to be written while resulting in finished applications that are open, extensible, and more easily adapted to other operating environments such as the event driven paradigm.

The idea of a portable toolkit, integrated into a wide variety of products, all of which thereby share a common programming language seems obvious once you consider its advantages. It's surprising that such packages aren't commonplace in the industry. In fact, the only true antecedent to Atlast I've encountered in my whole twisted path through this industry was the universal macro package developed in the mid 1970s by Kern Sibbald and Ben Cranston at the University of Maryland. That package, implemented on Univac mainframes, provided a common macro language shared by a wide variety of University of Maryland utilities, including a text editor, debugger, file dumper, and typesetting language. While Atlast is entirely different in structure and operation from the Maryland package, which was an interpretive string language, the concept of a cross-product macro language and appreciation of the benefits to be had from such a package are directly traceable to those roots.

Summary and Conclusions

Everything should be programmable. Everything! I have come to the conclusion that to write almost any program in a closed manner is a mistake that invites the expenditure of uncounted hours “enhancing” it over its life cycle. Further tweaks, “features,” and “fixes” often result in a product so massive and incomprehensible that it becomes unlearnable, unmaintainable, and eventually unusable.

Far better to invest the effort up front to create a product flexible enough to be adapted at will, by its users, to their immediate needs. If the product is programmable in a portable, open form, user extensions can be exchanged, compared, reviewed by the product developer, and eventually incorporated into the mainstream of the product.

It is far, far better to have thousands of creative users expanding the scope of one's product in ways the original developers didn't anticipate—in fact, working for the vendor without pay, than it is to have thousands of frustrated users writing up wish list requests that the vendor can comply with only by hiring people and paying them to try to accommodate the perceived needs of the users. Open architecture and programmability not only benefits the user, not only makes a product better in the technical and marketing sense, but confers a direct economic advantage upon the vendor of such a product—one mirrored in a commensurate disadvantage to the vendor of a closed product.

The chief argument against programmability has been the extra investment needed to create open products. Atlast provides a way of building open products in the same, or less, time than it takes to construct closed ones. Just as no C programmer in his right mind would sit down and write his own buffered file I/O package when a perfectly fine one was sitting in the library, why re-invent a macro language or other parameterisation and programming facility when there's one just sitting there that's as fast as native C code for all but the most absurd misapplications, takes very little memory with every gew-gaw and optional feature at its command enabled all at once, is portable to any machine that supports C by simply recompiling a single file, and can be integrated into a typical application at a basic level in less than 15 minutes?

Am I proposing that every application suddenly look like FORTH? Of course not; no more than output from PostScript printers looks like PostScript, or applications that run on 80386 processors resemble 80386 assembly language. Atlast is an intermediate language, seen only by those engaged in implementing and extending the product. Even then, Atlast is a chameleon which, with properly defined words, can look like almost anything you like, even at the primitive level of the interpreter.

Again and again, I have been faced with design situations where I knew that I really needed programmability, but didn't have the time, the memory, or the fortitude to face the problem squarely and solve it the right way. Instead, I ended up creating a kludge that continued to burden me through time. This is just a higher level manifestation of the nightmares perpetrated by old-time programmers who didn't have access to a proper dynamic memory allocator or linked list package. Just because programmability is the magic smoke of computing doesn't mean we should be spooked by the ghost in the machine or hesitant to confer its power upon our customers.

Don't think of Atlast as FORTH. Don't think of it as a language at all. The best way to think of Atlast is as a library routine that gives you programmability, in the same sense other libraries provide file access, window management, or graphics facilities. The whole concept of “programmability in a can” is odd—it took me two years to really got my end effector around it and crush it into submission. Think about it; play with it; and you may discover a better way to build applications.

Open is better. Atlast lets you build open programs in less time than you used to spend writing closed ones. Programs that inherit their open architecture from Atlast will share, across the entire product line and among all hardware platforms that support it, a common, clean, and efficient means of user extensibility. The potential benefits of this are immense.

32- or 64-bit?

Ever since its initial release in 1990, Atlast has supported a memory architecture in which integers and pointers were 32-bit values and floating point numbers were 64 bits. Because FORTH-derived languages allow the user access to memory at a low level, these assumptions were visible to the programmer and had consequences in code. Programs which used floating point numbers had to be conscious of the fact that floating point quantities occupied two stack items.

The advent of 64-bit systems poses a problem for such code but simultaneously makes it possible to dramatically simplify the memory architecture of Atlast. Rather than creating a kludge which would run in 64-bit mode while preserving compatibility with existing programs written for the 32-bit version, I have opted to create a new 64-bit version in which all data types—integers, pointers, and floating point numbers—are the same length. This means that floating point code can now use the same stack operators as integer code, and keeping track of the stack is much simpler. This comes at the cost of breaking source compatibility with existing programs, especially those which use floating point values. There is, however, a simple solution. Users with existing programs can continue to use them by building a 32-bit version of Atlast from the 1.x source code and compiling it in 32-bit memory mode on their 64-bit system (this may require installing some libraries, but is otherwise straightforward). Then their existing programs will continue to run without any modifications.

Users wishing to develop new code who are confident that all of the platforms on which it is deployed will be 64-bit may opt to use an Atlast 2.x release which uses the simpler memory architecture and provides a larger address space and integers. If you are developing code for an embedded platform with a 32-bit architecture, you should use a 1.x release even if you're developing on a 64-bit system, but unless you use floating point (which few embedded applications do), it's unlikely you'll have problems migrating source code between versions.

Download Atlast