Sun Apr 26 21:09:00 BST 2009

Portable CLX darcs repository has been updated so that the hello-world example can rely on the default font being usable. Thanks Christophe.

The source code files are gathered in a tarfile for easy downloading. It contains three extra source files: mixed-input.lisp and animation.lisp log.op.lisp for which I haven't yet written the explanations.

There may be some inconsistancies over file names and function names. I still haven't proof read this.

Source file:min-pop-up-window

First I try to pop up a window. I aim for the simplest possible program, and point out where it is misleading as to how to write a proper program.

The two central assumptions of the X protocol are

So, by the miracle of multi-tasking, your machine pretends to be two machines, client and server, talking to each other over the internal loopback interface. This might be a good time to check that it is actually running.

Well, that is OK, and it makes the point, though I suspect that my machine actually has clients talk to the local X server using the socket /tmp/.X11-unix/X0, for efficiency reasons.

How does the minimal program do the handshaking between client and server? It doesn't. It just assumes that the window has popped up, sleeps a few seconds, and then takes it down again whether it actually appeared or not.

The first argument life-time is how long the program waits between asking the X server to display the window, and taking it down again. If nothing happens, you can try a longer time.

The second argument lets you open the window on a different server. There is authorisation stuff to sort out before this works. It is probably best not to try it yet, but it makes the point that if you use X your code is network enabled right from the start. This does mean that you always have to start by connecting to your chosen display

I've not imported the display-roots symbol, neither via import nor use-package. So I'm using the package qualifier, making it clear what function calls are specific to X11. However, xlib:display-roots returns an ordinary list. There is no special xlib:first accessor required. I'm just using first from the common-lisp package. I could have used car .

Windows live in a hierarchical trees, with each window having a parent. Each window that is, except the special root window of each screen. We ask the X server to tell us the root window, so that we can pass it back as the parent of our own window.

This is the line you have been waiting for, that creates a window. :x 0 and :y 0 specify the top left. Don't worry if that is not where you want it to go. Since it is a top level window, the window manager will intervene and put it where it thinks best. The window manager can also overrule your chosen width and height, though that is less common.

Now we have created a window we can get fancy. We can extract the window's ID with xlib:drawable-id, email it to a machine on the other side of the world, which can connect to the X server over the internet and map the window, causing it to appear on the screen. Perhaps not.

At this point the map-window request is sitting in the client's queue, waiting to be sent over the network to the server. X11 is always buffering and queuing, caching and prevaricating, to minimise network traffic.

This command flushes the output buffer, sending the events and updating the event queue with any responses before proceding. This is misleading as to how to write a proper program, and the second example will use the buffer flushing built into the event-case macro instead.

Most of the time, the built-in buffer flushing works well, minimising network traffic without inconveniencing the programmer. Every so often it doesn't work how you want. Typically this is infrequently enough that you have forgotten all about the queuing, and procede to waste half an hour looking for the bug in all the wrong places. So it seems like a good idea to introduce display-force-output and display-finish-output straight away, because of the frustration that unflushed buffers cause if you are not aware of them, even though you seldom use these commands.

This goes to Lisp's standard output. It appears just below where you typed in (pop-up-window 5), hoping to get a window to pop up for five seconds.

If your program has passed its window onto a different program it has to tell the X server before it is killed, by setting the close-down-mode. There is a kill-temporary-clients command for when you are completely finished with the window.

More realistic code for popping up a window

Source file: Soap Bubble

(defun blow-bubble (&optional (host ""))

The awful kludge of a delay is gone. This function responds to events from the X server.

The previous minimalist code pops up a transparent window. That is taking the window metaphor too far. We want a black background. What pixel value gives black? Remember that it is no use for the client to find out the code for the black pixel locally. The machine it is running on might not even have a X server. You have to go over the network to ask the X server you are connected to.

(black (xlib:screen-black-pixel screen))

Having obtained the black pixel code, we use it in xlib:create-window

:background black

The exposure event is crucial. The X server doesn't take notes on what was drawn in your window. If your window gets covered over the contents are lost. When it is uncovered the X server sends an expose event. This is not just to say the window is up, you can draw something now. It is also the X server saying, "I've lost the contents, remind me what was supposed to be in the window."

:event-mask (xlib:make-event-mask :exposure :enter-window)

(xlib:event-case (display :force-output-p t :discard-p t) (:exposure ()(format t "Exposed~%")) (:enter-notify () t))

xlib:display-force-output

:force-output-p

:force-output-p

The xlib:event-case macro has two clever features

xlib:event-case has a notion of whether the handler code has ticked off the event as finished with, or whether it should be left in the event queue.

has a notion of whether the handler code has ticked off the event as finished with, or whether it should be left in the event queue. xlib:event-case does more than just dispatch the events, it can also act as a loop, repeatedly dispatching events.

If true, the event is ticked off (`processed' in the documentation) and removed from the event queue. If false, the event is not ticked off (`unprocessed' in the documentation) and hangs about so it can come back to haunt you later.

If true, the implicit event-case loop has done its job and exits. If false, the implicit event-case loop goes round for another try.

The odd little line

:discard-p t

(:exposure ()(format t "Exposed~%"))

format stream --- output to stream, return nil format t --- output to standard output, return nil format nil --- output to string, return string

(:enter-notify () t))

Making a mark

Source file: graphic-x

It is probably rather frustrating that it is only at the third example that one gets to draw anything on the screen. I've delayed because you cannot just draw a line, you must first create a graphics context. It is worth taking a little while to review the two problems that graphics contexts are intended to solve.

The first problem is that one quickly finds that there are many variations on drawing a line. How thick is it, what colour is it. If it is thick, does it have an end cap? If so is it round or square. Is it dashed? If so, how long is the dash and how long the gap, or is there a more elaborate pattern. More subtly, is the gap just left or is it inked in the background colour? One certainly doesn't want to type in all these different parameters every time one wants to draw a line. There is a choice of solutions. Either one takes advantage of Lisp's keyword parameters, eg a :thickness keyword to set line thicknesses, or one has a graphics context that packages up your favourite settings.

The second problem is that one must remember that X11 is intended as a network window system. Obviously one must send the coordinates of the end points across the network, but one wants to avoid repeatedly sending colour and thickness data over the network. The graphics context should reside on the server.

The assumption underpining X11 is that you will often want to make changes to the graphics context, sometimes drawing in red, sometimes in blue, some lines thick, others thin, but that you will circulate round a limited set of options. For example, there might be hundreds of lines, but they will all fall into one of the four categories thick-red, thin-red, thick-blue, thin-blue.

The way it works in X11 is that the client sets up a modest number of graphics contexts on the server. Drawing commands quote the numerical ID of the graphics context. So it is very cheap to chop and change between graphics contexts.

Clients can also change the contents of a graphics context. CLX saves up the changes and keeps a local copy, only sending them over the network as required.

(defun graphic-x (width height &optional (host ""))

We are going to draw a big X across the window, with two lines. The lines will be drawn according to the function arguments, so if the window manager doesn't give you the size you ask for the lines will not go neatly into the corners as intended.

(grackon (xlib:create-gcontext :drawable root-window :foreground white :background black))

(describe grackon)

Now we respond to exposure events by drawing:

(:exposure () (xlib:draw-line my-window grackon 0 height width 0) (xlib:draw-line my-window grackon 0 0 width height))

I've sneaked in a little change to the event mask and the list of events in event-case. Now the window stays up until you click a button in it. So it is convenient to resize the window, and iconise it and de-iconise it, and see that you are redrawing the same old X.

Graphing a function

Source file: graph-f

We've just used Xlib's draw-line routine to draw a line. It is natural to plunge into plotting the graph of a function by writing a loop to draw many lines, from (0,f(0)) to (1,f(1)) and from (1,f(1)) to (2,f(2)) and so on. Wait. Xlib has a draw-lines command, which accepts a sequence of numbers, alternating x and y co-ordinates. It links the lines for you, using the joining style in the graphics context, and can even fill your figure for you if you want the lines to represent a polygon.

To obtain greater functionality, we simplify our CLX code! We respond to exposure events thus:

(:exposure () (xlib:draw-lines my-window grackon points) nil)

Actually the other code is CLX-style. The natural style for Common Lisp is to use a sequence of points. CLX uses a sequence twice as long alternating x and y. This results in clumsy code. I walk you through it.

First try out the single-graph function with something like

(single-graph #(0 0 100 100 200 300 300 0) 400 400)

We start with a routine to generate the array for plotting a graph of the function y=f(x). We take advantage of the fact that CL lets you use any characters in a symbol name to call our function x,f(x) which reminds us that it is generating a list which alternates x1,f(x1),x2,f(x2),... The vertical lines are the essential quoting characters. They let you use commas and parentheses in symbol names just like " lets you use them in strings.

|x,f(x)| is passed a function f and calls it repeatedly to build an array tabulating the function in the range x-min to x-max. The only concessions to this being graphics code are the inclusion of a resolution parameter saying how many points to plot and the use of CLX's native data layout.

If we are going to scale this data to fit a window we need to know the minimum and maximum values. This is calculated by bound-xy-vec.

fit-xy-to-window uses bound-xy-vec to find the minima and maxima, then builds a new array of integers, scaled and rounded for display. Note the natural use of the second argument to round to divide out the range from min to max.

Finally, normalised-graph duplicates its width and height arguments, to scale the plotting data and set the size of the window. This is where we lose the chance to resize the graph when the window is resized, so this is the code we need to fix in a later, not quite so simple example.

So we can plot a few cycles of a sine wave

(normalised-graph (|x,f(x)| 100 (- pi) (* 3 pi) #'sin) 400 200)

(normalised-graph (|x,f(x)| 100 -3 3 #'(lambda(x)(* x x))) 400 400)

|x(t),y(t)|

So one draws a circle with

(normalised-graph (|x(t),y(t)| 100 0 (* 2 pi) #'cos #'sin) 400 400)

(normalised-graph (|x(t),y(t)| 100 0 (* 2 pi) #'(lambda(x)(sin (* 2 x))) #'sin) 400 400)

|z(t)|

(normalised-graph (|z(t)| 100 0 (* 2 pi) #'(lambda(theta)(exp (* #c(0 1) theta)))) 400 400)

(normalised-graph (|z(t)| 100 0 (* 3 pi) #'(lambda(theta) (+ theta (exp (* #c(0 1) (- (* 3/2 pi) theta)))))) 800 200)

(normalised-graph (|z(t)| 1000 0 (* 2 pi) (cycloid 3 10 13 5)) 400 400)

Understanding exposure

Source file: understanding-exposure

The purpose of exposure events is to let the client redraw freshly revealed parts of the window. Simple example code does not have to confine itself within this intention. The function

(defun show-exposure-events (width height &optional (host ""))

(:exposure (count x y width height) (format t "~A~%" count) (xlib:draw-line my-window grackon x y width height t) (xlib:draw-line my-window grackon x (+ y height) (+ x width) y))

Is the exposed region always rectangular? No.

Shrink one of your X terms so that it fits inside this programs window, and leave it on top. Expand another window so that it will cover both, and place it on top. Now iconise the large window, exposing this programs window, and the smaller window sitting on top of it. The exposed region forms a ring. It is not rectangular. It is not even simply connected. Worse, one can use two small windows to create an event that exposes a region with two holes in it. Try it and see. It will become apparent what the count variable is for.

The classic use of count is to spot that a single action has exposed a region that is not rectangular and to simply give up. The expose events are guaranteed to be contiguous in the event queue, with the count counting down to zero, so the program discards expose events with a positive value of count and when the countdown ends, it redraws the whole window.

Is this an adequate approach? You'll have seen the problem when you moved other windows over the show-exposure-events function's window. Lots of narrow rectangles are generated. If a real program embarks on an elaborate recomputation of the whole window for each of many narrow rectangular expose events, it could get hopelessly backlogged. Unfortunately the X server cannot provide a count down. It doesn't know when the user is going to stop waving the uppper window around, so what would the count start at?

Two possible answers: One is that clients that are presenting the results of slow computations should cache the results on their side of the network and retransmit from their explicitly managed cache. Alternatively, create a pixmap at the server end. Write the data into the pixmap and have the client program issue instructions to the X server to copy the pixmap into the client programs window. But where are pixmaps stored? In the graphics card? In the server machines main memory. I don't know.

We don't have to tackle those problems now. It is enough just to have an idea of what causes exposure events, and what your client program has to cope with as far as numbers and shapes.

Hello World

Source file: hello-world

Simple examples make use of defaults. The clx commands to place text on the screen draw it under the control of the graphics context. The graphics context contains a field for the font information, and is supposed to default to something useable. So we can replace our crossed lines with text.

Ragged Right

Source file: ragged-right

Some examples are so simples that they are useless. In particular, one wants to call xlib:text-width to keep track of spacing, and that requires that you know what font you are using. Just letting it default doesn't really work.

The obvious thing to do is to have an extra variable font , initialised by a call to xlib:open-font

(font (xlib:open-font display "-*-lucida-medium-r-*-*-12-*-*-*-*-*-*")

xlib:create-gcontext

xlib:create-gcontext

xlib:create-gcontext

(xlib:text-width font word)

(xlib:text-width (xlib:gcontext-font grackon) word)

xlib:text-width

ragged-right.lisp

(xlib:text-width grackon word)

(xlib:font-ascent (xlib:gcontext-font grackon))

alan:font-ascent

alan:draw-line

xlib:draw-line

There is a more general point here about the difference between programming in statically typed languages and programming in dynamically typed languages such as CL. I'm used to writing statically typed code such as

(eat-fruit apple) (eat-fruit (peel bananna))

Enough of coder's chit-chat, back to running actual code.

The point of setting text is to allow the user of the program to resize the text window and have the text reflow to suit, but the positions of the words in the window are computed in the X-client (think: big machine in basement) while the window size is controlled by the X-server, the machine you are touching and seeing. The X-server must notify the X-client of changes to the configuration of the window.

We add :structure-notify to the event mask of our window, so that it is listening. We also put

(:configure-notify (width height) (setf actual-width width actual-height height) nil)

The Xlib manual is explicit that if the window manager resizes the window before it is mapped, the Configure-Notify event appears on the queue before the first Expose event. My code depends on the Configure-Notify event to initialise the actual width and height, even if the window manager gives me the size I ask for. I've not tracked down documentation on whether I'm allowed to do this. Please email me if you find it.

The function expects the text as a list of words, so

(ragged-right '("The" "cat" "sat" "on" "the" "mat."))

Representing the text as a list of words makes it trivial to write a non-breaking space.

(ragged-right '("The cat" "sat" "on" "the mat."))

(ragged-right (white-space-split "Ragged right setting is easier than justified setting. This is both a strength and a weakness. Although the regular word spacing of ragged right setting is easier on the reader's eye, in craft work there is honour and glory in doing things the hard way. The reader of justified text knows of the labour and expense, and is flattered to get something for nothing, even if it is worth what he paid."))

Notice that every expose event is generating oodles of network traffic. An xlib:draw-glyph for every word in the text, even if it doesn't fit in the window and is just discarded by the X-server. On my 166MHz Pentium, you can see slight lags. I doubt you can see the problem on your shiny modern machine. However, one cool idea is to deploy web services using X11, instead of HTTP. That would need much better code.

Coloured rectangles

Source file: Mondrian

Turning away from text for a few minutes, let us look at colour. An obvious demonstration program for colour is to display some coloured rectangles, so the source file starts in a fairly obvious way, with a structure of a coloured rectangle. Obviously the random-choice routine is to choose a colour from a list of colours.

Common Lisp's built in make-list function doesn't quite suit our needs because it evaluates the initial item form just the once, so we define our own version, cons-up, to call a constructor repeatedly.

Well, we have a list of colours, '(red green blue ... ), and we want a graphics context for each colour, so that we can swap from context to context as we go down the list of rectangles drawing each in its own colour. There are lots of ways to map from symbols to graphics contexts. I've chosen to put the graphics context onto the symbol's property list as its grackon property.

First we make additions to the propery lists I should have used mapc, or better yet, dolist.

(mapcar #'(lambda(colour-symbol) (setf (get colour-symbol 'grackon) (xlib:create-gcontext :drawable root-window :foreground (xlib:alloc-color (xlib:window-colormap root-window) (symbol-name colour-symbol)) :background black))) *colour-list*)

The call to xlib:draw-rectangle is mostly how you expect. The optional fill-p parameter could have just been t , but I find it annoying to have annonymous constants at the end of parameter lists. 'fill does just as well at saying "yes" to the computer, and also reminds the programmer what he was saying yes to.

More coloured rectangles

Source file: subwindow

Windows live in heirarchical trees. So far our sole window has been childless. Let us change that, creating some subwindows distinguished by their background colours.

The first thing that strikes you is the omission of expose events. You only ever omit expose events in simple example code. I'm making a point here. The X-server looks after window backgrounds for you. A command line such as

(graphic-x 300 300 50 50)

Also notice that the X-server is keeping track of which of your windows the cursor is in. Although X is quite low level, there are some important house keeping tasks that it does keep off your back.

Paragraphs, or Ragged Right continued

Source file paragraphs Sample text escape

I've made incremental improvements to ragged-right.lisp, re-arranging the code, and improving it to display more than one paragraph. One evaluates

(paragraphs "escape")

Analogue input

Source file: analogue-input

One of the points of learning CLX is to get pointer positions, so that you can equip your programs with analogue controls. Then the positions of the pointer on the screen can represent numbers, higher up the screen for a bigger number, further to the right for a bigger number.

This little routine lets the user input two numbers by clicking in the window. For example

(pick2numbers 400 100)

In understanding-exposure we extracted some parameters from the exposure event with

(:exposure (count x y width height)

(:button-press(x y)

Here though we have a clash of intuitions. If the display is some kind of mathematical graph, y should increase up the screen. The top to bottom numbering that X11 uses and is so natural for text, is the wrong way up for analogue input, so we correct it, just managing to avoid an off-by-one error.

(cons x (- y-range (+ y 1))

We have also put a title in the title bar of the window. This is a important cosmetic touch. It is worth pondering that it ought to be impossible. Certainly we can draw text inside our window, but X11 windows our drawing commands to fit inside the window, and the window manager fits its frame around our window, strictly outside it. How did we manage to write outside our own window?

We didn't. We talked to the window manager, and it put the our title on for use. Hence the new command

xlib:change-property