SBCL Win32 Notes

Introduction

Dmitry Kalyanov has done the great work of adding threading support to Steel Bank Common Lisp (SBCL) for Windows, and continues to improve on it. While this code is not added to official SBCL code base yet, I've started to use it in a real project as soon as it became stable enough.

SBCL for Windows is still experimental, and there are many places in need for platform-specific fixes. There are also some changes required specifically for threaded build: e.g. on other platforms where SBCL supports threading, blocked I/O operation will be interrupted by sb-thread:interrupt-thread, but current official implementation of win32 I/O (combined with safepoint-based interrupts used by Dmitry) doesn't allow it.

Therefore I created my own fork of Dmitry's GIT repository and started to work on Windows-specific issues, beginning with the imminent ones.

Dmitry and I merge our changes periodically, to lower the probability of long, boring conflict-resolution work in the future. His experimental tib-3 branch includes my changes, except a few ones done after the last merge.

This text is purported to provide a description of the problems I've tried to solve and of the solutions chosen. Sometimes I describe the peculiarities of Windows in depth; it could be boring for experienced Windows programmer, but I hope that it will be useful for SBCL maintainers working on normal platforms.

Windows is Special

This section describes the aspects where Windows platforms are different from Unix-like ones (the latter are traditional, widely-used targets for SBCL and its ancestor, CMUCL). SBCL internals and my changes are mentioned only in passing here.

What is Win32 and Is There Win64

There is a longstanding tradition of using "32" as an antonym for "16" in the Windows world. SBCL will probably be ported to 64-bit Windows systems in the future (if those systems survive until that time). Due to a peculiar meaning of "32" described above, unofficial terms like "Win32 API" are applied to those systems as well. Names of the system DLLs implementing the API still end with "32", too.

Therefore, most of the notes will be applicable to 64-bit Windows (only a few problems, like multiple calling conventions, are specific to 32-bit systems). And I would advise SBCL developers to retain :win32 in *FEATURES* for 64-bit port, following the tradition of Windows world in general.

Win32 History in a Nano-Nutshell

Initially there was two implementations of what is called "win32 API", providing the same interface with entirely different internals:

NT implementation, on top of 32-bit, multitasking, unicode-aware Windows'NT;

Win32s subsystem on WfW 3.11 and its glorified heirs on Windows 95/98/ME.

Modern Windows systems for PC (Windows'2000,XP,2003,Vista,Windows7...) are based on the Windows NT variant; the 95/98/ME generation's install base is already unnoticeable, and they're neither sold nor supported by the vendor any more.

Publicly available documentation on MSDN now reflects this fact by covering compatibility of the API functions only within NT family, for Windows'2000 and above. Everything before Win2k is antediluvian and isn't worth mentioning. Targetting pre-Win2k systems in our times is problematic in more than one way, and the usefullness of this endeavor is decreasing every day, if visible at all. Therefore it would be wise to limit our compatibility and porting efforts to Win2k and above. The next big bunch of API improvements arrived with Windows Vista, but older systems are still widely used, so I think it's too early to rely on these features unconditionally in a project like SBCL.

Since Windows XP, each major change in the OS codebase ends up in two flavors of Windows, marketed as two products with different names: "server" and "client" OS variants. From the programmer's point of view, both variants in each pair are almost the same:

Windows XP and Windows Server 2003

Windows Vista and Windows Server 2008

Windows 7 and Windows Server 2008 R2

MSDN documentation includes both client and server versions in its Compatibility sections for function descriptions.

Plain C Programmer's View: Ogres Have Layers

When we use printf or fputs on most platforms, we are calling a function residing in the C runtime library (CRT). On Windows, there is a canonical implementation of C runtime library, MSVCRT.DLL (i.e. Microsoft Visual C runtime). It was initially included into redistributable package coming with MSVC compilers; now it's officially a part of the OS. Newer versions of MSVC use runtime libraries with other names (for dynamically-linked runtime, DLL has a versioned name like MSVCR71.DLL ).

SBCL for Windows is compiled with MinGW. The latter is a port of GCC that is capable of building "native" applications, linked against MSVCRT.DLL by default. Since MSVCRT.DLL is pre-installed with Win2k or above, such applications don't require any redistributable runtime package to be installed: they work out of the box on any modern Windows system.

MSVCRT itself is implemented on top of "Win32 API" functions, provided by KERNEL32.DLL , USER32.DLL and other system libraries. Those libraries are supposed to be used by applications directly, when OS-specific features beyond basic file I/O are needed. Interfaces provided by them are stable and well-documented (well, comparatively); backward compatibility was mostly retained since pre-unicode, pre-NT times (WfW 3.11 + Win32s, Windows'95/98/ME) and early Windows'NT (3.x and above).

As of the NT family, system DLLs interact with OS kernel and drivers with so called "native API", which is a thin layer on real kernel system calls. Native API is available for applications as a functions exported from NTDLL.DLL . Some parts of it are documented as part of Windows DDK, other parts are undocumented (but some undocumented parts are still widely used). There are things that are impossible to do without native API, and some of them are potentially useful for SBCL. So far I was able to go without these functions, but I may decide to use them in the future. Given the pragmatic nature of Microsoft's policy regarding backward compatibility, calling a widely-used undocumented function is almost as safe as calling an officially documented one.

There is also a socket implementation (winsock2), WS2_32.DLL . Concrete protocols and address families are hooked into it in a plugin-like fashion. A set of BSD-like socket-related functions is provided (they work almost like the real thing, errr, BSD-style API on Unices), as well as other Winsock-specific or Windows-specific functions. Winsock2 was initially intended to be portable (though I don't know of any non-Windows implementation); some entities provided and used by Winsock2 are documented (or known) to be the aliases of corresponding Windows entities for Windows implementation:

WSACreateEvent and CreateEvent work with the same kind of events (officially);

and work with the same kind of events (officially); WSAGetLastError and GetLastError work with the same variable containing an error code (known);

and work with the same variable containing an error code (known); Each socket handle may be used as a Windows file handle for NT family (officially).

Though full socket I/O support doesn't belong to core SBCL runtime, there are things that require Winsock2 calls when an open I/O channel happens to be a socket. As SBCL process may inherit a socket handle as its standard input or output (though this approach is not used on Windows as frequently as on Unices), core SBCL runtime still has to use Winsock2 and link against it.

Streams, Descriptors, Handles and Objects

MSVCRT itself, with regard to I/O operations, contains two layers at once:

"lowio" layer: POSIX-like functions working with integer file descriptors, not providing full buffering but providing CRLF/LF conversion;

"stdio" layer: ANSI C stream functions built on "lowio" level.

Each stdio stream (FILE*) references a lowio descriptor. Each descriptor, in turn, is an index into a table inside CRT , containing Windows file handle as well as other useful information. If we go on with this story (going into the native API), each file handle is either a kernel-level handle, or a console handle (console handle operations are intercepted by KERNEL32.DLL and redirected to CSRSS.EXE using client-server remote procedure calls). Some kernel-level file handles are also Winsock socket handles.

Any handle of a kind discussed so far may be duplicated with DuplicateHandle() (though there are some problems for socket handles that will be discussed below). When a handle is duplicated, it still designates the same kernel object; the only private thing that it has is a possibility of inheritance.

As the lowio functions are almost a drop-in replacement for their Unix counterparts, SBCL developers decided to use it as an emulation layer. It is a non-trivial design decision (many other language runtimes preferred raw file handles on Windows as an equivalent for Unix descriptors). This decision has its merits, especially for Common Lisp implementation: for MSVCRT lowio, 2047 is a maximum possible value for a descriptor, so all valid descriptors are fixnums.

There is a way to wrap an existing file handle into a lowio descriptor ( _open_osfhandle ) and to get it back ( _get_osfhandle ). However, after a file handle is associated with a descriptor, there is no way to destroy or alter this association without closing the descriptor (the handle is closed with the descriptor unconditionally, too). Closed descriptors are associated with INVALID_HANDLE_VALUE ((HANDLE)-1).

Standard Input, Output and Error

As of Win32 API layer, standard channel handles are set and retrieved using SetStdHandle and GetStdHandle function, respectively. When MSVCRT.DLL is loaded, it associates those handles with descriptors 0, 1 and 2, and creates FILE* stdin, stdout and stderr ANSI streams.

If SetStdHandle is used later, associations for descriptors 0, 1 and 2 remain the same.

Values of standard channel handles are specified when the process is created ( CreateProcess in Win32 API), or simply inherited from the parent process when appropriate flag is given.

On Unix there is a tradition, not mandated but widely respected, to make normal processes inherit descriptors 0, 1 and 2 (pointing to /dev/null if there is no other sensible input or output). On Windows, the same stands for console applications (including the fact that it's not mandated or enforced).

Locking and Serialization

When threads are used, the consequences of doing things on the same channel concurrently suddenly become important. Newer MSVC runtimes introduce some locking on stdio level, along with non-locking counterparts. As of the lowio level, there is no locking for the duration of an operation itself, but internal MSVCRT descriptor table is protected enough for concurrent access to be safe: no corruption will occur on attempt to _dup2 into the same target descriptor concurrently; only one of concurrent _close() calls on the same lowio descriptor will succeed, etc.

Calls to lowio _read() and _write() are translated into Win32 API synchronous I/O calls ReadFile() and WriteFile() without any additional locking at the CRT side for the duration of I/O itself. Therefore, further discussion of locking and serialization will be concerned with the behavior of Win32 API routines, not their lowio wrappers.

Synchronous I/O operations that are used by lowio code are serialized by the OS kernel itself (or by KERNEL32.DLL for console handles). There are some useful aspects in it for C application developers, but there are also some problems that made me avoid _read() and _write() completely at the end.

The main problem is that kernel object locking is not direction-specific: outstanding read operation causes concurrent write operation to block. The important aspect is that it's the kernel object that is locked, not a handle; consequently,

duplicating the handle will never solve locking problems,

if your process isn't doing anything with a handle, it may still be locked by another process.

It's not uncommon for Common Lisp projects to use a separate "writer" and "reader" threads for the same bidirectional stream (SLIME works this way with socket streams in multi-threaded mode; my own in-house project works this way with serial ports). We have to support it, so this kind of locking granularity is too large for us.

Another problem with synchronous I/O is its synchronous nature: when a thread is blocked, it can't be interrupted. This problem has a couple of interesting solutions while keeping the operation synchronous: one solution uses new functions introduced in Vista, another one uses native API. I haven't implemented any of them for SBCL yet, but it might make sense: with some kinds of handles we have no other option but the synchronous I/O, so some operations still cannot be interrupted.

OVERLAPPED (Asynchronous) I/O Facility

Win32 API provides a special I/O mode, called OVERLAPPED I/O: application shall issue a request to start an operation; when the operation completes, OS notifies the application. Any outstanding operation may be cancelled by the thread that initiated it; it's also cancelled automatically if the thread exits. Overlapped call doesn't block even if there is another outstanding operation on the same kernel object in another thread. However, an operation itself is done atomically: when two blocks of data are written simultaneously, one of them won't end up in the middle of the other.

It seems that it's exactly what we need (and what I'm using now when it's possible). There are some problems surrounding this solution, however, arising both from the Win32 API side and the CRT side:

OVERLAPPED mode is enabled or disabled for kernel objects at creation time, and it can't be toggled afterwards. If our process inherits a handle, it's either OVERLAPPED or not (usually not — and there's a reason why). Our process has to live with it.

For every file channels except stream sockets, synchronous I/O calls ( ReadFile() or WriteFile() with lpOverlapped == NULL) have undefined behavior. Lowio routines use synchronous calls, so for non-socket OVERLAPPED handles, they always fail.

or with lpOverlapped == NULL) have undefined behavior. Lowio routines use synchronous calls, so for non-socket OVERLAPPED handles, they always fail. For on-disk files, the file offset where the operation should start is specified in completely different way for OVERLAPPED calls.

Console handles are never in OVERLAPPED mode (remember, they are not "real" handles from the native API point of view).

Anonymous pipe handles, though they are real kernel handles, aren't OVERLAPPED as well.

Event Objects

Overlapped I/O completion can be signalled in two ways: with a queued callback (APC), or with an event object. However, we have to work with file handles of both OVERLAPPED and non-OVERLAPPED kinds, and there is no simple way to request this property for a handle (native API has to be used for it). ReadFile() and WriteFile() provides such an "agnostic" solution: they fall back to synchronous mode if an OVERLAPPED structure is given but the handle is not OVERLAPPED. Event object, however, is the only way that those functions may use to notify the completion of an asynchronous operation.

Event objects are kernel objects, not unlike a sort of dumbed-down boolean semaphores. As far as I know, they are unique for Win32 API and have no equivalents in other modern systems. They are also notorious for looking as a right thing to use when they aren't: race conditions, missing signals and fairness problems are to be expected around them. I've tried to be careful this time, but I'm also notorious for misusing event objects, so watch out.

Waitable Timers

Implementation of (SLEEP) that can be interrupted and continued requires something other than Sleep or SleepEx call (the latter may be interrupted with APC if it's alertable, but can't be resumed). Waitable timer objects provided by Win2k and above are the thing that I decided to use (for threaded builds only, by now, but they would work on non-threaded builds as well, so maybe it's better to unify the code: pre-Win2k portability is already unattainable).

Kinds of I/O Handles

Some operations that are unified for Unix descriptors (like checking if a future read() operation would block) don't have lowio equivalents; as of Win32 API equivalents, they are sometimes available for a particular kind of a file-like object, or for some kinds with different things to do for each one.

SBCL code sometimes check for the following kinds of file-like objects to handle them specially:

sockets;

pipes (named or anonymous, doesn't matter);

consoles (well, console, though there can be many handles);

As of my own changes, another special case is added for

communication resources (it's the MSWinSpeak for "character devices");

All other I/O handles are treated as "ordinary or unknown".

Error reporting

MSVCRT provides a traditional errno "variable" (it's actually a macro expanding into dereferencing the result of a function call, allowing this thing to be thread-local). Symbolic constants for error codes are defined in errno.h , and a corresponding textual message may be retrieved with strerror() .

Win32 API returns its error codes in entirely different place, available with GetLastError() and SetLastError() (its address is unofficially known to be FS:[0x34] for NT family). FormatMessage function is used to retrieve a textual error description. Interpretation of error codes is incompatible with CRT errno . The logic used by Win32 API for updating it is also different: it is used to return supplementary information for successfull calls, so any Win32 function will modify it unconditionally, resetting it to ERROR_SUCCESS (0) if nothing else makes sense.

Winsock2 provides WSAGetLastError() and WSASetLastError() functions, accessing the same place as GetLastError() and SetLastError() on the NT family. Error codes used by Winsock2 are in a separate range, and the same FormatMessage() call retrieves a corresponding error message for Winsock2 errors.

One interesting fact about error code symbolic constants: we have both WSAEINTR and EINTR , they are different in value, and the places where we expect to find them are different too.

Calling Conventions

This section is entirely unrelated to I/O, and it's valid for 32-bit systems only.

On x86 (including i386 and above) different compiler vendors used a lot of different ways to call a function and to return from it. Two of them survived (for public interfaces) on Windows systems:

"cdecl", that is used on x86 Unices as well; it's normally the default for any C compiler;

"stdcall" (WINAPI, PASCAL, __pascal, __stdcall), that is used in Win32 API functions.

The only noticeable difference reflected in the object code is who cleans up the stack: it's a caller for stdcall and a callee for cdecl. Win32 port of SBCL foreign function interface is designed to call external functions of both kinds without explicit convention specified: ESP register is saved before callout and restored after return, so it doesn't matter if a callee adjusted it.

As of SBCL, there are two aspects where calling convention still matter:

"stdcall" function referenced by C code is turned into a symbol with a special suffix: @n, where "n" is a decimal number of bytes occupied by arguments; in the C world, this kind of name mangling prevents resolving a function with missing or wrong prototype in a way that would misplace the stack pointer;

while a convention-agnostic callout support is easy, convention-agnostic callback support is impossible: there is no way to make a foreign code adjust stack after our callback returns. All callbacks used by Win32 API are stdcall, and all callbacks generated by SBCL are currently cdecl.

SBCL Modifications: Present

This section describe major changes to SBCL code in my branch relative to the upstream tree, occasionally mentioning remaining problems and plans to solve them.

Input and Output

My current branch of SBCL code provides its own lowio-like wrappers for read and write operations. Those wrappers are ready to work with both OVERLAPPED and non-OVERLAPPED handles (in the latter case there is no way to interrupt a blocking call; in the former case, an operation is cancelled if an interrupt is received).

Replacement for lowio _open() is written in Lisp. It calls CreateFile() with FILE_FLAG_OVERLAPPED; it makes sense for communication devices and named pipes, and does no harm for ordinary files, as long as native _read and _write are not used. SetCommTimeout() is called immediately after opening the file: if it's a communication resource, timeout settings are adjusted, so ReadFile() will return when there are some data available, not wait until the whole input buffer is filled (this "short read" semantics is what is expected by SBCL fd-stream layer).

Winsock socket() function is known to return OVERLAPPED socket handles by default, and it doesn't hurt synchronous operations. However, SB-BSD-SOCKETS module was going to some length to ensure non-OVERLAPPED socket creation, apparently, for no reason at all (though I understand the fear of unexpected problems with synchronous I/O if a socket is made OVERLAPPED, it is still unfounded: OVERLAPPED sockets are documented (and known) to work synchronously as well).

My lowio equivalents wait for I/O completion or for an interrupt, and return EINTR if the latter has happened first (operation is cancelled with CancelIo if it happens). Two event objects are created for each thread: one for I/O completion signalling, another one for interrupt signalling.

As described above, some handles can't be OVERLAPPED, so it's not the final solution: some operation are uninterruptible yet.

Concurrent operations with the same handle and direction are not serialized; for seekable files, they are even non-atomic, so concurrent writes produce files with undefined content (fixable with a critical section). The thing I find comforting is that for buffered FD-STREAMs , SBCL will screw it up anyway.

There is also a replacement (actually, a wrapper) for (UNIX-CLOSE) as well. The only thing it does, beyond calling _close() for lowio descriptor, is calling closesocket() if a handle was detected to be a socket when it was alive. Two things to keep in mind:

closesocket() has to be used if we don't want Winsock2 to leak resources;

has to be used if we don't want Winsock2 to leak resources; closesocket() call after CloseHandle() on the same handle is valid; according to Winsock, the handle is still alive; it won't reuse it for another socket, nor will it complain for closing a closed handle.

(SB!UNIX:UNIX-LSEEK) is now implemented using _lseeki64() function from MSVCRT ; type declarations are adjusted as well, so (FILE-POSITION) now works with large files.

See

SB!WIN32:UNIXLIKE-OPEN

SB!WIN32:UNIXLIKE-CLOSE

SB!UNIX:UNIX-READ

SB!UNIX:UNIX-WRITE

win32-os.c: win32_unix_read

win32-os.c: win32_unix_write

ANSI Standard for CL specifies :if-exists :append to set file position to the end of file when it's opened. Common Lisp implementations for Unix-like platforms traditionally translate it into O_APPEND flag for the call to open() , and SBCL is not the exception.

It's interesting that O_APPEND semantics is not the same thing as "position to the end while opening" required by the CL Standard. O_APPEND positions to the end of file before each write operation, not when opening the file; modern systems also promise positioning and writing to be atomic as a whole, with a usual exception of network filesystems (see Unix Haters Handbook for details).

MSVCRT interpretation of O_APPEND for lowio descriptors is almost the same as the Unix one, except that positioning and writing is not atomic: they are done as two separate calls with no locking around them (mutex won't help here anyway: even if other thread couldn't step in between positioning and writing, other process could).

Win32 API CreateFile() function provides an equivalent for O_APPEND (as part of desired access flags, for some reason). I decided to use it in SB!WIN32:UNIXLIKE-OPEN if O_APPEND is given (some modification is probably required here: O_APPEND currently forbids reading).

Conclusion: with my replacement for _open() , O_APPEND gets its Unix-like semantics, and :if-exists :append is interpreted in a way closer to other platforms but farther from CL Standard requirements.

See

SB!WIN32:UNIXLIKE-OPEN

SB!WIN32:HANDLE-LISTEN tries to read communication resource statistics with ClearCommError. If the call succeeds, COMSTAT structure contains a count of bytes queued for reading in the system buffer.

Other kinds of objects supported by HANDLE-LISTEN in the original code base are pipes, consoles and sockets (thank Dmitry Kalyanov for the latter). Unfortunately, support for console objects is broken: when there is a keyboard event for some "extended key" in the input buffer, PeekConsoleInput sees it but ReadFile doesn't (no ANSI or Unicode character is generated, so there is "nothing" to read and ReadFile blocks).

SB!SYS:WAIT-UNTIL-FD-USABLE internals (polling for readiness) doesn't end up in (UNIX-FAST-SELECT) or (UNIX-SELECT) . Both -SELECT's simply don't work on win32; they should be either eliminated or rewritten (the latter is not easy).

See

SB!WIN32:HANDLE-LISTEN

SB!WIN32:COMM-INPUT-AVAILABLE

CreateFile()

SB-WIN32:UNIXLIKE-OPEN

_stat()

GetFileAttributes()

GetFileType()

While we can open "//./COM5" now, the solution is far from perfect. If we probe a named pipe in this way, it will count as a connection attempt, so if a named pipe expects exactly one client connection, the second CreateFile call will fail (instead of really opening the file).

As of named pipes, we shouldn't touch them in any way unless it's needed (by the way, it's a good policy for other files and even other platforms). (OPEN) implementation should eventually be redesigned not to probe file at all before opening it (among other things, it is a race condition). It seems to be possible for any combination of arguments to (OPEN) .

SLEEP and WITH-TIMEOUT

Waitable timer objects are used instead of a simple call to Sleep() . Consequently, sleeping threads are now interruptible with sb-thread:interrupt-thread, and if the interrupt function doesn't unwind, thread continues to sleep after it returns. Deadline of interrupted (SLEEP) call is not moved when interrupt occurs.

(SLEEP) implementation for Windows now accepts really big integer intervals: very long sleep is translated into a loop of many moderate sleeps. This paragraph also applies to a traditional Sleep() -based implementation, that is still used for non-threaded builds (I'm going to use waitable timers in both cases soon).

It turned out to be easy to implement the equivalent of (UNIX-SETITIMER) for threaded builds with a separate signalling thread and a designated waitable timer object. I have done it, so sb-sys:with-timeout now works on threaded win32 builds.

See

CL:SLEEP

SB!WIN32:MICROSLEEP

SB!IMPL::WIN32-ITIMER-SCHEDULE

SB!IMPL::WIN32-ITIMER-CANCEL

SB!IMPL::WIN32-ITIMER-DEINIT

STDCALL Name Mangling For Static Foreign Calls: Gone

After SBCL runtime is built, its symbol table is exported into sbcl.nm. During all further steps of the build process, foreign symbol references are resolved to their static addresses listed in sbcl.nm.

Name mangling convention described above is not used in system DLLs when a symbol is resolved dynamically with LoadLibrary() and GetProcAddress() : those DLLs export unmangled names. However, both symbol references created by the C compiler and symbol definitions provided by the import libraries use mangled names. Each address listed in sbcl.nm for a foreign function points to a tiny piece of wrapping code from the import library. Foreign stdcall function names in sbcl.nm are therefore mangled.

Win32.lisp is full of code like (alien-funcall "Sleep@4" ...). It was a great distraction for non-interactive development and a great obstacle for interactive one: mangled name can't be resolved if it's still dynamic for current interactive session; unmangled name works interactively but breaks compilation.

The code that parses sbcl.nm was modified to remember both mangled and unmangled variant of a symbol name, getting rid of this maintainance hell. This change is already accepted into upstream SBCL tree.

Error Handling

For some functions, both CRT error codes from errno and Win32 error codes from GetLastError() make sense. That's why I hacked SB!INT:STRERROR to accept a negative FIXNUM designating a Win32 or Winsock error. For such error code, my version of STRERROR calls FormatMessage with its absolute value. This change is mostly cosmetic; STRERROR was not used and shouldn't be used to detect a type of error programmatically, only to provide a user-readable message.

SB-BSD-SOCKETS module expected to find useful value in the CRT's errno variable. I've factored out the (SOCKET-ERRNO) function, that returns an error code for a socket operation in a platform-specific way: WSAGetLastError() on Win32, errno on other platforms.

There are some simple error code mappings in my lowio equivalents: ERROR_BROKEN_PIPE on reading is translated to EOF (0 bytes read, error code doesn't matter); ERROR_OPERATION_ABORTED , that we get after CancelIo() , is translated into EINTR ( errno value). Unhandled error codes for read and write operation are turned into EIO ( errno value).

(UNIXLIKE-OPEN) , if an error occurs, maps appropriate Win32 error codes to ENOENT or EEXIST (they are not assigned to errno but returned as a secondary value). For any other error, the secondary value is a negated GetLastError() result: (STRERROR) will convert it into readable message (see above).

(SB-EXT:RUN-PROGRAM) function had a problem with shell argument escaping. On Windows the program's caller is responsible for turning an array of argument into a plain command-line.

I've added (SB-IMPL::MSWIN-ESCAPE-COMMAND-ARGUMENTS) function that escape the arguments in such a way that CommandLineToArgvW() will unescape them back. Old escaping code was naive to the core, handling only the most basic cases (single-word v. multi-word) and ignoring weird Windows rules regarding "backslashes before a double quote" and "backslashes before something other".

Sockets are now created with OVERLAPPED flag turned on. When a socket is wrapped into a lowio descriptor, and this descriptor is used for reading or writing, blocked operation can be interrupted now.

Winsock2 provides a BSD-style non-blocking mode for sockets, but I don't know if it's possible to retrieve this setting for a socket handle. While adding support for (NON-BLOCKING-MODE) accessor, I had to add a slot containing this flag into a socket class: when we can't introspect, we should remember.

Socket non-blocking mode on windows doesn't affect file-like I/O operations, e.g. the ones used by FD-STREAM layer. Socket-specific functions, like SOCKET-RECEIVE , are affected.

Calls like select() and WSAEventSelect() reset socket to non-blocking mode. SB-BSD-SOCKET doesn't use these functions. I believe that if some external library, like USOCKET, calls one of them, responsibility to restore the old non-blocking state (if it matters) belongs to that library.

Blocking calls currently are not interruptible (it's possible, even without native API, but not done yet).

Some trivial changes were required to make it use SB!WIN32:UNIXLIKE-OPEN for files, and some changes of similar trivial nature are yet to be done (calling UNIXLIKE-CLOSE where appropriate). SB-SIMPLE-STREAMS currently passes all tests on Windows.

Memory-mapped files seem to work (underlying implementation of mmap() with MapViewOfFile() under the hood is included in the SBCL runtime). My version of munmap() is a cheat: it ignores the length argument, unmapping the whole block mapped by mmap() . UnmapViewOfFile() can't unmap partially, so we have to live with it (SB-SIMPLE-STREAMS internals don't use partial unmapping anyway).

Simple streams test suite proved to be an excellent testbed for my lowio function replacements: a problem with file positioning and another problem with UNIXLIKE-OPEN flag interpretation were detected by it. Both problems were obscure enough to stay undetected in normal conditions; both solutions caused rewriting and simplification of underlying code.

A known but overlooked peculiarity of Windows manifested itself (and required a tiny modification of SB-SIMPLE-STREAMS): normally, an open file can't be deleted. There is a way to open file so it can be deleted; however, it should be arranged when opening the file. It requires an extra access flag, and the whole CreateFile() operation may be denied when it would succeed without this flag.

I've modified UNIXLIKE-OPEN to take an optional flag designating the desire to delete an open file later, but I didn't modify SIMPLE-STREAMS to use it yet.

SBCL Modifications: Future

This section describes my ideas of further SBCL improvements (usually specific for Win32 platform) and the problems in desperate need for solution.

Thread-Specific System Objects

Sometimes a thread needs a thread-local system object, allocated once in a thread lifetime and deallocated in a type-specific manner when thread exits. Thread-specific events used for I/O completion and interrupt signalling are an example. They are stored in "struct thread" now, but a nice alternative is possible.

Given the TLS symbol value implementation used by SBCL, it's easy to implement the allocation/deallocation protocol described above in Lisp. Not only the pair of private events: e.g. timer objects used for (SLEEP) should be allocated the same way, not created and destroyed on each call to (SLEEP) . Per-thread events that are used in the condition variable implementation are good candidates too (they're now allocated and destroyed each time a thread starts/completes waiting on a conditional variable).

Stdcall Callbacks

Alastair Bridgewater made available his implementation of stdcall convention support for alien callbacks. TODO: check it out, test it, fix it if it's obsolete, ask people why it's not integrated yet(!)...

Dynamic Extent Callbacks

Each alien-lambda (persistent callback) eats a piece of static space. Most use cases for callbacks don't require them to be persistent at all. This use case should better be supported by a special macro, like (DX-ALIEN-FLET ...) ; it should use a temporal system memory segment, or a stack, or even a currently-compiled code segment... Variants are plenty.

Foreign Thread Callbacks

It seems not to be too hard, but requires some design decisions first:

signalling, representation or real thread assignment?

foreign thread extent (creation and destruction)

transparent or explicit support (should the ordinary callback work in a foreign thread?)

More Interruptible I/O Operations

Those that are currently uninterruptible:

synchronous winsock calls: for OVERLAPPED sockets, queueing an APC doing CancelIo will interrupt the operation.

non-OVERLAPPED handles (anything inherited, console, anonymous pipes): mad skills are required for console, but it's possible; native API can be used for other things to turn on FILE_SYNCHRONOUS_IO_ALERT (so APC will interrupt synchronous calls).

Open should Open Once

The problems caused by probing a file before opening it are described above. (CL:OPEN) implementation can (and should) be rewritten to avoid multiple underlying CreateFile() calls (NB: _stat() does it as well).

:UCS-2 External Format For Console I/O

Consider using ReadConsoleW and WriteConsoleW for console handles, and set its external format to :UCS-2 unconditionally.

Communication Device Parameters

For opened communication devices, we can emulate termios, or provide a cross-platform API on top of #+unix termios #+win32 Win32 Communication functions.

Invalidating SAP Objects When Dumping Core

All SAPs except those that point into Lisp memory spaces should be either set to zero address or changed to primitive objects with another widetag. BAD things may happen if a SAP survives dumping and restart and is used afterwards, and nothing prevents it currently.

Generate Static Function References Automatically

Another maintainability problem with SBCL code: foreign functions referenced by Lisp code but not C code have to be manually added to win32-os.c:scratch() (don't know if the same amount of work is required for ldso-stubs.S; is it maintained manually too?).

Undefined references detected between the first and second genesis should be added to a separate platform-specific file automatically (probably after ensuring that the symbols are available in C libraries).

UNIX-SELECT Imlementation (or Elimination)

We need something waitable for each handle to implement select() . There is an obscure control code for pipes (FSCTL_PIPE_ASSIGN_EVENT: works, but unpopular; native API required); we may also use the fact that 0-sized synchronous read on pipe will block (and FILE_SYNCHRONOUS_IO_ALERT could make it interruptible). WSAEventSelect for sockets, some mad technique (too long to describe) for consoles.

If this idea is abandoned (or until it's implemented) remove lisp-side calls to select() : they call winsock's select() , passing a non-winsock fdset made of CRT handles. Usually it simply fails, but something causes EXCEPTION_ACCESS_VIOLATION.

The ultimate goal is a working (SERVE-EVENT) , of course.

Adding FlushFileBuffers where appropriate is easy. However, making it interruptible when it could block is a good idea. FlushFileBuffers doesn't have this feature.

:IF-EXISTS :APPEND Redesign

An ideal solution would provide atomic appends until the first explicit FILE-POSITION adjustment; after the adjustment, normal random-access file I/O semantics should be provided.

This kind of thing will be useful on non-win32 platforms as well.

Don't know if it's possible with only one Win32 file handle (it may be).

SetConsoleCtrlHandler

Redirect control-c event to a foreground session (the same thing that SIGINT causes on other systems).

Concurrent I/O Issues for FD-STREAMS

Same-file, same-direction concurrent I/O operation could make sense (atomic read-sequence/write-sequence, etc). No bright ideas for the implementation yet.

Distinguish Handle Kinds on FD-STREAM level

Instead of trying to deal with file handle as if it were a console, a pipe, a socket, a communication resource... each time we need it, detect it once and set up FD-STREAM functions accordingly.

NB: dup2() may change it after the fact.

It seems that no one really needed it on fork-enabled systems. It would be a great thing for win32, however; is it so hard indeed?