LFS

Short history and Description

The LFS, or Log-structured File System, is an alternative filesystem design proposed by Rosenblum and Ousterhout and originally implemented for the Sprite operating system at the University of California, Berkeley. A different version of LFS, called BSD-LFS, was written by Margo Seltzer for 4.4BSD. The NetBSD LFS is directly derived from the latter, via the 4.4lite2 release.

LFS presents a radical new design for filesystem layout which makes rather different assumptions than traditional filesystems. The most important difference is that whereas traditional filesystems lay out files on disk once, near each other, and rewrite those blocks in the same place whenever they need to be rewritten, LFS instead always writes whatever blocks it has to write, all in the same place (i.e. all blocks to be written at time t 0 are written to adjacent disk locations, regardless of what files they belong to), so that the same block of a file, written multiple times, will exist on the disk in multiple different locations. This allows safe, asynchronous file creation (the old version of the parent directory still exists in the case of a crash); fast file writes (all blocks are written together, without intervening head seek); and near-instantaneous crash recovery (the fs recovers from the last checkpoint and rolls forward, rather than having to be checked in its entirety for consistency).

The NetBSD LFS was orphaned for some years before I picked up work on it. I believe the version in NetBSD-1.6 to be almost completely working, although there is still work to be done (see To Do, below, and the NetBSD problem report database).

LFS Papers

Rosenblum, M., and Ousterhout, J. ``The LFS Storage Manager.'' Proceedings of the 1990 Summer Usenix, Anaheim, CA, June 1990, pp. 315-324. [ROSE90]

Rosenblum, M., and Ousterhout, J. ``The Design and Implementation of a Log-Structured Filesystem.'' ACM Transactions on Computer Systems, 10(1), February 1992, pp. 26-52. [ROSE92]

Seltzer, M., Bostic, K., McKisick, M., and Staelin, C., ``The Design and Implementation of the 4.4BSD Log-structured File System,'' Proceedings of the 1993 Winter Usenix, San Diego, CA, January 1993. [SELT93]

Seltzer, M., Smith, K., Balakrishnan, H., Chang, J., McMains, S., and Padmanabhan, V. ``File System Logging Versus Clustering: A Performance Comparison'', Proceedings of the 1995 Winter Usenix, January 1995, pp. 249-264. [SELT97]

Matthews, J., Roselli, D., Costello, A., Wang, R., Anderson, T. ``Improving the Performance of Log-Structured File Systems with Adaptive Methods.'' Proceedings of the Sixteenth ACM SOSP, Saint Malo, France, October 1997. [NEEF97]

Benchmark results

It is difficult to find an appropriate benchmark, in general: you want to know how fast the system will do what you ask it to do, not how fast it will run benchmark x. Nevertheless, you may find some benchmark results useful: I have a build test and Sean Davis has volunteered bonnie and tar results.

To Do:

Userland

fsck_lfs: fragmentation measuring and repair as an option.

lfs_cleanerd: do file coalescing. [done, needs to be optimized]

Somehow coordinate "external" processes with the cleaner: dump_lfs fsck_lfs alternative cleaner write strategies (hole-plugging?) get rid of the Ifile entry in the root dir, since no user process needs to see it any more (use fhopen/fhstat instead)



Social

Code buy-back/exchange request to Dr. Seltzer [done, but I could always try again]

Kernel

Write up lfs{get,put}pages for UBC, test for basic operation.

Fix the cleaner so that fragment extension works correctly.

Testing