shakeThreads

Some APIs are global in nature, if you run two programs that access the Excel API at the same time things start to fail.

Many people have large numbers of CPUs, but only one slow rotating hard drive. If you run ten hard-drive thrashing linkers simultaneously the computer is likely to grind to a halt.

Some proprietary software requires licenses, a fixed number of which can be purchased and managed using a license manager. As an example, the Kansas Lava team only have access to 48 licenses for modelsim.

Limit the number of CPUs to hit your target - for example, the Lava build system could cap the number of CPUs to the number of licenses. People with 24 CPUs might ask the build system to use only 8, so the linkers do not make their machines unusable (and even then, a link heavy rebuild may still harm interactive performance). This solution wastes CPU resources, leaving CPUs that could be building your code idling.

Add locks to suspend jobs that are competing for the shared resource. For example any rule using Excel could take the Excel lock, either a mutex/MVar in some build systems, or creating a file to serve as the lock in make based build systems. Locking can be made to work, but is tricky if you have to fake locks using the file system, and still squanders CPU resources - instead of blocking the CPU should be running another rule.

Resource

Resource

newResource

withResource

shake shakeOptions{shakeThreads=2} $ do want ["a.xls","b.xls"] excel \out -> withResource excel 1 $ system' "excel" [out,...]

disk "exe" | i \out -> withResource disk 1 $ system' "ld" ["-o",out,...] "*.o" *> \out -> system' "cl" ["-o",out,...]

newResource

newResourceIO

newResource

newResourceIO

newResourceIO

shake

newResource

Rules

shakeVersion

String

Int

I've just released Shake 0.9 , a build system library, with a few bug fixes and a bunch of new features (the change log has a complete list). This release contains an incompatible change which makes the Resource feature easier to use, so I thought I'd describe the motivation and use of Resources in Shake. A full upgrade guide is at the bottom of this post.When you run -j10 (=10) you are asking the build system to limit computation so it uses no more than ten CPU resources at a time. The CPU is certainly a precious resource, but there are other resource limitations a build system may need to obey:I know of two approaches used by other build systems to obey resource constraints:The one exception is the Ninja build system which has a concept of "pools" , which properly model finite resources.In Shake thetype represents a finite resource, which multiple build rules can use.values are created withand used with. As an example, only one set of calls to the Excel API can occur at one time, therefore Excel is a finite resource of quantity 1. You can write:Now we will never run two copies of excel simultaneously. Moreover, it will never block waiting for excel if there are other rules that could be run.For most programming languages the compiler is CPU bound but the linker is disk bound. Running 8 linkers will often cause an 8 CPU system to grid to a halt. We can limit ourselves to 4 linkers with:Now we can use 7 or 8 CPUs while still leaving the computer responsive enough to browse the web.Software licenses are another finite resource and can be managed in the same way. For a complete example see the Kansas Lava test program , which uses Shake.In Shake 0.9 thefunction has been renamed to- renametoeverywhere and your code will work again.However, you may have noticed that(as it is now called) forces you to create the resource before calling thefunction, meaning that often the creation and use of the resource are far apart. I have introduced a functionwhich runs in themonad, allowing you to create a resource and then use it nearby. Moving the creation and use of resources closer together makes it much easier to check your resource constraints are met.The only other breaking change is thathas become arather than an, allowing you to store more precise information about the version (for example, your build system might want to encode the GHC version and the version of the build system in the string).Ninja also supports finite resources.