job_stream is a straightforward and effective way to implement distributed computations, geared towards scientific applications. How straightforward? If we wanted to find all primes between 0 and 999:

# Import the main Work object that manages the computation's graph. from job_stream.inline import Work # Start by declaring work based on the list of numbers between 0 and 999 as a # piece of `Work`. When the w object goes out of context, the job_stream will # be executed. with Work ( range ( 1000 )) as w : # For each of those numbers, see if that number is prime. @w.job def isPrime ( x ): for i in range ( 2 , int ( x ** 0.5 ) + 1 ): if x % i == 0 : return print ( x )

If you wanted to run this script, we’ll call it test.py , on every machine in your cluster:

$ job_stream -- python test.py

If your process is long-running, and machines in the cluster sometimes crash or have other issues that interrupt processes, using job_stream ’s built-in checkpointing can stop your application from having to re-process the same work over and over: