Posted on: July 19, 2014

A few months ago at Railsconf, I attended a workshop on Elixir, given by Chris McCord, and have been tinkering with it off and on ever since.

I'd done some functional-style programming in Javascript and Ruby, in the sense of using things like map , function currying, and the like. And I'd played with Clojure a little.

But aside from liking Elixir's syntax more, I'm also struck by its foundation. Building a functional language on top of Erlang is less impressive than building one on Java; Erlang is already functional. But if the task is easier, it's probably easier to do well. Furthermore, with the Erlang foundation already laid, Elixir exists only to make the programmer's job easier.

I'm still pretty new to Elixir, but one of the more tantalizing things I've seen comes from the beginning of Dave Thomas' Programming Elixir. Dave takes the familiar map function and, with a tiny bit of tweaking, makes it run in parallel with as many processes as necessary. (These are VM-level Erlang processes, not system processes; they're cheap, so making 1,000 is no big deal.)

I've copied that example here and added my own comments to clarify what's happening. Note that in Elixir, |> is a "pipe", akin to Unix pipes, so that instead of saying use_data(get_data()) , you can say get_data() |> use_data() .

defmodule Parallel do # Allows mapping over a collection using N parallel processes def pmap ( collection , function ) do # Get this process's PID me = self collection |> Enum . map ( fn ( elem ) -> # For each element in the collection, spawn a process and # tell it to: # - Run the given function on that element # - Call up the parent process # - Send the parent its PID and its result # Each call to spawn_link returns the child PID immediately. spawn_link fn -> ( send me , { self , function . ( elem ) }) end end ) |> # Here we have the complete list of child PIDs. We don't yet know # which, if any, have completed their work Enum . map ( fn ( pid ) -> # For each child PID, in order, block until we receive an # answer from that PID and return the answer # While we're waiting on something from the first pid, we may # get results from others, but we won't "get those out of the # mailbox" until we finish with the first one. receive do { ^ pid , result } -> result end end ) end end

This can be used as follows:

# Calculate squares of these numbers Parallel . pmap ( 1 .. 1_000 , fn ( integer ) -> integer * integer end )

Coming from Ruby, where parallelism is not nearly as straightforward, I find the possibilities here intriguing.

Update 2016-07-09

My friend Jay Hayes points out that this can be done more succinctly with Task as follows:

defmodule Parallel do @doc """ iex> Parallel.map([1,2,3], &(&1*2)) [2,4,6] """ def map ( collection , function ) do collection |> Enum . map ( & Task . async ( fn -> function . ( & 1 ) end )) |> Enum . map ( & Task . await ( & 1 )) end end