Erlang is a functional language with strong support for concurrency and distribution. It is quite trivial to start Erlang on multiple hosts and connect them. However, you have to write custom scripts to start these nodes. For example, on a set of local machines, ssh with public/private keys can be used to start cluster of nodes. If you don’t have the computing environment, you can lease your servers using Amazon’s EC2 webservice. In this blog, I am going to show how to start Erlang nodes on the instances of EC2. I have broken the instructions into two parts, setting up the EC2 instances and starting the Erlang cluster.

Setting up EC2

Get an Account

If you don’t have S3 and EC2 account, you can get account for Amazon Simple Storage Service

and account for Amazon Elastic Compute Cloud.

Create X.509 certificate

Select the “AWS Access Key Identifiers under “Your Web Services Account” and follow the “Create New” button in this section to create a new X.509 certificate. Also, save them locally.

Download EC2 Toolkit

Download the Command Line Tools.

Setup Environment Variables

export EC2_HOME= export PATH=$PATH:$EC2_HOME/bin export EC2_PRIVATE_KEY=$EC2_HOME/pk-AR7Z7HJUXAIIHL6CR43HD4PGN75KEJNV.pem export EC2_CERT=$EC2_HOME/cert-AR7Z7HJUXAIIHL6CR43HD4PGN75KEJNV.pem

Finding a Suitable AMI

Use following command to view default images:

ec2-describe-images -o self -o amazon

Look for the line containing the public image identified by the ec2-public-images/getting-started.manifest.xml value in the third column. You can also use an image that I created, which includes Erlang distribution with an id of “ami-23c92c4a”.

Generating a Keypair

Use following command to create a keypair:

ec2-add-keypair gsg-keypair

create a file named id_rsa-gsg-keypair and paste everything between (and including) the “—–BEGIN RSA PRIVATE KEY—–” and “—–END RSA PRIVATE KEY—–” lines into it.

Running an Instance

Use following command to start EC2 instance:

ec2-run-instances ami-23c92c4a -k gsg-keypair

and you may see something like:

RESERVATION r-d8fd14b1 275961154068 default INSTANCE i-1246b27b ami-23c92c4a pending gsg-keypair 0 m1.small 2007-12-04T17:34:10+0000

Check status

ec2-describe-instances i-1246b27b

Authorizing Network Access to Your Instances

You can open certain ports you need using

ec2-authorize default -p 22 ec2-authorize default -p 80 ec2-authorize default -p 4369

Connecting to your Instance

ssh -i id_rsa-gsg-keypair root@ec2-72-44-51-166.z-1.compute-1.amazonaws.com

open browser to http://ec2-72-44-51-166.z-1.compute-1.amazonaws.com/

Creating an Image

Modifying an Existing Image

sed -i -e 's/Congratulations!/Congratulations Shahzad Bhatti/' /var/www/html/index.html ls -l /var/www/html/index.html

Preparing for Bundling

Copy your private key and certificate to the machine being bundled.

cd $EC2_HOME scp -i id_rsa-gsg-keypair pk-AR7Z7HJUXAIIHL6CR43HD4PGN75KEJNV.pem cert-AR7Z7HJUXAIIHL6CR43HD4PGN75KEJNV.pem root@ec2-72-44-51-166.z-1.compute-1.amazonaws.com:/mnt

ec2-bundle-vol -d /mnt -k /mnt/pk-AR7Z7HJUXAIIHL6CR43HD4PGN75KEJNV.pem -c /mnt/cert-AR7Z7HJUXAIIHL6CR43HD4PGN75KEJNV.pem -u your-id -s 1536

Bundling

ec2-upload-bundle -b ec2-bucket -m /mnt/image.manifest.xml -a key -s password

Registering the AMI

cd /home/shahbhat/ec2-api-tools-1.2-13740 ec2-register ec2-bucket/image.manifest.xml

Deregistering Your AMI

ec2-deregister ami-47c0252e

Removing Your AMI from Amazon S3

ec2-delete-bundle -b -p image -a -s

Terminating Your Instances

ec2-terminate-instances i-b15bb0d8

/sbin/shutdown -h now

Setting up Erlang Cluster

Here is the fun and easy part:

Starting instances

I am going to start three instances from my Erlang image by opening three different shells and typing following commands:

ssh -i id_rsa-gsg-keypair root@ec2-67-202-33-171.compute-1.amazonaws.com erl -sname master -setcookie ABC Eshell V5.5 (abort with ^G) (master@domU-12-31-38-00-6C-81)1> ssh -i id_rsa-gsg-keypair root@ec2-67-202-33-171.compute-1.amazonaws.com erl -sname slave1 -setcookie ABC Eshell V5.5 (abort with ^G) (slave1@domU-12-31-38-00-6C-81)1> ssh -i id_rsa-gsg-keypair root@ec2-67-202-20-199.compute-1.amazonaws.com erl -sname slave2 -setcookie ABC Eshell V5.5 (abort with ^G) (slave2@domU-12-31-38-00-40-F6)1>

Checking status of instances

Starting instances take a couple of minutes and you can check status with following command:

ec2-describe-instances

and it showed me something like:

RESERVATION r-f7fd149e 275961154068 default INSTANCE i-0146b268 ami-23c92c4a ec2-67-202-33-171.compute-1.amazonaws.com domU-12-31-38-00-6C-81.compute-1.internal running gsg-keypair 0 m1.small 2007-12-04T17:29:39+0000 RESERVATION r-c6fd14af 275961154068 default INSTANCE i-1046b279 ami-23c92c4a ec2-67-202-27-186.compute-1.amazonaws.com domU-12-31-38-00-35-D6.compute-1.internal running gsg-keypair 0 m1.small 2007-12-04T17:33:47+0000 RESERVATION r-d8fd14b1 275961154068 default INSTANCE i-1246b27b ami-23c92c4a ec2-67-202-20-199.compute-1.amazonaws.com domU-12-31-38-00-40-F6.compute-1.internal running gsg-keypair 0 m1.small 2007-12-04T17:34:10+0000

Connecting to the slaves from master

From the master node, I typed following commands to connect to the slave nodes:

net_adm:ping(list_to_atom("slave1@domU-12-31-38-00-6C-81")). net_adm:ping(list_to_atom("slave1@domU-12-31-38-00-6C-81")).

and both returned pong

I then checked connected nodes with following command on the master node:

nodes().

and it returned

[‘slave1@domU-12-31-38-00-6C-81′,’slave2@domU-12-31-38-00-40-F6’]

I also ran the same command on slave1 and slave2 and got

nodes().

[‘master@domU-12-31-38-00-6C-81′,’slave2@domU-12-31-38-00-40-F6’]

(slave1@domU-12-31-38-00-6C-81)2>

nodes().

[‘master@domU-12-31-38-00-6C-81′,’slave1@domU-12-31-38-00-6C-81’]

Testing

I then typed following simple command on the master node to test cluster:

rpc:multicall(nodes(), io, format, ["Hello world~n", []]). and it returned: Hello world Hello world {[ok,ok],[]}

Shutting down instances

Since, EC2 charges based on usage, you need to shutdown instances after you are done.

ec2-describe-instances

which returned:

RESERVATION r-f7fd149e 275961154068 default INSTANCE i-0146b268 ami-23c92c4a ec2-67-202-33-171.compute-1.amazonaws.com domU-12-31-38-00-6C-81.compute-1.internal running gsg-keypair 0 m1.small 2007-12-04T17:29:39+0000 RESERVATION r-c6fd14af 275961154068 default INSTANCE i-1046b279 ami-23c92c4a ec2-67-202-27-186.compute-1.amazonaws.com domU-12-31-38-00-35-D6.compute-1.internal running gsg-keypair 0 m1.small 2007-12-04T17:33:47+0000 RESERVATION r-d8fd14b1 275961154068 default INSTANCE i-1246b27b ami-23c92c4a ec2-67-202-20-199.compute-1.amazonaws.com domU-12-31-38-00-40-F6.compute-1.internal running gsg-keypair 0 m1.small 2007-12-04T17:34:10+0000

I then ran ec2-terminate-instances i-0146b268 i-1046b279 i-1246b27b

Conclusion

As you can see once you have the basic EC2 instances running, starting the Erlang cluster is pretty trivial.