Execution of Assembly-Based Performance Tests on the Ethereum Virtual Machine

This document is intended to walk you through the process of running one set of the performance benchmark tests on the Ethereum Virtual Machine (EVM) on macOS.

To deploy these tests you need to be in the following directory:

/cpp-ethereum/test/unittests/performance

Which is (obviously) a sub-directory of the cpp-ethereum client. Personally I wasn’t able to get that client itself to build from source on my machine, but that isn’t required in order to run these tests- so don’t worry about it.

Here you can see I’ve created system links [1], which connect evm and solc to the directory where I’m executing these tests.

An interesting aside is that, as I’d downloaded these tools using Homebrew, when I issue the command which evm or which solc I get a response like this:

However, if I visit that location I find the following situation:

Cellar is where the packages from Homebrew live, so actually this /usr/local/bin/evm is itself a kind of symlink.

Anyway, with the following command I create a symlink (yo dawg- I heard you like symlinks… ) to make those tools local to /cpp-ethereum/test/unittests/performance :

ln -s /usr/local/Cellar/solidity/0.4.13/bin/solc .

You can also link from the links that Homebrew provides you, like this:

ln -s /usr/local/bin/evm .

If you make a mistake attaching a symlink you can remove it with unlink solc like this:

Now you can run tests.mk , and if your experience is like mine, you’ll get the following error:

Major Impediment / Major Breakthrough:

So here was the big breakthrough I made, after first issuing the command we’ve been considering, namely:

make -f tests.mk SOLC=./solc EVM=./evm all

and seeing the error message:

make: *** No rule to make target `popa.ran', needed by `ops'. Stop.

it became clear to me that I had to learn more about this popa.ran , opening my text editor of choice, Sublime Text I saw the following:

Once I deleted popa.ran I was able to run the script correctly, like this:

Afterwards, if you want to run it again and you issue the same command as we demonstrated above, you’ll be met with the following error:

So then you need to first run this one:

make -f tests.mk SOLC=./solc EVM=./evm clean

The current home for this codebase is here:

Discussion:

Some of the choices described here (such as symlinks) are dirty workarounds for things not being installed right — but workarounds worth knowing. (The makefile lets you use full paths on the command line also.) For solc and evm there are a few choices — you can copy it there, copy it to your working directory, use symlink in your working directory, use full path, add it to PATH, etc. The point is that there is more than one way to skin a cat.

Additional Resources:

Acknowledgements:

These implementation insights, and the tests which we’ve applied them to, are the brain children of Greg Colvin.