Passing the toggle value from a full-stack test

Keeping the toggle value in a process’ state won’t help us much when writing full-stack tests using a browser to interact with our site.

Requests to our web application are typically initiated by an action inside the browser, like clicking a link, as instructed by the test. So an integration test passes information to the browser, running in a different operating system process, which then issues a web request. The request is then handled in an Erlang process different from the one running the test.

We need a mechanism for communicating from the test process to the process handling the request.

The SQL Sandbox already does this!

Ecto and Phoenix allow us to run end-to-end tests in parallel, to the effect that the rendered page content reflects the state of the database as set up by a test. This window into the database content is state shared between the test and the controller servicing the request — across process boundaries!

Indeed, the Phoenix/Ecto stack has already solved a problem similar to ours. I give a brief overview of the stack and the data flow involved:

each test process checks out a connection from the SQL Sandbox and claims ownership. All database interaction through this connection happens inside a transaction, and all database effects are invisible outside of it.

the test configures the framework responsible for controlling the browser session (Hound or Wallaby) with metadata — containing the test’s PID

when the web request is processed, this metadata is used to grant the process handling the request allowance to the connection owned by the test process

any queries in the web request will subsequently use the same database connection, and act inside the same transaction as the test code.

For the curious, the cross-process mechanism works by adding a payload to the user-agent header, to be parsed in the code starting here.

Although Phoenix.Ecto.SQL.Sandbox has SQL in its name, we can use it for our purposes as well.

There is a test case template for feature tests (these execute the application code end-to-end), a file named similar to test/support/feature_case.ex , that roughly looks like this:

defmodule MyApplication.FeatureCase do

use ExUnit.CaseTemplate



using do

quote do

... # aliases and imports

end

end



setup tags do

:ok = Ecto.Adapters.SQL.Sandbox.checkout(YourApp.Repo)



unless tags[:async] do

Ecto.Adapters.SQL.Sandbox.mode(YourApp.Repo, {:shared, self()})

end



metadata = Phoenix.Ecto.SQL.Sandbox.metadata_for(YourApp.Repo, self())

# Wallaby specific, but looks almost the same when using Hound

{:ok, session} = Wallaby.start_session(metadata: metadata)

{:ok, session: session}

end

end

The last paragraph of this code computes the necessary metadata for the Ecto SQL sandbox mechanism and passes it on to the end-to-end testing framework (Wallaby in our case). We add one line to amend the test framework metadata with information from the test metadata tags :

metadata = Map.put(metadata, :use_new_code?, tags.use_new_code?)

Step 2: Mark tests to use old or new code

The setup in step 1 takes exactly the same test tags as we used above for request tests. We tag all end-to-end tests that require our mechanism in the same way as we did for request tests.

If we forget to tag an end-to-end test, we get an immediate failure because the above setup code is executed, and tags.use_new_code? requires the :use_new_code? key to be present in the metadata map tags .

Step 3: Extract the metadata and pass the flag on to the toggle router

As part of the standard setup for asynchronous end-to-end tests, a plug in the application endpoint is used to extract the metadata and pass it on to Ecto’s SQL sandbox. We do a similar thing right next to it:

defmodule MyApp.Endpoint do

use Phoenix.Endpoint, otp_app: :my_app



if Application.get_env(:my_app, :sql_sandbox) do

plug Phoenix.Ecto.SQL.Sandbox

plug :extract_feature_toggle # <-- ours!

end



def extract_feature_toggle(conn, _) do

conn

|> get_req_header("user-agent")

|> List.first

|> Phoenix.Ecto.SQL.Sandbox.decode_metadata

|> case do

%{use_new_code?: flag} ->

Process.put(:test_use_new_logic?, flag)

_ ->

# No metadata was passed. Happens when hit by request test,

# not end-to-end test. Do nothing.

:ok

end



conn

end



...

end

In the setup for end-to-end tests, we instructed the browser testing framework to add the value for our feature toggle as metadata to all requests. The extract_feature_toggle function plug tries to extract this value.

If present, it writes it to the process dictionary. We have already written our toggle function to accept the toggle value from there because our request tests use that mechanism.