Purpose

This article aims to teach you how to run unit tests from terminal and what is the output of those unit tests.

Knowledge of this issue is the first step to automating unit tests execution process.

Beginning

Let’s assume that you just finished the basic funcionality of your android application. The project consists of code but because you care about quality you have also written unit tests. It is just the beginning of your Android adventure so you are not using any continuous integration tool. You always run all of your unit tests manually in the Android Studio IDE :

Right-button click with mouse on test/java folder -> Run Tests in java

You are satisfied because mostly all your tests are passing successfully. Every time you make any changes to your code base, you call them again in same way to check the results. At this stage it is sufficient for your needs.

Motivation

Along with the development of the project, code base becomes bigger and bigger. Instead of one Android developer several people start working on the project with you to meet the business demand for more new features.

You realize that it’s time for automation – tool for calling tests automatically.

The important question arises in your head:

What is needed to call unit test automatically (not with IDE, but by script) and to track results of calling these tests?

I will take you by the hand and guide you to answer this question.

Running unit tests

As you know Gradle build system and Android plugin for Gradle are necessary tools to build your Android application. They also help with unit testing.

Android plugin for Gradle creates specific tasks (for every build variant) that are used to run your unit tests + one general task used to run all build-variant specific tasks.

If your project doesn’t contain any specific flavours it will generate the following tasks:

testDebugUnitTest task will run all unit tests with debug build type testReleaseUnitTest task will run all unit tests with release build type test task will run all specific test gradle tasks (in this case testDebugUnitTest and testReleaseUnitTest)

You can always find these tasks with your gradle tasks tree inside tasks group named verification

If your project contains two flavours (of same dimension) like pro and free it will generate:

test Free DebugUnitTest task will run all unit tests for flavour free with debug build type test Free ReleaseUnitTest task will run all unit tests for flavour free with release build type test Pro DebugUnitTest task will run all unit tests for flavour pro with debug build type testProReleaseUnitTest task will run all unit tests for flavour pro with release build type test task will run all specific test gradle tasks (all above)

You can also read more about those task generation on developer.android site.

Now we know that there are generated tasks, we can simply call specific task from gradle terminal:

gradlew testDebugUnitTest

Calling this test task will:

ensure that the application of specific build type and flavour will be built run all unit tests on this build generate test reports in two formats as a result

Result reports of gradle unit test task

The result reports are generated in two formats: XML and HTML :

Format Type Purpose Directory location XML test result used by different tools projectPath/moduleName/ build/test-results/test Debug UnitTest HTML test result readable for humans projectPath/moduleName /build/reports/tests/test Debug UnitTest

HTML test result

Test result in HTML format is for you. Intended to find out and browse what went well and which test failed. It is easily human readable but paradoxically you will probably not see this report often because XML format with more advanced tools will have more value for you.

XML test result

The result in this format indirectly will be more useful to you.

Result in XML format is set of XML files. Each file represents one test suite and describes test cases with result of execution.

Example XML test suite file with single test case <?xml version="1.0" encoding="UTF-8"?> <testsuite name="com.propaneapps.SimpleCalculatorTest" tests="3" skipped="0" failures="0" errors="0" timestamp="2018-10-22T17:30:29" hostname="DESKTOP-6ESJK06" time="0.072"> <properties/> <testcase name="subtract" classname="com.propaneapps.SimpleCalculatorTest" time="0.006"/> <testcase name="add" classname="com.propaneapps.SimpleCalculatorTest" time="0.011"/> <testcase name="multiply" classname="com.propaneapps.SimpleCalculatorTest" time="0.055"/> <system-out><![CDATA[]]></system-out> <system-err><![CDATA[]]></system-err> </testsuite> 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 <? xml version = "1.0" encoding = "UTF-8" ?> <testsuite name = "com.propaneapps.SimpleCalculatorTest" tests = "3" skipped = "0" failures = "0" errors = "0" timestamp = "2018-10-22T17:30:29" hostname = "DESKTOP-6ESJK06" time = "0.072" > <properties/> <testcase name = "subtract" classname = "com.propaneapps.SimpleCalculatorTest" time = "0.006" /> <testcase name = "add" classname = "com.propaneapps.SimpleCalculatorTest" time = "0.011" /> <testcase name = "multiply" classname = "com.propaneapps.SimpleCalculatorTest" time = "0.055" /> <system-out> <![CDATA[]]> </system-out> <system-err> <![CDATA[]]> </system-err> </testsuite>

Don’t be afraid! You will not see these files content too often. Continuous integration tools like Jenkins, Hudson or Travis will use these tests reports in XML format the best way.

When you want to call unit tests e.g. via Jenkins CI and later process the results of those tests reports in XML format will be the main comunication channel.

What’s next?

Now, when you know how to run from terminal your unit tests and what and where the results are you can use this knowledge to make next step:

use one of the CI automation tools to periodically call these tests for you.

I encourage you to wait for the next upcoming “android tests” series articles (now in progress):