Test Automation nowadays plays very important aspect in product testing. Hence more companies are focusing on providing Automated Testing services. There are multiple positives to go for test automation. This includes minimizing test execution time, ensuring larger coverage in less time in critical release phases, reliable and repetitive runs during product development phase to ensure no regression issues are introduced. Also it minimizes risk of human error or negligence while doing repetitive regression testing cycles.

There are lot of tools available in market, both open source and paid ones. Though there are pros and cons of using both paid and open source tools, later category of tools are widely used across automation testing community. Choosing right automation tool according to project requirements is very tricky part, as it could get your test strategy past winners mark on other side it could prove to be disaster, decoupling testing and development planning in agile methodologies.

Below are 5 top open source iOS and android automation tools are most widely used and have wide community support.

1) Calabash (For Android and iOS)

What it is – calabash is open source acceptance testing framework for android and iOS automation. Calabash provides separate library for android and iOS automation. It’s a cross platform framework and supports cucumber which enables to express behaviour of app in natural English language implementing BDD (Behaviour Driven Development).

Calabash provides a bridge that allows cucumber tests to run and validate on iOS and Android. Cucumber tests are written as list of statements which forms number of test scenarios. All the statements in cucumber are defined using Ruby language.

In Calabash, a cucumber statement must be defined only once but can be repeated for different scenarios of a cucumber script.

The actual tests are written in Gherkin, backed with Ruby code, and run in the context of the Cucumber Framework.

Pros



Large Community Support. Simple and expressive English like test statements Supports all actions on screen such as swipe, pinch, rotate, tap. Large and enthusiastic community. Cross platform development support(same code works for android as well as iOS devices)

Cons



After failure of test steps, all subsequent steps will be skipped. This can cause more severe product issue to be missed out. Takes time to run tests as it always installs App first by default. However this setting can be overridden by hook that we can configure in code. Requires calabash framework to be instrumented inside ipa file for iOS. We must have code of App in case of iOS. Not friendly to languages other than ruby.

2) Appium (Android and iOS)

What it is – Appium is an open source test automation framework by sauce labs for use with native, hybrid and mobile web apps. Appium library functions inside framework make calls to Appium server running in background which operates the connected device. It uses JSONWireProtocol internally to interact with iOS and Android apps using Selenium’s WebDriver.

Unlike calabash which mainly supports development in ruby, Appium can be used in framework with language of your choice from Java, python, ruby and all other that selenium-webdriver supports.

Appium server gets hosted on Node server. One can start Appium server by firing set of Node commands. If using Appium Standalone Application as server (installable downloaded from Appium website), ‘Inspector’ tool provides ability to locate/identify/operate on all locators of Application configured.

Pros

Supports multiple languages. Does not require Source code access. Cross platform script development. Large community support. Supports script recording on Mac Can extract identifiers using Appium Server application’s ‘Inspector’ tool. In built support for ‘Selendroid’ by Appium Server desktop application. It also uses the vendor provided framework: UIAutomation for iOS, UIAutomator and Selendroid for android 4.2+ and 2.3+ respectively. Supports physical device as well as Emulators. Supports native, hybrid and mobile web application automation.

Cons



Frequent unstable releases for Appium Server desktop application. Script recording not available for Application server desktop application for Windows OS.

What it is – Robotium is open source test framework for developing functional, system and acceptance test scenarios. It is very similar to selenium however only for android. It’s listed under Apache Licence 2.0.

Because of simplicity and ability to create powerful and robust automation scenarios, it’s gaining popularity in test automation community.

It uses run time binding to GUI components hence. It installs test case suite as an application on android device or emulator and hence provides real environment for execution of tests.

Pros

Easy to write test scripts in minimal time. Automation of pre-installed apps is possible. Automatically follows current activity. Faster and robust test execution as compared to Appium due to run time binding with GUI components. Can work without access to code or knowing the implementation of app. Support for Activities, Dialogs, Toasts, Menus, Context Menus and other Android SDK controls.

Cons

Can’t handle flash or web components. Supports development in Java. Can be slow on older devices. Does not support automation of iOS devices which can be deal breaker where test strategy involves automation of both android and iOS device automation. No inbuilt support for record and playback. Paid tools available for recording such as TestDroid and Robotium Recorder.

What it is – Frank is iOS app automation framework and provides ability to write tests scenarios in structured English sentences using cucumber.

Frank requires it to be compiled inside application under test and hence changes are mandatory to your source code. It’s a tool written mainly by some fellow Thought workers that uses a combination of Cucumber and JSON commands that are sent to a server running inside your native application, and leverages UISpec to run the commands.

Pros

Test scenarios are written in understandable English sentences with help of cucumber framework. Symbiote – Live inspection tool is included. Proves efficient when team is having experience on web automation framework with selenium, cucumber. Active community support. Continuously expanding library.

Cons



Limited support for gestures. Bit difficult to run tests on device. Modification to configuration file is required to run on real devices. Recording feature not available.

What it is – UIAutomator is test framework by Google that provides advance UI testing of native android apps and games. It’s java library containing API to create functional UI tests and also an execution engine to run the tests. This library comes in build with android SDK. Many tutorials are available for beginners to get started. Advantage is it gives special privileges to Junit test cases while running to access different processes. Even though it’s good and simpler for native apps automation, it has very limited or almost no support web view automation. It only supports devices with API level 16 and above, however that should not be big factor or deal breaker as most of apps these days support API level 19 and above.

Pros



Simple and easy to learn tutorials are available. Library supported and maintained by google community. Third party paid integration with cloud based test management available.

Cons

Supports only android 4.1 and above. Script recording is not supported. Support is focused in for Java. You can’t get current activity or Instrumentation. ‘WebView’ is not currently supported and hence hybrid apps. Library support is in Java only so it’s very hard if someone wants to blend it with cucumber which is in ruby. However, Java has its own BDD frameworks which can be used, not much in practice though.

Cheat Sheet for Open Source Automation Tools