Author
Test coverage for Android UI Automated Tests

Today we are starting the series of important topics on Quality Assurance – all about creating of high-quality, stable, and perfect products from the very beginning of developing process. Our QA Team Lead Yana will share some professional tips and the most interesting cases. Currently we are discussing Android automation tests and sharing our solution of one of the most common complicated cases. Stay tuned!

The profession of QA Engineer used to be associated with the people, who are “not good enough” to become a developer. I’ve heard for several times that “the easiest way to get into IT sphere is to become a tester”. But in fact, it is required to study new technologies and constantly improve yourself to become a professional tester.

1. 2015’s trends in testing:
integration, automation and more

The notion “Quality Assurance” as the separate branch of IT-industry has appeared not so long ago. Full-scale testing  departments existed only within large software/hardware developement companies, where the mistakes were inadmissible. Revolution in creating the standards has become the next step. It became clear, that certain rules must be obeyed for the correct functioning of IT products. So the support departments appeared to receive and customers’ feedbacks and solve problems.

QA priorities

Nowadays most CEOs and managers want their products to meet all the requirements. Companies started to invest more and more in the process of testing, therefore the profession of QA Engineer is just on the start of its development.

QA investment

QA spending cost

The most popular testing trends in 2015 are the following:

1. integrated testing – to involve testers at the very beginning of development process and throughout the whole lifecycle of a product. As it is said: It is much cheaper to fix bug in a documentation than during the release.

bug and feature

2. automation testing: helps to substitute regression testing, e.g the product is developed using the agile method and so on.

automation testing
3. cloud sevices, which provide virtual testing tools;
4. crowdsourced testing – testing, which involves a great number of users (effective for beta-testing).

Using these tools is the key to providing better mobile testing services.Future of testing belongs to automation

2. Future of testing belongs to automation

Nowadays automation of processes becomes popular in all spheres of human life. Society is striving to delegate all the routine responsibilities to machines. Things that appeared to be impossible few years ago, in our time have turned into ordinary processes and testing is not an exception. Though, according to the data of “STATE OF TESTING 2015”, the percentage of QA Automation Engineers on the market equals only 7,5%, more and more clients demand to run the automated tests for checking the programs.

automation

Unit, UI automation, Load tests, as well as usage of automated testing tools, gain popularity for checking the compatibility on different OS and devices.

But, as it has appeared, that’s not enough. Consequently, if you don’t use the tools for calculating code coverage, automated testing won’t guarantee the testing of all functions, test-cases and scenarios. Clover, JaCoCo, Code Coverage, the standard tools of code coverage IntellijIDEA, can be used for this purpose. However, most of them were designed for calculation of Unit test code coverage (for example, when starting test via Clover, the message “Clover Test Optimization supports JUnit configurations only” is shown by IDE). Otherwise, there is no ready-made and full-scale solution for presenting the code coverage of UI automated tests on the market.

clover test optimization

3. How to present the results of automated UI tests

We started to investigate these issues on clients demand to present the visual result of our automated tests for mobile apps testing. After investigating some articles, blogs and forums we found the great project by Oleksandr Kucherenko called Android Test + Espresso + JaCoCo. He has created AndroidJacocoTestRunner, which we have successfully used to get the code coverage of our automated tests.

Unfortunately, Espresso has the range of problems while writing code. For example, this library doesn’t provide the ready-made scroll methods and so on. This is the reason our team have decided to elaborate Oleksandr’s project for it to be able to interact with the Robotium library. Thanks to AndroidJacocoTestRunner’s source code simplicity and clearness, just a few lines had to be changed:

  • change the path to AndroidJacocoTestRunner in build.gradle

defaultConfig { 
    applicationId "com.example.android"
    minSdkVersion 15
    targetSdkVersion 21
    versionCode 1
    versionName "${ getBuildName() }"
    testInstrumentationRunner 
    "com.example.android.AndroidJacocoTestRunner"
}

  • change dependency espresso to robotium and synchronize build.gradle

androidTestCompile'com.jayway.android.robotium:robotium-solo:5.4.1'

After project has been cleaned and rebuilt, IDE notified about successfully passed configuration 🙂 The team started
to write code. Code of the tests that are written with Robotium, doesn’t differ from a usual code and gives us a chance to get a graphic result and there’s no need to write any additional conditions. For example, the test for authorization in the app has been created:


public class TestLogin extends 
ActivityInstrumentationTestCase2<SplashActivity> {
   private Solo solo;
   public TestLogin() {
      super(SplashActivity.class);
   }
   @Override
   public void setUp() throws Exception {
      super.setUp();
      solo = new Solo(this.getInstrumentation(), 
      getActivity());
   }
   @SmallTest
   public void testProfile() throws Exception{
   //check Logout or not
      if (solo.waitForView(R.id.search_button)) {
         solo.clickOnText("Staff");
         solo.scrollToBottom();
         solo.clickOnText("Logout");
      }
      solo.typeText(0, "username");
      solo.clickOnView(solo.getView(R.id.etPassword_ASM));
      solo.typeText(1, "123456");
      solo.clickOnText("LOG IN");
   }
   @Override
   public void tearDown() throws Exception {
      solo.finishOpenedActivities();
   }
}

To get a demonstrative result, test may be started in two ways:
1. from the command line: gradlew :app:connectedAndroidTest

gradlew
2. from IDE – create Android Test in Edit configuration, choose AndroidJacocoTestRunner and press “Run”. Start gradlew :app:jacocoTestReport with a command line to get a graphic result.

IDE graphic result

Result in the form of table (file “index.html”) may be found in <Project name>appbuildreportscoveragedebug directory:

result table
Also there is a possibility to get Test Summary:

test summary

and see which test hasn’t passed and why:

test summary 2

Conclusions

Usage of the tools for calculating code coverage of automated tests increases the speed and efficiency of QA engineer’s work. There is no need to create manual reports and to demonstrate the effectivness of your work. Customer receives the clear result, tester resigns from the obligation of manual reporting. So both sides are satisfied:). Though automized testing is much more complicated than manual. So learn it, as it is interesting, extraordinary and promising. And we will continue investigating this topic in our next articles!