English
汉语
English
Français
Deutsch
By Mike Russo, TSI Senior Software Engineer
Ensuring the quality of software at TSI is serious business. Along with the usual, highly detailed verification testing that is performed on a software application before its release to customers, TSI goes well beyond that by continuously testing the software all the while it is being developed. Our Continuous Integration system can perform both automatic unit- and end-to-end testing automatically whenever developers contribute each increment of work to an application. Automated unit testing is a common part of modern software development and is well supported by a myriad of tools including Mocha and Jest. Therefore, this article will focus on the trickier feat of automated end-to-end testing on mobile apps.
How does End-to-end testing, or E2E, work?
End-to-end (E2E) testing provides a very thorough test of the software by exercising complete uses cases. It goes beyond unit-testing by manipulating a complete application while executing on its intended target platform or an emulator. Therefore, it provides a superior integration test because features can be tested while the app is connected to all external systems. What is needed to perform this testing on a mobile device is a sort of electronic finger that reaches in to the device and virtually taps and swipes at it just like an actual user would. It must also be capable of monitoring the software so that the resulting behavior can be verified against the desired expectations.
Using Appium to run E2E tests
To meet this difficult automation challenge, TSI chose a tool known as Appium. With Appium, we are able to build into our continuous integration pipeline the ability to run E2E tests on both the Android and iOS mobile platforms. Better still, the exact same test script can usually be run against the two platforms, thus increasing developer productivity.
As seen in the figure on the right, an Appium testing environment centers around the Appium server, which accepts UI automation commands from a test client (left) and passes them on to a mobile device (right) that’s running your app. The commands between the client and server conform to the Webdriver API and enable the client to request things from the device such as, “tap the app’s system menu button,” and “tell me what text the app is displaying.”
The device then returns responses to the server, which passes them back to the test client where they can be checked to see if the app is in the expected state. If not, the client reports a failure back to the continuous integration pipeline and the developer can be notified that a quality problem exists. Key to executing the UI automation commands on a mobile device are vendor-provided frameworks. For an iOS device, Apple's XCUITest is used; for Android, Google's UiAutomator/UiAutomator2 is used.
A view inside an E2E test
To illustrate the running of an E2E test on an emulator, I’ve provided screen shots of an app-under-test that simply goes through a typical login scenario. Interspersed with the screen shots, I’ve shown the lines of an Appium test script that automates the UI interaction. The script, written in JavaScript, logs in by entering an operator’s name and tapping a CONTINUE button. It then verifies that the login succeeded by checking that the operator’s name is shown in the app’s header.
const myOperator = 'Jane Doe';
const client = await wdio.remote(opts); // Start a session with Appium server
const textField = await client.$('input');
textField.setValue(myOperator); // Enter the operator’s name in the text box
const button = await client.$('.login-container > button');
await button.click(); // Click/tap the Continue button to log in
const pageHdr = await client.$('table');
const operatorField = await pageHdr.$('td.content');
const operText = await operatorField.getText(); // Get the header text
assert.equal(operText, myOperator); // Verify it contains the entered name
await client.deleteSession(); // All done - shut down the session
What does the E2E test tell us?
Since the UI shows the expected operator name, the application software has PASSED the test! Take note that the test involved more than just the processing done by a login dialog box but extended well beyond, possibly to the communications with an external authentication system! It’s a lot of fun watching your app being driven by a magical electronic finger as Appium runs through all of the steps of your test script! However, when the continuous integration pipeline is executing the test, it doesn’t need to actually see the UI and so Appium can be configured to run the device emulator in a “headless” fashion. In other words, the UI is rendered to memory only and not an actual display.
The above test script is pretty basic in that it locates elements of the UI and performs operations upon them.
In Part 2, I’ll focus on an advanced technique known as Page Object Models that test-writers can use to achieve code reuse and improve maintainability.