Swarm of XeBees

• Tweet
• Sharebar

• Tweet

What are the key features an automation framework should have for being successful at an implementation and execution level? Should that be robust, easy-to-use, business-driven, tester friendly? What are the conceptual guidelines that we should follow while designing automation framework? These are the questions that we understand when we have created/seen/used an automation framework that does not have any/most of above-mentioned features. Also some of the root-level requirements of an automation framework that we hear from our clients are:

• It should be easy to update test scripts when there is any UI change on web pages.
• It should be easy to read and understand test script code. Even a quick glance on snippet of code should tell about web page being referred and its underlying functionality.
• Test script execution should be fast.
• Failed test methods should enter enough information into logs so that it becomes easy and quick to pin-point the issue and report to different stakeholders.
• It should have ability to re-run failed test methods.
• It should be able to pick test data from external data sources (txt, xls, csv, sql, xml etc)

So in this post I am going to talk about tools that we should select while targeting Web UI Automation and then how should we leverage benefits of well known patterns to make maintainable,  extensible and usable automation framework .

Selection of Tools
Following are the details of recommended tools for Web UI Automation:

Selenium WebDriver
Selenium is an open source testing framework that allows you to emulate the actions of a user when navigating and interacting with a web site through standard browsers (such as IE, Firefox or Chrome). When your scripts are running your chosen browser pops up and can be watched as your steps are replayed back at you. Selenium 2.0 has many new exciting features and improvements over Selenium 1. The primary new feature is the integration of the WebDriver API. This addresses a number of limitations along with providing an alternative, and simpler, programming interface.

WebDriver supports not only real browsers (IE, Chrome & Firefox) but also headless ones (using HTMLUnit).

Java
Selenium WebDriver’s Java binding is the most preferred platform where one gets access to vast array of open source libraries that exist for Java and moreover Java code is maintainable because it has relatively simple syntax and encourages a fairly standardized way of doing things.

TestNG
TestNG is a testing framework inspired from JUnit and NUnit but it has some new functionalities that make it more powerful and easier to use like:

• Using TestNG's dependsOnMethods attribute, one can easily specify dependent methods. So in case we want the test method 'b' should only execute when test method 'a' has passed, we will be able to do this using dependsOnMethods attribute.
• Ability to rerun failed tests is especially handy in large test suites, and it's a feature you'll only find in TestNG. Anytime there is a failure in TestNG, it creates an XML configuration file that delineates the failed tests.
• One can group our test cases under different test groups like sanity, regression, critical-path or module-wise like payment, third-party-integration, user-management, admin-console etc. and can execute specific test group later on.

Maven
The easiest way to set up a Selenium WebDriver Java project is to use Maven. Maven will download the java bindings (the Selenium 2.0 java client library, TestNG and other required jars) and all its dependencies, and will create the project for us, using a maven pom.xml (project configuration) file. Once we are done with this, we can import the maven project into our preferred IDE, IntelliJ IDEA or Eclipse.

CI Integration
Selenium WebDriver project is a great fit for the continuous integration tools like Jenkins, Hudson, CruiseControl as it can be installed on server testing box and drive/execute test cases remotely on different machines to target browser compatibility with different browsers on different platforms.

Strategies & Patterns

There are lot of issues that people face while doing Web Automation with respect to their maintainability, scalability and difficulty in debugging the failed scenarios. However these are generic issues with every automated testing tool and most likely due to improper design of testing framework. I am listing down few strategies and patterns that can help us in building a good framework:

GUI Repository & Page Factory Pattern
A UI map is a mechanism that stores Identifiers/locators for an application’s UI elements. The test script then uses the UI Map for locating the elements to be tested. Basically, a UI map is a repository of test script objects that correspond to UI elements of the application being tested. UI map has two significant advantages:

• Using a centralized location for UI objects instead of having them scattered throughout the script. This makes script maintenance more efficient.
• Cryptic HTML Identifiers and names can be given more human-readable names improving the readability of test scripts.

In order to support the PageObject pattern, WebDriver's support library contains a factory class (org.openqa.selenium.support.PageFactory) where we use the annotation @FindBy for making the WebElement object to know to which element it belongs to on web page. By using this pattern we need not to write the code for finding each and every Web Element.

Page-Object Pattern
Page Object Pattern is a pattern that displays web page's UI as a class. In addition to UI, functionality of the page is also described in this class. This provides a bridge between page and test.

The idea is to create a level of abstraction to separate the tests from the test subjects, and to provide a simple interface to the elements on the page. Here are the main advantages of Page Object Pattern using:

• Simple and clear tests.
• Good support of tests, because everything is stored in one place.
• Easy creation of new tests. In fact, tests can be created by a person not knowing the features of automation tools.

PageObjects are used as an abstraction to a concrete web page and may be constructed using a PageFactory .

Utilities
There should be a utility package that will have classes providing common functionalities that are needed by scripts. For example method to read configuration/properties file, method to send test automation results via email, method to take screenshot on every failure etc.

Error Handling Module

This module should be designed in such a way so that it provides exact information about state of app when there is an assertion failure. The objective is to minimize the debug time by making available all the resources/info we need so that we can be sure about the bug, steps to reproduce the bug etc. This can be leveraged using:

• Ability to take screen shot on every assertion failure.
• Ability to take screen shot on every exception and log stack trace for easy debugging.
• Ability to log current user name, password and current URL of browser whenever any assertion failure or exception occurs that helps us in debugging.

Reporting &  Logging

ReportNG can be used for Reporting and Logging that will give us clear concise HTML report having statistical information about which all test suites, test groups, test scripts and test methods got executed and how many got passed, failed and skipped.

As logging is an important component of the automated test scripts and a well written logging code offers quick debugging, easy maintenance, and structured storage of an application's runtime information so on the logging front we can use log4j.

There should be  an exhaustive end-2-end HTML report having detailed logs for each and every WebDriver action. This HTML file should have direct links to failed screenshots. Logs might also be available in txt format to reduce the size of the log file. All of these types of logs can be generated via Log4J.

Leveraging TestNG fixtures

Using testng annotations we can use setup and teardown methods at various levels which will allow us to meet pre and post test conditions at various levels.

• @BeforeSuite: The annotated method will be run before all tests in this suite have run.
• @AfterSuite: The annotated method will be run after all tests in this suite have run.
• @BeforeTest: The annotated method will be run before any test method belonging to the classes inside the <test> tag is run.
• @AfterTest: The annotated method will be run after all the test methods belonging to the classes inside the <test> tag have run.
• @BeforeClass: The annotated method will be run before the first test method in the current class is invoked.
• @AfterClass: The annotated method will be run after all the test methods in the current class have been run.
• @BeforeMethod: The annotated method will be run before each test method.
• @AfterMethod: The annotated method will be run after each test method.

Test Data
While writing the test scripts we should not be using hard coded data in test scripts. Both input and expected data should be picked/read from either txt, xls or XML files.

Verification Checkpoints

• We should assert text, value, list of values, presence of element(s) etc. In fact everything displayed on web page can be asserted against expected result so required methods can be created under utilities for same.
• For ajax request/responses, we should keep polling using a loop until a given time period and look for required expected result. So we should take care of that special ajax functionality.
• We should also verify actual results with Database validations checkpoints wherever required.

Best Locator Strategy
Out of available locator strategies (ID, name, link, xpath, css, dom, JSEvents) selection should be based on best suited locator that works for all browsers and is fast also.

There are couple of CSS locators that do not work for Internet Explorer however in general CSS locators are much faster than XPATH locators so we should be using best suited locators. Understand/parse the surrounding HTML for a particular element and create the  locator rather than using locators given by record and playback tools as those locators are not optimized.

Keeping these strategies and design patterns in mind and using these to make informed choices should go a long way in helping build and sustain a successful automation program across any project.

Filed under: Agile Testing, Design Patterns, Quality, Quality Assurance, Selenium, Test Auotmation, WebDriver