Maintainability 101

One of the biggest problem in the automation script development is the script maintenance. Even that there are a lot of widely accepted development methods for improving script maintenance, there are misconceptions that can easily direct testers in writing unmaintainable scripts.

The most popular methods in developing maintainable test code is the Page Objects pattern. Although using the Page Objects pattern will lead in a more maintainable code, the misuse of the pattern will only result in no more than adding an abstraction layer between the page under test and the test script each self. The misunderstandings in applying the pattern in order to create more maintainable test scripts come in the usage of the objects locators. The proper usage of the objects locators would result in the preferred maintainability and not the usage of the pattern. Taking into account that our object locators are locally declared (I totally agree with the analysis of Adam Goucher in his post http://element34.ca/blog/the-great-locator-location-debate) a poorly implementation of the pattern could result in using the page locator in more than one place inside the page object. For example having an input field for which we need to implement an object method to type in a value and an object method to read the inputed value we should use a locally declared locator as a static string instead of having a locator declaration in each method.

public void typeInputValue(Stringvalue){  
    selenium.type(“//input[@name='x']”);
} 

public void readInputValue(){ 
   selenium.getValue(“//input[@name='x']”); 
}

public static final MY_LOCATOR=”//input[@name='x']”;

public void typeInputValue(String value){ 
   selenium.type(MY_LOCATOR); 
} 

public void readInputValue(){ 
   selenium.getValue(MY_LOCATOR); 
}

In the first example a change in the objects locator would result in updating the object in to two places inside the page object rather than in one place as shown in the second example.
One of the goals in test script development is to make our test code more robust in locator breakages. As easily understood the single appearance of object locators in the page objects leads us toward this goal.
At this point I would like to thank Adam Goucher for his wonderful and extremely useful posts regarding test automation. For all it worth I would like to say that his posts lead the way in changing our test scripts to a more robust more maintainable test scripts.

Read More

Selenium WaitFor ...

One of the most difficult problems to overcome when developing automated test scripts is time synchronization. Black box testing is based on expecting specific results upon a specific user actions irrespectively of time. Thus the big question that comes in mind is “when to expect the results?” are the results produced instantly or it takes some time depending on the system load?

One of the solutions presented, is using the time delay available to the programming language used for the test script development. Using Selenium with Java Thread.sleep(“3000”); will do the job. But is this effective? If you are developing automated test scripts for small scale projects and time is not an issue maybe this is your solution; but its advisable to use time delays large enough to account for system slow downs under load.

In large scale projects static time delays are inefficient. Think the amount of time wasted when  hundreds of scripts, having  static time delays, are executed. For these cases the solution to time waiting problem is to use a dynamic waiting period. Dynamic waiting period is the time duration until the system reacts. Testing a web application the reaction of the system would be a page loading. For these cases selenium.WaitForPageToLoad(“3000”); comes in handy. Using WaitForPageToLoad command insures that the system reacts before results verification. Again in this process it is critical to allow large amount of delays to account for system slow downs due to overloading. In our tests it was observed that sometimes a page would load in 10 seconds and under load the page loading could take up to 1 minute, so we adjusted time delays to the observed maximum. The debate in using the WaitForPageToLoad method is what is an acceptable waiting period for a page to load? Is it 1 ,2 ,3 minutes? In debates like that I always like to consider the criticality of the system and the defined requirements for the time responses. In non critical systems with no time requirements defined, it is acceptable to use large delay times (do what you can for the functional tests to complete and the performance tests will optimize the system to react faster under load). In critical systems usually time reaction requirement are well defined so time delays can be extracted through them.

But what happens in systems implemented with Ajax were elements are dynamically refreshed without a page loading? Then the selenium.WaitForPageToLoad(“3000”); it is to no use. In these cases prefer to use the selenium.waitForCondition(); command. When I came up to this situation most of the examples I found to help me were referring me in using JavaScript with the WaitForCondition command. Although this was feasible, using Javascript would have increased the difficulty scale of test script development (taking into account that testers are not required to be experienced programmers) I preferred staying to selenium commands and Xpath references to find dynamically refreshed elements. In some cases I replaced the selenium.WaitForPageToLoad(“3000”); with the WaitForCondition statement in order to have more control into what I am expecting to be loaded.

So a selenium.WaitForPageToLoad(“3000”); can be replaced by

selenium.waitForCondition("selenium.browserbot.getCurrentWindow().document.getElementById('page-heading').innerHTML='New Offer'","10000");

In the above example the waiting condition is specifying which page load to expect according to the page heading. Obviously using the  WaitForPageToLoad statement could result in navigating to the wrong page something that could be avoided using the WaitForCondition statement.
In the above  WaitForCondition example I used the DOM to access the page heading and in the following example I will show how to use the  WaitForCondition using the Xpath

selenium.waitForCondition("selenium.isElementPresent(\"//div[@id='cboxContent']\");", "10000");

In the above example we wait until an element is present (The element's Xpath is specified by the id).

In conclusion WaitForCondition offers better control over the aforementioned solutions controlling what to expect and when thus resulting in test robustness.

Read More

TestNG vs Junit

When I start setting up the test environment for the functional tests I decided to use Selenium RC. The development of the test scripts was decided to be made in Java and the obvious choice for the testing framework was the Junit. After couple of months in development it came to my attention that there was another testing framework that offered more choices than Junit was. The test framework was TestNG and the decision for switching the framework was made after considering two major factors. The first factor was the adaptation of the existing test scripts to the new framework and the second factor was the flexibility the new framework offered over Junit.

The migration to the new framework was a breeze using the eclipse TestNG pluging refactoring support. The Junit assertions were resolved by importing org.testng.AssertJUnit.*; making the transition effortless.

As explained above the decision for changing the frameworks was based on the flexibility TestNG offered over Junit. For starters the big difference between the testing frameworks is the purpose designed for. Junit was designed for unit testing while TestNG for high level testing. In that sense the capabilities TestNG offers are more aligned to the needs of a functional tester that the needs of an application developer. To make things more clear lets see some of the capabilities TestNG offers that Junit doesn't.

One of the capabilities TestNG offers is the parametric testing. With this capability placing a parameter in the testng.xml file makes the parameter reusable in the test scripts. This capability come handy to our test team in parameterizing the time delay in the selenium.WaitForPageToLoad(“1000”) statements used in the test scripts. With the parametrization of the time delay it became easy to adjust the time delays by changing a file in seconds.

Another capability of the TestNG that Junit does not have is the capability to rerun failed tests. Although I have not used this capability yet, it is pretty obvious that if you have a test suite taking hours to execute; its time saving to rerun only failed tests without re-executing the suite.

One the TestNG capabilities extensively used in our test cases is the capability to depend test steps on other test step. In this way when one of the dependencies fail then the tests depending on the failed one are marked as skipped and not as failed. In this case by making our test steps dependent on the preconditions (@Beforelass) we bind the tests to the preconditions. In this way we know that if one of the preconditions fail the test is not considered failed but skipped.

TestNG is an easy to install and use testing framework. You can use it with Eclipse and lately with Netbeans. Documentation can be found at: http://testng.org/

To generate nice reports you can use ReportNG you can find more information on setting up and using ReportNG to the following:

http://reportng.uncommons.org/

For maven users there is a blog by Marcin Zajączkowski explaining how to setup reportng using maven.

Read More

Selenium IDE vs Selenium RC

Starting the road trip to the world of automation testing, the choices of testing tools are limitless. One of the most popular and widely used tools for web application testing is the Selenium Suite.

Couple of the options the Selenium suite offers for testing, is the Selenium IDE and the Selenium Resource Control (RC). Both options have pros and cons and it is up to one needs to decide which one to use. 

One of the major disadvantages of the IDE is that only it works for Firefox, that is if you need to test for multiple browsers this is clearly not your choice.

A second major drawback of IDE is that the recorded scripts are easily maintainable for small scale projects but impossible to maintain in big ones. One of the major advantages of the IDE is that it is easily usable by non-technical personnel the simplicity of recording and replaying scenarios is an appealing choice for small scale projects. 

In this blog I will explore the Selenium RC ways for testing large scale web applications. I will present techniques for implementing test scripts and I   will pass my experience for avoiding common mistakes. The experiences I will share with you are real life ones gained in participating in various test teams.

Read More