Saturday, September 24, 2011

TestNG vs Junit

When I start setting up the test environment for the functional tests I decided to use Selenium RC. The development of the test scripts was decided to be made in Java and the obvious choice for the testing framework was the Junit. After couple of months in development it came to my attention that there was another testing framework that offered more choices than Junit was. The test framework was TestNG and the decision for switching the framework was made after considering two major factors. The first factor was the adaptation of the existing test scripts to the new framework and the second factor was the flexibility the new framework offered over Junit.

The migration to the new framework was a breeze using the eclipse TestNG pluging refactoring support. The Junit assertions were resolved by importing org.testng.AssertJUnit.*; making the transition effortless.

As explained above the decision for changing the frameworks was based on the flexibility TestNG offered over Junit. For starters the big difference between the testing frameworks is the purpose designed for. Junit was designed for unit testing while TestNG for high level testing. In that sense the capabilities TestNG offers are more aligned to the needs of a functional tester that the needs of an application developer. To make things more clear lets see some of the capabilities TestNG offers that Junit doesn't.

One of the capabilities TestNG offers is the parametric testing. With this capability placing a parameter in the testng.xml file makes the parameter reusable in the test scripts. This capability come handy to our test team in parameterizing the time delay in the selenium.WaitForPageToLoad(“1000”) statements used in the test scripts. With the parametrization of the time delay it became easy to adjust the time delays by changing a file in seconds.

Another capability of the TestNG that Junit does not have is the capability to rerun failed tests. Although I have not used this capability yet, it is pretty obvious that if you have a test suite taking hours to execute; its time saving to rerun only failed tests without re-executing the suite.

One the TestNG capabilities extensively used in our test cases is the capability to depend test steps on other test step. In this way when one of the dependencies fail then the tests depending on the failed one are marked as skipped and not as failed. In this case by making our test steps dependent on the preconditions (@Beforelass) we bind the tests to the preconditions. In this way we know that if one of the preconditions fail the test is not considered failed but skipped.
TestNG is an easy to install and use testing framework. You can use it with Eclipse and lately with Netbeans. Documentation can be found at:

To generate nice reports you can use ReportNG you can find more information on setting up and using ReportNG to the following:

For maven users there is a blog by Marcin Zajączkowski explaining how to setup reportng using maven.
Read More

Sunday, September 18, 2011

Selenium IDE vs Selenium RC

Starting the road trip to the world of automation testing, the choices of testing tools are limitless. One of the most popular and widely used tools for web application testing is the Selenium Suite.
Couple of the options the Selenium suite offers for testing, is the Selenium IDE and the Selenium Resource Control (RC). Both options have pros and cons and it is up to one needs to decide which one to use. 
One of the major disadvantages of the IDE is that only it works for Firefox, that is if you need to test for multiple browsers this is clearly not your choice.
A second major drawback of IDE is that the recorded scripts are easily maintainable for small scale projects but impossible to maintain in big ones. One of the major advantages of the IDE is that it is easily usable by non-technical personnel the simplicity of recording and replaying scenarios is an appealing choice for small scale projects. 
In this blog I will explore the Selenium RC ways for testing large scale web applications. I will present techniques for implementing test scripts and I   will pass my experience for avoiding common mistakes. The experiences I will share with you are real life ones gained in participating in various test teams.
Read More