Saturday, September 24, 2011

TestNG vs Junit

When I start setting up the test environment for the functional tests I decided to use Selenium RC. The development of the test scripts was decided to be made in Java and the obvious choice for the testing framework was the Junit. After couple of months in development it came to my attention that there was another testing framework that offered more choices than Junit was. The test framework was TestNG and the decision for switching the framework was made after considering two major factors. The first factor was the adaptation of the existing test scripts to the new framework and the second factor was the flexibility the new framework offered over Junit.

The migration to the new framework was a breeze using the eclipse TestNG pluging refactoring support. The Junit assertions were resolved by importing org.testng.AssertJUnit.*; making the transition effortless.

As explained above the decision for changing the frameworks was based on the flexibility TestNG offered over Junit. For starters the big difference between the testing frameworks is the purpose designed for. Junit was designed for unit testing while TestNG for high level testing. In that sense the capabilities TestNG offers are more aligned to the needs of a functional tester that the needs of an application developer. To make things more clear lets see some of the capabilities TestNG offers that Junit doesn't.

One of the capabilities TestNG offers is the parametric testing. With this capability placing a parameter in the testng.xml file makes the parameter reusable in the test scripts. This capability come handy to our test team in parameterizing the time delay in the selenium.WaitForPageToLoad(“1000”) statements used in the test scripts. With the parametrization of the time delay it became easy to adjust the time delays by changing a file in seconds.

Another capability of the TestNG that Junit does not have is the capability to rerun failed tests. Although I have not used this capability yet, it is pretty obvious that if you have a test suite taking hours to execute; its time saving to rerun only failed tests without re-executing the suite.

One the TestNG capabilities extensively used in our test cases is the capability to depend test steps on other test step. In this way when one of the dependencies fail then the tests depending on the failed one are marked as skipped and not as failed. In this case by making our test steps dependent on the preconditions (@Beforelass) we bind the tests to the preconditions. In this way we know that if one of the preconditions fail the test is not considered failed but skipped.
TestNG is an easy to install and use testing framework. You can use it with Eclipse and lately with Netbeans. Documentation can be found at:

To generate nice reports you can use ReportNG you can find more information on setting up and using ReportNG to the following:

For maven users there is a blog by Marcin Zajączkowski explaining how to setup reportng using maven.