Regression Suites

In our agile project we utilized regression testing with test automation and continuous executions of our automated test scripts. This process served us well while the test execution lasted less than 4 hours but started to create major problems when the total number of test scripts increased and the execution time exceeded 1 day. A quick relief to our problem came with introduction of parallel execution of test scripts but still in cases were the management wanted a quick answer to “Do we deploy on production or not?” it wasn’t enough.

One way to design effective regression suites, to ensure the continuation of business functions, is to follow the rule:

Regression test suite!= sum(Functional test cases)

The purpose of the functional tests is to explore the behavior of specific business functions and highlight corner cases while the purpose of regression tests is to give an overall overview of the entire system. In this way the design of the regression suite should include test scripts inspecting the end-to-end business functions that a system encapsulates.

For the purpose of our project (agile environment) the regression suites were designed with the hypothesis that the GUI, boundary and GUI negative tests should be separated from the runtime tests because:

1.      Changes in the code in boundaries are rare
2.      GUI is indirectly tested (application its self is used for the preconditions of each test case)

This separation leads us automatically in designing different regression suites containing these special categories.

More over, a vertical separation according to execution frequency is needed. In an agile environment in a 2 weeks sprint, 4 sprints release circle, effective execution regression times could be at the end of each sprint, in every other sprint and at the end of the release. The schedule of each regression suite could be decided according to how often this part of the code is changed. For example a boundary test is highly unusual to change thus a planned execution time could be only once per release and preferably one sprint before the release sprint (latest spring usually reserved for sprint new features).

As mentioned before at the end of each sprint the new features suite should executed additively. For example in sprint 1 we have 5 features and in sprint 2 we have 3 features. The regression suite should have cases testing the 5 features (end of sprint 1) and cases 8 features (end of sprint 2)

Read More

How to perform multiple mouse and keyboard actions in Selenium RC and Web Driver

Many applications allow or require a user, to perform multiple or complicated actions, like drag and drop.

In this post we will describe how we can implement, such actions with the use of  SeleniumRC and the Web Driver.

Selenium RC

In the Selenium RC case, a user can perform all the actions upon a selenium instance.

All user interactions are performed sequentially. Some examples are the following:

selenium.dragAndDropToObject(locatorFrom, locatorTo);
selenium.keyDownNative (thekey.getEvent());

Web Driver

In Web Driver, things are a little more complicating, thus interesting. First of all we will introduce class Actions of the selenium (package org.openqa.selenium.interactions).

The Actions class is user-facing API for emulating complex user gestures. Web Driver users can use this class to simulate usage of keyboard or mouse events.

The Actions class contains the keyboard and mouse related actions. For example:

 click(WebElement onElement)  
 clickAndHold(WebElement onElement)
doubleClick()  
 doubleClick(WebElement onElement)
dragAndDropBy(WebElement source, int xOffset,int yOffset) 
 keyDown(Keys theKey) 
 keyDown(WebElement element, Keys theKey)
keyUp(Keys theKey)
keyUp(WebElement element, Keys theKey)
moveByOffset(int xOffset, int yOffset)

In order to use the above mentioned actions the user should do the following

First of all, the user must create an instance Actions class

Actions builder = new Actions(driver);

where driver is a Web Driver instance.

Let us explain all the above, a little...

In the Actions class, the user, can perform one action, by simply calling it into the driver instance, followed by the method perform().

For example, if I wanted to double click somewhere:

Actions builder = new Actions(driver);
driver.doubleclick().perform();

This is ok, in the case that the user wants to perform only one action at a time.

But what happens when a user wants to perform many actions? The obvious way to do it, is to perform these actions sequentially. This is made easier, by the fact that all Actions methods return an Actions object.

builder.clickAndHold(ElementA)
       .moveToElement(ElementB)
       .release(ElementA)
       .keyUp(Keys.CONTROL).perform();

A smoother and cooler way to do the above, would be to wrap all these actions in one method, and just call it.

This can be done with the use of the interface Action. The Action interface only has one action - perform().

Action dropAToB= builder.build();
dropAToB.perform();

 In the above example, we dragged ElementA and dropped it to ElementB, using multiple actions.

As a conclusion, Web Driver wins on points Selenium RC, in terms of simplicity. A selenium user will have to execute all actions separately, and sequentially, whereas, a Web Driver user, will just create an actions-chain, upon the driver instance, and execute all the actions in one line.

Read More

Handling pop ups/alert/confirmations/prompts in Selenium RC / WebDriver

In most web applications the handling of pop ups/alert/confirmations/prompts during the automation testing is a common issue.
In this post we will describe how we can use the SeleniumRC and the Webdriver in order to manage them.

Selenium RC

Selenium RC provides the 3 following methods in order to handle alert-confirmation and prompt pop-ups.

void chooseOkOnNextConfirmation();
void chooseCancelOnNextConfirmation();
void answerOnNextPrompt(answer);

The first two are used for hanling of alert-confirmation pop-ups and the third for prompt pop-ups
The tricky part with these methods is that they must called BEFORE the action that fires the pop-up.

Consider for example the following scenario:

Select a object from the page, Click on Delete button. It triggers a alert saying "Object is going to be deleted, Click OK to Confirm or Cancel to Cancel the Operation".

So if selenium is a Selenium instance, in order to Delete the object, the code snippet in out test automation script should be as follows:

selenium.chooseOkOnNextConfirmation();
selenium.click('locator of Delete button');

In order to Cancel the deletion of the object, the code snippet in out test automation script should be as follows:

selenium.chooseCancelOnNextConfirmation();
selenium.click('locator of Delete button');

Keep in mind that in both cases during the execution of the test automation script, the user WILL NOT SEE the actual alert pop-up but the action is done 'behind the scenes';

The code is similar in the case the action fires a prompt pop-up(the user is prompted to enter a text inside the pop-up)

selenium.answerOnNextPrompt('text');
selenium.click('locator of element that triggered the prompt');

Web Driver

Web Driver provides a more efficient manner to handle all these events through the Alert interface, using the following methods:

void dismiss();
void accept();
String getText();
void sendKeys(String keysToSend);

So if driver is a WebDriver instance, regarding the scenarios we mentioned earlier the code snippet in out test automation script should be as follows:

1st case

driver.findElement(By.id('id of Delete button')).click(); 
Alert myAlert = driver.switchTo().alert(); //Switch to alert pop-up
myAlert.accept(); //accept the alert - equivalent of pressing OK

2nd case

driver.findElement(By.id('id of Delete button')).click(); 
Alert myAlert = driver.switchTo().alert(); 
myAlert.dismiss(); //cancel the alert - equivalent of pressing CANCEL

3rd case

driver.findElement(By.id('id of web element')).click(); 
Alert myAlert = driver.switchTo().alert(); 
alert.sendkeys('text'); //Write text inside the prompt
myAlert.accept();

If you need to wait until the Alert present you can use the following code:

WebDriverWait driver = new WebDriverWait(driver, 10);
Alert myAlert = driver.until(ExpectedConditions.alertIsPresent());

It is important to notice two things here:

1. In the case of WebDriver the methods are called AFTER the action that fires the pop-up(which is the real case scenario), and 
2. During the execution of the test automation script, the user WILL BE ABLE TO SEE the actual alert pop-up (Web Driver simulates exactly the user case scenario).

Have fun...

Read More

Stevia is Released

Stevia 0.6.0 has been released to Maven Central! fetch the latest artifact directly from the Central Repository. To use Stevia, add this to your pom.xml:

<dependency>
    <groupId>com.persado.oss.quality.stevia</groupId>
    <artifactId>stevia-core</artifactId>
    <version>0.6.0</version>
</dependency>

Discover how easy is made to create and execute tests. Run the example script and see how a test can be run in Selenium and WebDriver using Firefox, Chrome, InternetExplorer, Safari and Opera.

Happy hunting!

Read More

Stevia is coming ...

Back in 2011 while implementing the page object maintainability technique in our test code the idea of creating a framework for testing web apps using java and selenium was created. Couple of years later and after some back and forth the framework became to realization and the name of it is: STEVIA.

Stevia is a Java-based framework created by the encapsulation of three major technologies

·         TestNG (a well-known unit testing framework)

·         Selenium (a well-known framework for automation of user actions via browsers)

·         Spring (the most commonplace user library in Java).

While the release of Stevia is imminent to Github  and in the open source community, a first release of our technical document is uploaded in the following link: http://goo.gl/Is3lA for user reactions gathering. Please have a good read through and/or comment if you think more details are needed.

The document will stay for a week uploaded for everyone who wishes to comment and all constructive reviews are more than welcome. For any technical inquiries and details of the release, feel free to ping as at stevia-release [at] persado.com

Stay tuned for the upcoming release early next week.

UPDATE: We have released it! Check here and on GitHub!

Read More

ReportNG Enrichment With Screenshots

Having a vast number of regression tests it became obvious to our QA team that a good reporting tool for reporting the execution results was of great essence. Our first approach in reporting was to use a simple HTML reporting plug-in for TestNG, known as ReportNG. ReportNG was the obvious way to go just because of its simplicity and easiness in integration with TestNG which is our testing framework. ReportNG provided us with a html report presenting each @Test with a pass / fail state.

Although ReportNG presented an excellent solution to our reporting problems it became apparent that for debugging purposes as well as for defect evidence purposes an extension to include screenshots was needed. These screenshots should be attached to every @Test annotation which fails along with the assertion failure.

In this post I will present you how we implemented the enrichment of ReportNG with screenshots taken by using the Selenium RC api.

ReportNG was used simply by included the dependency tag in the pom.xml of our Project, as:

<dependencies>
   <dependency>
     <groupId>org.uncommons</groupId>
     <artifactId>reportng</artifactId>
     <version>${reportngVersion}</version>
     <scope>test</scope>
 <exclusions>
   <exclusion>
      <groupId>org.testng</groupId>
      <artifactId>testng</artifactId>
   </exclusion>
 </exclusions>
    <dependency>
</dependencies>

Selenium provides a function which capture a screenshot of the current window of the browser. We depend on this and we implement a function in Java which captures the screenshot and places it in the corresponding path in the filesystem as:

public void createScreenshot(ITestResult tr) {
 String testName=parseTestCaseName(tr.getTestClass().getRealClass().getName());
 String imagePath="screenshot-" + testName + "-"
                    + (new SimpleDateFormat("yyyyMMddHHmmss").format(new Date()))                    + ".png";
  try {
   Selenium sel = (Selenium) tr.getAttribute("selenium");
   log.info("Parameter selenium is set to :" + sel);
   if (sel != null) {
    String base64Screenshot = sel.captureEntirePageScreenshotToString("");
    byte[] decodedScreenshot = Base64.decodeBase64(base64Screenshot.getBytes());
    FileOutputStream fos = null;
    try {
     File screenShotFname = new File(currentPath,imagePath);
     fos = new FileOutputStream(screenShotFname);
     fos.write(decodedScreenshot);
     log.info("Stored screenshot in file "+screenShotFname.getAbsolutePath());
     reportLogScreenshot(screenShotFname);
    } finally {
     if (null != fos) {
      fos.close();
     }
     } else {
      log.warn("selenium object is not set, cannot get screenshot");    
      log.warn(new Exception("STACKTRACE"));
     }

    } catch (Exception e) {
     e.printStackTrace();
    }

Ok, now with this function we have the screenshot stored in the corresponding path then we need to draw it into the reportNG log,the implementation is the following:

protected void reportLogScreenshot(File file) {
      System.setProperty("org.uncommons.reportng.escape-output", "false");
  
      String absolute = file.getAbsolutePath();
      int beginIndex = absolute.indexOf(".");
      String relative = absolute.substring(beginIndex).replace(".\\","");
      String screenShot = relative.replace('\\','/');
  
  
Reporter.log("<a href=\"" + screenShot + "\"><p align=\"left\">Error screenshot at " + new Date()+ "</p>");
Reporter.log("<p><img width=\"1024\" src=\"" + file.getAbsoluteFile()  + "\" alt=\"screenshot at " + new Date()+ "\"/></p></a><br />"); 
}

HINT: By default ReportNG escape the special characters, so in order not to see the href link in the reported log instead of your screenshot you will need to disable this escaping as:

System.setProperty("org.uncommons.reportng.escape-output", "false");

Voilà, and now the report log print the capture screenshot and display it as:

ReportNG

For the above implementation to work one should encapsulate it in a TestNG listener which calls these functions when the Test fails. We intent to release the aforementioned implementation along with the listener, as part of an upcoming open source release of our testing framework called Stevia. This framework will provide

• Selenium and WebDriver API unification: common API methods used for UI Testing, working for both Selenium and WebDriver!, with a flick of a switch 
• Enhanced verification methods providing element highlighting both in execution and reporting phases
• Enhanced ReportNG reporting with element highlighting and screenshot inclusion, both for Remote (Selenium Grid) and Local (Selenium RC, WebDriver) methods.
• Technical documentation, examples, and a lot more. 

Stay tuned for the official release and corresponding download link.... 

Read More

Setup Groovy in Eclipse

Groovy came in my Test Automation life here in Persado in order to make my daily life more easy and also make my coding more exciting. In this post I will present the way Groovy became part of my IDE, in order to start "groovy"ing.

Firstly, I installed the eclipse Groovy plugin, this was a piece of cake since I was only a few of steps away of it. Via the Marketplace of my Spring Tool Suite (STS): Help - Eclipse Marketplace - Search for Groovy -Install

Install Groovy plugin

Groovy plugin for eclipse can be also found under the following URL: Groovy Plugin for Eclipse

Since, the first step was completed, I moved to the second one and I created a new Groovy class by following the path in Eclipse: File - New - Other - Groovy - Groovy Class.

New Groovy class

  
My java code was already under a Maven project, so I followed the standard maven approach and placed my Groovy classes under src/main/groovy and src/test/groovy and keep my Java classes under src/main/java and src/test/java structure.

Maven Structure

HINT: In case the Groovy folders /src/test/groovy and /scr/main/groovy are not displayed, select your project and by right clicking select Build Path - Configure Build Path - Add Folder then include the folders on your view as:

Display Groovy Folders

Then, as a third step, I converted my project to a Groovy one by just selecting my project then by right clicking on option Configure - Convert to Groovy Project.

Convert to Groovy

Moving forward, I thought that I need to adapt Groovy in my maven word in order to compile my Groovy classes.
In order to achieve it, in the pom.xml, I included the dependency tag for Groovy:

<dependencies>
   <dependency>
     <groupId>org.codehaus.groovy</groupId>
     <artifactId>groovy-all</artifactId>
     <version>${groovyVersion}</version>
   <dependency>
 </dependencies> 

also, i needed a Groovy plugin with the goal “generateStubs”,so included the following tags:

<plugin>
  <groupId>org.codehaus.gmaven</groupId>
    <artifactId>gmaven-plugin</artifactId>
      <configuration>
       <providerSelection>1.8</providerSelection>
      </configuration>
        <executions>
         <execution>
           <goals>
           <goal>generateStubs</goal>
           <goal>compile</goal>
           <goal>generateTestStubs</goal>
           <goal>testCompile</goal>
           </goals>
         </execution>
       </executions>
</plugin>

The goal "generateStubs" was I a surprise for me when I realized that, as its name dedicate,creates java stubs of groovy classes.To understand it, consider a java class which needs a Groovy one to compile with and since maven standard compile process starts with /main and /test java packages and then with the /main and /test Groovy packages, compilation would stop with errors without it.

Finally, on my maven project i run Maven target "clean Install" to install all the dependencies and compile my source Java and Groovy code and i was ready to start rocking with Groovy…..That's all folks!!

Read More