Concurrency testing made easy

Having a multi tenant application under test, concurrency testing is the next step of the testing cycle. Testing using concurrent users takes a lot of time and usually ends up in defects not fully reproducible because of timing issues (synchronization) thus making automation of these tests a must.

Automating concurrent tests using selenium / webdriver always brings up one of the most common question in automation testing; Do I need a new window or open a new tab? The correct question should be: how do I open a new session? And here is the tricky part of the tests. If you open a new window or tab you will carry the session from the window you started from. Some sites will suggest, “clearing the cookies” but this involves the whole browser not a specific window thus the newly assigned session id is the same on all existing windows and tabs.

The solution to the above explained problem is to start two or more selenium / webdriver sessions. To simplify these procedures we use a specific Stevia functionality in order to instantiate 2 webdriver controllers. The instantiation is declared in a spring xml file we call test-controllers and includes the following:

<bean id="webDriverControllerSession1"  class="controllers.ControllerSession1" scope="prototype"/>
<bean id="webdriver-s1-driver" class="com.persado.oss.quality.stevia.selenium.core.controllers.registry.Driver"
   p:name="webdriver-s1"
   p:className="controllers.WebDriverS1WebControllerFactoryImpl"/>
<bean id="webDriverControllerSession2"  class="controllers.ControllerSession2" scope="prototype"/>
<bean id="webdriver-s2-driver" class="com.persado.oss.quality.stevia.selenium.core.controllers.registry.Driver"
   p:name="webdriver-s2"
   p:className="controllers.ControllerFactoryImpl"/>

Note: The spring framework schema p should be included in the xml file, and the controller factory implementation should be created in the appropriate packages.

Having the two controllers at hand we can use a Stevia annotation @RunsWithController(controller=controllers.ControllerSession1.class) before any @BeforeClass or @Test annotations.

The next big problem at hand is how to synchronize the sessions in order to have control of the test execution; synchronization in java is quite advanced, there are a lot of techniques, to mention a few: Cyclic barrier and Count Down Latch. These techniques involve heavy coding and it is an overkill to use them. The most appropriate way to synchronize the execution of tests for different controllers is to use the dependency attribute provided by testNG.

Lets use a specific example to put things into perspective:

 Lets assume there is a phonebook application where you can create a custom phone book and assign contacts. You can edit or delete the phonebook as long as it is empty. So lets assume that in session 1 a user created a phonebook and in session 2 a second user assigns a contact. In session 1 the user is not aware that an entry has been added to the phonebook and wants to edit the phonebook name. As soon as the user presses the edit button he / she is presented with an error message stating that you are not allowed to edit on non-empty phonebook.
In the aforementioned scenario there are 2 clear preconditions, as follows:
• User in session 1 creates phonebook
• User in session 2 adds a contact in phonebook
And a test step with the following action and expected results
User 1 presses the edit button => User is presented with error message

The synchronization sequence is shown below:

For the two preconditions the synchronization could be done using testNG BeforeClass annotations as follows:

@RunsWithController(controllers.ControllerSession1.class)
public void createPreconditionsSession1(String parallelTestsFlag) {
     // User in session 1 creates phonebook
}

@RunsWithController(controllers.ControllerSession2.class)
@BeforeClass(dependsOnMethods = "createPreconditionsSession1")
public void createPreconditionsSession2() throws Exception {
    // User in session 2 adds a contact in phonebook
}

@RunsWithController(controllers.ControllerSession1.class)
@Test(alwaysRun = true, description = "Edit phonebook name")
public void testStep_1() {
    // User presses the edit button => User is presented with error message
}

TestNG will execute createPreconditionsSession1 -> createPreconditionsSession2 -> testStep_1

The next big question that comes to mind with this implementation is how do I run preconditions on different sessions for each step? Lacking a before specific test method and the problem that the usage of a Test annotation is adding noise (step is counted to results as pass even if it is a precondition) we need an extra annotation to execute the precondition. The latest SNAPSHOT of Stevia solves the above problem by introducing an annotation to allow execution of a series of pre and post conditions on a Test annotated step. Lets use the previous example and add a step to our test verifying the story “ User from session 1 created a new phonebook. User in session2 adds a contact. User in session 1 presses the delete button for created phonebook.

@Preconditions(controller=controllers.ControllerSession2.class,value={"test3Pre1", "test3Pre2"}) 
@RunsWithController(controllers. ControllerSession1.class)
@Test(alwaysRun = true, description = "", dependsOnMethods = {"testStep_1"})
public void testStep_2() {
    // User presses the delete button => User is presented with error message
}

public void test3Pre1(){
    // User in session 2 adds a contact on created bucket
}

Note: The second phonebook (delete action test step) is created in the preconditions section along with the first phonebook (edit action test step).

So having these powerful annotations in stevia you can manipulate any test script execution without sacrificing any of the testing standards. As a result the quality is amplified to a new level by having test step pre and post conditions independent of your testing framework (Junit or TestNG).

Read More

Stevia is coming ...

Back in 2011 while implementing the page object maintainability technique in our test code the idea of creating a framework for testing web apps using java and selenium was created. Couple of years later and after some back and forth the framework became to realization and the name of it is: STEVIA.

Stevia is a Java-based framework created by the encapsulation of three major technologies

·         TestNG (a well-known unit testing framework)

·         Selenium (a well-known framework for automation of user actions via browsers)

·         Spring (the most commonplace user library in Java).

While the release of Stevia is imminent to Github  and in the open source community, a first release of our technical document is uploaded in the following link: http://goo.gl/Is3lA for user reactions gathering. Please have a good read through and/or comment if you think more details are needed.

The document will stay for a week uploaded for everyone who wishes to comment and all constructive reviews are more than welcome. For any technical inquiries and details of the release, feel free to ping as at stevia-release [at] persado.com

Stay tuned for the upcoming release early next week.

UPDATE: We have released it! Check here and on GitHub!

Read More

Missing Page Factories in Selenium RC ? Autowire

One of the new features introduced in selenium 2 was the usage of the page factories to easily instantiate a page object. For those who still use selenium and want to have a feature as the aforementioned one the solution is to use the spring source autowire functionality.

Although spring is a powerful framework for java enterprise development, testers that are usually not familiar with the dependency injection model seem to avoid it. The usage of the spring source Autowire feature can help to instantiate your page objects in an easy and efficient way.

In order to start Autowire you need to declare the instantiated a bean (page object) to src/test/resources/applicationContext-test.xml as follows:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:context="http://www.springframework.org/schema/context"
       xmlns:util="http://www.springframework.org/schema/util"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
       http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
       http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-3.0.xsd">
 <context:annotation-config />
 <context:property-placeholder location="classpath:mcs-config.properties" ignore-resource-not-found="true" system-properties-mode="OVERRIDE"  />
 <bean id="firstPageObject" class="com.mycompany.FirstPageObject"/>
</beans>

After the bean has been declared just use the Autowire annotation inside the script to instantiate the page object

@Autowire
FirstPageObject firstPageObject

Likewise for actions resulting to new page objects (click, select) introduce a method returning the resulting page object and Autowire the resulting page object

public FirstPageObject myMethod(){
   return firstPageObject;
}

Read More