Concurrency testing made easy

Having a multi tenant application under test, concurrency testing is the next step of the testing cycle. Testing using concurrent users takes a lot of time and usually ends up in defects not fully reproducible because of timing issues (synchronization) thus making automation of these tests a must.

Automating concurrent tests using selenium / webdriver always brings up one of the most common question in automation testing; Do I need a new window or open a new tab? The correct question should be: how do I open a new session? And here is the tricky part of the tests. If you open a new window or tab you will carry the session from the window you started from. Some sites will suggest, “clearing the cookies” but this involves the whole browser not a specific window thus the newly assigned session id is the same on all existing windows and tabs.

The solution to the above explained problem is to start two or more selenium / webdriver sessions. To simplify these procedures we use a specific Stevia functionality in order to instantiate 2 webdriver controllers. The instantiation is declared in a spring xml file we call test-controllers and includes the following:

<bean id="webDriverControllerSession1"  class="controllers.ControllerSession1" scope="prototype"/>
<bean id="webdriver-s1-driver" class="com.persado.oss.quality.stevia.selenium.core.controllers.registry.Driver"
   p:name="webdriver-s1"
   p:className="controllers.WebDriverS1WebControllerFactoryImpl"/>
<bean id="webDriverControllerSession2"  class="controllers.ControllerSession2" scope="prototype"/>
<bean id="webdriver-s2-driver" class="com.persado.oss.quality.stevia.selenium.core.controllers.registry.Driver"
   p:name="webdriver-s2"
   p:className="controllers.ControllerFactoryImpl"/>

Note: The spring framework schema p should be included in the xml file, and the controller factory implementation should be created in the appropriate packages.

Having the two controllers at hand we can use a Stevia annotation @RunsWithController(controller=controllers.ControllerSession1.class) before any @BeforeClass or @Test annotations.

The next big problem at hand is how to synchronize the sessions in order to have control of the test execution; synchronization in java is quite advanced, there are a lot of techniques, to mention a few: Cyclic barrier and Count Down Latch. These techniques involve heavy coding and it is an overkill to use them. The most appropriate way to synchronize the execution of tests for different controllers is to use the dependency attribute provided by testNG.

Lets use a specific example to put things into perspective:

 Lets assume there is a phonebook application where you can create a custom phone book and assign contacts. You can edit or delete the phonebook as long as it is empty. So lets assume that in session 1 a user created a phonebook and in session 2 a second user assigns a contact. In session 1 the user is not aware that an entry has been added to the phonebook and wants to edit the phonebook name. As soon as the user presses the edit button he / she is presented with an error message stating that you are not allowed to edit on non-empty phonebook.
In the aforementioned scenario there are 2 clear preconditions, as follows:
• User in session 1 creates phonebook
• User in session 2 adds a contact in phonebook
And a test step with the following action and expected results
User 1 presses the edit button => User is presented with error message

The synchronization sequence is shown below:

For the two preconditions the synchronization could be done using testNG BeforeClass annotations as follows:

@RunsWithController(controllers.ControllerSession1.class)
public void createPreconditionsSession1(String parallelTestsFlag) {
     // User in session 1 creates phonebook
}

@RunsWithController(controllers.ControllerSession2.class)
@BeforeClass(dependsOnMethods = "createPreconditionsSession1")
public void createPreconditionsSession2() throws Exception {
    // User in session 2 adds a contact in phonebook
}

@RunsWithController(controllers.ControllerSession1.class)
@Test(alwaysRun = true, description = "Edit phonebook name")
public void testStep_1() {
    // User presses the edit button => User is presented with error message
}

TestNG will execute createPreconditionsSession1 -> createPreconditionsSession2 -> testStep_1

The next big question that comes to mind with this implementation is how do I run preconditions on different sessions for each step? Lacking a before specific test method and the problem that the usage of a Test annotation is adding noise (step is counted to results as pass even if it is a precondition) we need an extra annotation to execute the precondition. The latest SNAPSHOT of Stevia solves the above problem by introducing an annotation to allow execution of a series of pre and post conditions on a Test annotated step. Lets use the previous example and add a step to our test verifying the story “ User from session 1 created a new phonebook. User in session2 adds a contact. User in session 1 presses the delete button for created phonebook.

@Preconditions(controller=controllers.ControllerSession2.class,value={"test3Pre1", "test3Pre2"}) 
@RunsWithController(controllers. ControllerSession1.class)
@Test(alwaysRun = true, description = "", dependsOnMethods = {"testStep_1"})
public void testStep_2() {
    // User presses the delete button => User is presented with error message
}

public void test3Pre1(){
    // User in session 2 adds a contact on created bucket
}

Note: The second phonebook (delete action test step) is created in the preconditions section along with the first phonebook (edit action test step).

So having these powerful annotations in stevia you can manipulate any test script execution without sacrificing any of the testing standards. As a result the quality is amplified to a new level by having test step pre and post conditions independent of your testing framework (Junit or TestNG).

Categories: Concurrency testing, spring, TestNG, webdriver