Adding the Sizzle CSS Selector library in Webdriver

Selenium Webdriver has a weird opinion on Sizzle. Although (correct me if wrong) in the older Selenium-RC Sizzle was extensively used, in Webdriver it is only injected if the browser does not support native css selectors (see here). This is a pain if you've learned to love Sizzle's ability to "extend"  the normal CSS selector lingo with more advanced spices, and you're mainly working with Firefox, for which Webdriver uses native css.

History

Some background: In our little "ecosystem", we have lots of Selenium-RC code. And when saying lots, we mean LOTS. We are actively thinking to (at some point) take most of it and write it using Webdriver API, but the Sizzle-Webdriver saga is getting in the way. Or better said, was getting in the way.

Overriding Webdriver API

Webdriver uses the "By" class to encapsulate all possible ways of locating an element. This shows nice in your code, because you can have nice little snippets that make sense, e.g.

ExpectedConditions
   .visibilityOfElementLocated(
      By.cssSelector(
        "css=a.confirmation_link:not(.hidden)")) ;

Of course, the above is a Sizzle CSS locator so it would not work in native CSS. Bummer. Studying the Webdriver API for "By" (are you not? check here), we see the class itself is a holder for implementing classes of itself - a nice escape from the typical programming practices. We decided that this class should support Sizzle, and to do so, we extend it, initially to add our hook in it's cssSelector method:

public abstract class ByExtended extends By { 
public static By cssSelector(final String selector) {
   if (selector == null)
         throw new IllegalArgumentException(
           "Cannot find elements when the selector is null");
   return new ByCssSelectorExtended(selector); 
}

See our ByCssSelectorExtended ? (Yes, we're fond of large class names). We've basically extended the inner class inside By, named ByCssSelector, to do our magic. How? Well, basically we used the existing code, only to check if it actually matches something. The code has a nice "habit" of throwing an exception if it does not, so we cleverly "wrapped" it to do our biddings (for brevity, I only give you the code for the 1st method ;):

public static class ByCssSelectorExtended extends ByCssSelector {
  private String ownSelector;
  public ByCssSelectorExtended(String selector) {
    super(selector);
    ownSelector = selector;
  }

  @Override
  public WebElement findElement(SearchContext context) {
     try {
       if (context instanceof FindsByCssSelector) {
         return ((FindsByCssSelector) context) 
           .findElementByCssSelector(ownSelector);
       }
     } catch (InvalidElementStateException e) {
       return findElementBySizzleCss(ownSelector);
     }
     throw new WebDriverException(
       "Driver does not support finding an element by selector: "
       + ownSelector);
  }
  // ... ommitted code ...
}

Key here is our use of InvalidElementStateException - it is the exception thrown by Webdriver if the CSS cannot be located. We're not dealing with the other possibilities thus trying to make this extension function as the original in all other cases.

Our solution

Hope you're not tired so far... here go the juicy parts:

• we want to try the failed css locator with Sizzle (you may want the opposite, Sizzle first then native CSS)
• we want to check if Sizzle exists in the page, if not inject Sizzle so this may work
• finally, we want to try via Sizzle the failed css locator.

To do so, we want two helpers: isSizzleLoaded() and injectSizzleIfNeeded(), both using the Webdriver API to execute stuff via the JavascriptExecutor. I leave these to the readers' imagination and Google Search. Having those, the method findElementBySizzleCss is as simple as:

public WebElement findElementBySizzleCss(String cssLocator) {
  injectSizzleIfNeeded();
  String javascriptExpression = "return Sizzle(\"" +cssLocator + "\")";
  List elements = (List) 
       ((JavascriptExecutor) getDriver())
       .executeScript(javascriptExpression);
  if (elements.size() > 0)
    return (WebElement) elements.get(0);
  return null;
}

In conclusion, going back to our original example, here it is using our "Extended" version of By:

ExpectedConditions
   .visibilityOfElementLocated(
      ByExtended.cssSelector(
        "css=a.confirmation_link:not(.hidden)")) ;

And that's it. Voila!

Afterthoughts

We have given an example of how to override the default Webdriver behaviour towards locators - we are using this as part of our greater API that unifies Webdriver and Selenium-RC to produce a global/common access API for our QA/developers. For the specific "extension", we have created a "wrapper" function that hides our extending of By; all our locators go through that. This is another pattern we generally use - more on that on a subsequent post. Off for now...

UPDATE: We have released our API as open source! Check here and on GitHub! The full solution is there for you to find!

UPDATE #2: The By* extensibility for Sizzle seems very popular! To keep you up to date, our current 0.61.0-SNAPSHOT of Stevia now contains a very important fix for our method:
It was discovered by our users that when a wrong Sizzle selector is provided (one that is syntactically wrong) the Sizzle engine returns null. However, when trying to log this via an exception logger, Webdriver receives an NPE trying to output which selector and type was used; we never did set this up! The exception logged is the  NPE and not the actual exception about the missing/deformed selector, causing confusion. To fix this, we went as true commandos, since the API was protected.

private void fixLocator(SearchContext context, String cssLocator,
    WebElement element) {
 if (element instanceof RemoteWebElement) 
 try {
  Class[] parameterTypes = new Class[] { SearchContext.class,
   String.class, String.class };
  Method m = element.getClass().getDeclaredMethod(
   "setFoundBy", parameterTypes);
  m.setAccessible(true);
  Object[] parameters = new Object[] { context,
   "css selector", cssLocator };
  m.invoke(element, parameters);
 } catch (Exception fail) {
  //NOOP Would like to log here? 
 }
}

As you can see from the extract of code above, we have to modify the accessibility of "setFoundBy" so we can invoke it. After we do that, the call (using the Reflection API) will succeed and hence, no NPE anymore. Enjoy!!! - The actual commit is over here.

Categories: css, sizzle, webdriver