Hello Readers!! It gives me immense pleasure in meeting you with yet another article on WebDriver. WebDriver has been creating a lot of buzz among the Selenium community lately and BrowserMob, an active member of the Selenium community, wasted no time in announcing its support for the WebDriver API.

I hope you had the chance to read the great introduction article on the Webdriver API by fellow blogger Ben.

In this article I would like to touch on the basics of scripting with the WebDriver and the BrowserMob API.
I’m sure many of you, who are new to this API are asking yourselves these questions just like I did.

• What is WebDriver?
• Why is it needed?

WebDriver is a framework for automated testing of web applications. WebDriver is a part of Selenium 2.0, which is a major departure from the 1.0 model. Selenium 1.0 is a client-server architecture which uses the RC libraries to program tests that communicate with the server, and the server relays those commands to a browser.

The primary downside of the 1.0 model is that it is written in JavaScript. Browsers impose a strict security model on any JavaScript that they execute in order to protect a user from malicious scripts. Additionally, the API for RC has grown over time that users find it hard to keep up with this ever growing dictionary of methods. The WebDriver helps overcome both these hurdles with an object oriented API that controls the browser itself rather than running as a JavaScript application within the browser.

WebDriver has five major Interfaces – ImeHandler, Navigation, Options, TargetLocator , Timeouts, the methods of which fall into following three categories.

• Control of the browser itself

  Eg: get(), navigate(), close()etc

• Selection of WebElements

  Eg: findElement(), findElements() etc

• Debugging aids

  Eg: getCurrentUrl(), getPageSource(), getTitle() etc

Now that we have a clear understanding of what WebDriver is and what its benefits are, let’s take a deep dive into the various useful WebDriver methods exposed via BrowserMob through some example scripts. Let’s look at this simple script below, to begin with.

var driver = browserMob.openBrowserWebDriver();
 
var tx = browserMob.beginTransaction();
 
var step = browserMob.beginStep("Home Page");
 
driver.get("http://seleniumhq.org/docs/06_test_design_considerations.html");
 
// polls DOM every 500 ms for ten seconds until the text 'Test Design Considerations' is found
driver.manage().timeouts().implicitlyWait(10,  TimeUnit.SECONDS);
 
// Content validation - find element by xpath
driver.findElement(By.xpath("//div[@id='test-design-considerations']/h1[contains(text(), 'Test Design Considerations')]"));
 
browserMob.endStep();
 
browserMob.endTransaction();

Deciphering the script…

• var driver = browserMob.openBrowserWebDriver();
  The openBrowserWebDriver()method of WebDriver starts a WebDriver browser session.
• driver.get(“http://seleniumhq.org/docs/06_test_design_considerations.html”);
  The ‘get’ method which is equivalent to selenium.open() is used to navigate to any web page.
• driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
  driver.findElement(By.xpath(“//div[@id='test-design-considerations']/h1[contains(text(), 'Test Design Considerations')]“));

  WebDriver has a blocking API. i.e. WebDriver will wait until the page has fully loaded (that is, the “onload” event has fired) before returning control to your test or script. However, under some conditions it is possible for a ‘get’ call to return before the page has finished loading. The classic example is JavaScript starting to run after the page has loaded (triggered by onload). Browsers (e.g. Firefox) will notify WebDriver when the basic HTML content has been loaded, which is when WebDriver returns. It’s difficult to know when JavaScript has finished executing, since JS code may schedule functions to be called in the future, depend on server response, etc. WebDriver cannot wait for all conditions to be met before the test proceeds because it does not know them. The solution to ensure whether the element involved in the next interaction is present and ready is to use the ‘Wait’ class to wait for a specific element to appear. This class simply calls findElement over and over, until the element is found (or a timeout has expired). Since this is the behavior desired by default for many users, a mechanism for implicitly-waiting for elements to appear has been implemented. This is accessible through the Timeouts() interface of the WebDriver class (WebDriver.manage().timeouts() ). In the example script above, the driver.manage.timeouts().implicitlyWait call polls the DOM every 500 ms for 10 seconds until the text ‘Test Design Considerations’ is found.

Tips:

• Optionally you can reset the Implicit Timeout to zero using the code below, as once set, it remains effective until the life of the WebDriver instance. The default setting is zero, i.e zero wait time.

  driver.manage().timeouts().implicitlyWait(0,  TimeUnit.SECONDS);

• It is possible to set a timeout for the entire step as such using the setStepTimeout method of the browserMob API.

  // Setting a step timeout of 10 seconds
  browserMob.setStepTimeout(10000);
   
  browserMob.beginStep();
  …….
  browserMob.endStep();

• Alternatively, browserMob’s waitFor() method could be used in place of the implicitlyWait() for content validation. Modifying our script with waitFor()…

  var step = browserMob.beginStep("Home Page");
  driver.get("http://seleniumhq.org/docs/06_test_design_considerations.html");
  browserMob.waitFor(function() {
        return check = driver.findElement(By.xpath("//div[@id='test-design-considerations']/h1")).getText().toLowerCase().startsWith("test design considerations");
      }, 10000);
  if (!check)
  	throw "Content validation failed";
  browserMob.endStep();

Great!! We just wrote our first WebDriver script. We now know how to interact with page elements via the findElement method of WebDriver’s WebElement Interface using XPath. But does WebDriver provide other alternatives for interaction? Absolutely. Below are few alternative ways. As implied through the script, using the ‘findElement’ is a great way to validate page content before proceeding to the next step in the script.

By.cssSelector:
driver.findElement(By.cssSelector(“div#test-design-considerations > h1″));

By.linkText:
driver.findElement(By.linkText(“Data Driven Testing”));

By.partialLinkText:
driver.findElement(By.partialLinkText(“Data Driven”));

By.id:
driver.findElement(By.id(“side”));

By.className:
driver.findElement(By.className(“expandable”));

By.name:
driver.findElement(By.name(“q”));

By.tagName:
driver.findElement(By.tagName(“input”));

driver.findElements:
In situations where you have to iterate through multiple elements with the same id/class/tag name etc or have to pick one randomly, the ‘findElements’ method comes in handy. The example script below uses this method to pick a random suggestion from an AutoSuggest list.

browserMob.beginStep("AutoSuggest");
 
driver.get("http://www.simon.com/default.aspx");
 
var elem = driver.findElement(By.cssSelector("input[id*='KeywordTextBox']"));
 
// clear the search box first
elem.clear();
 
// type Carlsbad in the search box
elem.sendKeys("carlsbad");
 
// wait for autosuggest to populate
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
 
var autoSuggest = driver.findElements(By.xpath("//div[@class='ac_results']/ul/li"));
 
// 'autoSuggest' from above is a Java List. Either use size() to get its length 
// or convert this to an array - autoSuggest.toArray().length
 
browserMob.log(autoSuggest.size());
 
// click on a random element – picking the last element in this case
var selected = driver.findElement(By.xpath("//div[@class='ac_results']/ul/li["+autoSuggest.size()+"]")).click();
 
// throw an error if content validation returns false
if (!driver.getTitle().contains("Search Results"))
	throw 'Content error: "Search Results" not found';
 
browserMob.endStep();

As you can see, I’ve also managed to introduce a few other useful methods through this script. The comments above each one of them describe their purpose.

Tips:

• sendKeys can also be used to simulate pressing of the keyboard keys by using the “Keys” class. It is possible to call sendKeys on any element. The code below triggers the “ENTER’ key press to force a form submission. The list of Keys supported can be found here.

  driver.findElement(By.id('finalExpenses')).sendKeys(Keys.RETURN);

• With advanced locators, it is possible to traverse down the DOM with a single line of code as in below:

  driver.findElements(By.tagName("ul")).get(1).findElements(By.tagName("li")).get(2).click();

  As the index starts with zero, the line of code above would click on the third “li” of the second “ul” on the page.

A few more common page element interactions in detail…

How to handle frames?

Swinging between frames is easy with the switchTo() method of the TargetLocator interface of the WebDriver class.

• Switch to the frame from main window
• By index

  driver.switchTo().frame(0);

• By frame name

  driver.switchTo().frame("frameName");

• Switch back to the parent window

  driver.switchTo().defaultContent();

The following example illustrates a simple swinging between two frames on a web page. Let’s assume the page has two frames, frame1 and frame2, out of which frame1 has a link and frame2 has a form.

driver.switchTo().frame("frame1");
 
driver.findElement(By.linkText("SPAZ")).click();
 
driver.switchTo().defaultContent();
 
driver.switchTo().frame("frame2");
 
driver.findElement(By.id("field_0")).sendKeys("anu");
 
driver.findElement(By.className("form_button")).submit();
 
driver.switchTo().defaultContent();

Try running this snippet commenting out the “driver.switchTo().defaultContent();” before switching to frame2 and you would end up with an error that frame2 cannot be located. This explains the need for switching back to the parent page from inside a frame first, before navigating to another frame on the same page.

How to handle JavaScript Alert and Confirmation boxes?

Confirmation boxes are handled the same way as the Alerts.

• Switch focus to the alert/confirmation box

  var alert = driver.switchTo().alert();

• Dismiss – equivalent to hitting “Cancel” button on the Confirmation box or ignoring an Alert by closing it.

  alert.dismiss();

• Alternatively accept – equivalent to hitting “OK” on the Alert or the Confirmation box

  alert.accept();

• Handle Confirmation boxes the JavaScript way

  driver.executeScript("window.confirm = function(msg){return true;};");

How to handle Pop-up Windows?

browserMob.beginStep("Handle Popup Windows");
 
driver.get("http://www.javascript-coder.com/files/window-popup/javascript-window-open-example1.html");
 
// Get the handle to the current window
var currentHandle = driver.getWindowHandle(); 
 
driver.manage().timeouts().implicitlyWait(10,  TimeUnit.SECONDS);
 
// Open Pop-up
driver.findElement(By.partialLinkText("Open the JavaScript Window")).click();
 
// Switch focus to Pop-up using its name
driver.switchTo().window('mywindow');
 
// Alternate way - if the Pop-up window doesn't have a name, which is very common - 
// get the handle to the last opened window
var handles = driver.getWindowHandles().toArray();
var id = handles.length - 1
driver.switchTo().window(handles[id]);
 
//close popup
driver.executeScript("self.close ();");
 
//switch focus to parent window
driver.switchTo().window(currentHandle);
 
browserMob.endStep();

How to interact with SELECT drop-downs?

var lastItem = driver.findElement(By.xpath("//select[@class='dropDown']/option[last()]"));
 
lastItem.click();

How to simulate advanced interaction such as Drag and Drop?

The Actions subclass of the Selenium Interactions class makes emulation of these complex user gestures easier.

• Drag an element to drop on to another element

  driver.get("http://devfiles.myopera.com/articles/735/example.html");
  var from = driver.findElement(By.xpath("//ul[@id='Fav']/li[3]"));
  var to = driver.findElement(By.xpath("//ul[@id='Tol']/li"));
  Actions(driver).dragAndDrop(from,to).build().perform();

• Drag an element to an offset

  driver.get("http://jqueryui.com/demos/draggable/");
  var draggable = browser.findElement(By.id("draggable")); 
  Actions(driver).dragAndDropBy(draggable, 200, 10).build().perform();

I hope this article served as a comprehensive tutorial, covering most of the basics of WebDriver, if not all. See you soon with more useful WebDriver tips and tricks. Happy testing until then!!!

Select any time coordinate in your test plan and simply add a comment in the annotation field.  Your annotations are tracked and numbered with a flag, so to review, simply reference the number on the flag in the list of annotations made for that test plan.  Don’t worry about making mistakes, you can delete an annotation at any time.

Hello Readers!! Welcome back to Part 3 of our Load Testing Best Practices series, which focuses on writing good load testing scripts.

As I’m sure you would agree to, load testing is an imperative part of every development effort. One of the key components to your load testing efforts is a good load testing script that simulates real user behavior on your web site in the most realistic and accurate way possible.

Before you take a deep dive into the scripting, there are few things you need to plan ahead of time.

• The first one obviously is the scenario outline. Many clients seek the help of their analytics to identify their most visited pages.
• Secondly, identify a reasonable range of pause times between steps.
• As the next step, identify if your script will involve randomization or parameterization. If so, are you planning on using a datafile or an array of random values in the script?

If you have the answers for all of the above then you are half way through already as what’s remaining is packaging this into a script which is the objective of this post. This post is a step by step tutorial on how to write a good load testing script using the Selenium API and the BrowserMob platform. However, some of the tips and information can be used to help with script creation on other platforms as well. We will be focusing on how to organize your scripts, how and why to perform content validation, how to build robust and reusable scripts with the help of optimized selenium locators amongst other tips and guidelines on the best practices. I’m confident that at the end of this article you will be able to create your own load testing scripts in no time.

If you haven’t done so already, you are encouraged to install the following to assist in building the load testing scripts.

• Selenium IDE
• Firebug
• Xpath Finder

What makes a good load testing script?

1. Detailed Comments and Descriptions

Although the presence or absence of these is in no way going to affect the performance of your load test, it is always a good practice to include comments wherever possible as they make a good reference for later use, irrespective of the scripting language under use. It is easy to add comments while scripting with the IDE. Right click on the IDE and choose ‘Insert New Comment’.



Tip: All comments added in the IDE are automatically converted to “Step Descriptions” when the script is uploaded to BrowserMob. i.e the comment in the picture above would be converted to a BrowserMob step description like below:

browserMob.beginStep("Open CNN Home Page");

Giving each step/page a brief description rather than just “Step 1”, “Step 2” etc also helps find out easily how each step/page is performing during the load testing.



2. Content validation combined with proper use of ‘waitForPageToLoad’ and ‘waitForNetworkTrafficToStop’

“Content validation” is an essential part of every load testing script. Content validation is the only means to confirm the right page got downloaded or the right action was performed during a load test and this can be accomplished by searching for an appropriate piece of content in every step that confirms the validity of the page being downloaded. Selenium has various useful commands that can be used for content validation such as verifyTextPresent, VerifyElementPresent, assertTextPresent, assertElementPresent, waitForTextPresent, waitForElementPresent etc.

Tip: While recording actions with IDE, it is easy to add content validation at the appropriate places. Highlight the appropriate piece of content in each page load, right click and choose the appropriate content validation command as in the screenshot below:



Consider the snippets below:

browserMob.beginStep("Open CNN Home Page");
selenium.open("http://www.cnn.com/");
selenium.verifyTextPresent("Latest news");
browserMob.endStep();

[Example: 1a]

browserMob.beginStep("Open CNN Home Page");
selenium.open("http://www.cnn.com/");
selenium.waitForTextPresent("Latest news");
browserMob.endStep();

[Example: 1b]

Which one of the above is the right one?

The first one will fail immediately because the page needs to load completely or at least partially until the content “Latest news” becomes visible. The second one may not fail but is recommended only if you intend to measure the “Perceived Load Time” rather than the complete “Page Load Time”. What is the difference between the two? In the second snippet above (script 1b), the script is going to wait only until the content “Latest news” becomes visible before moving on to the next step. To a user that intends to load the CNN Home page and click on the first article under “Latest news”, this perceived load time is all they are going to care about. Whereas, the complete “Page Load Time” is measured only if the script waits for the page to load completely before moving on to the next step and this can be accomplished by using selenium API’s waitForPageToLoad and/or BrowserMob API’s waitForNetworkTrafficToStop.

Modifying the second snippet from above:

var timeout = 60000;
selenium.setTimeout(timeout);
 
browserMob.beginStep("Open CNN Home Page");
selenium.open("http://www.cnn.com/");
selenium.waitForPageToLoad(timeout);
selenium.verifyTextPresent("Latest news");
browserMob.endStep();

[Example: 1c]

When is BrowserMob API’s waitForNetworkTrafficToStop helpful?

Consider the scenario wherein the user loads the site united.com and tries to find the airport code using the site’s AutoSuggest feature.



Let’s consider the snippet below that tries to simulate the above scenario:

browserMob.beginStep("Step 1: United Airlines Home");
selenium.open("http://www.united.com/");
selenium.waitForPageToLoad(timeout);
selenium.verifyTextPresent(“Travel information”);
browserMob.endStep();
 
browserMob.beginStep("Step 2: Choose Flights using AutoSuggest");
selenium.click("id=shop_from0_temp");
selenium.typeKeys("id=shop_from0_temp", "SAN ");
selenium.mouseOver("css=div#autoSuggestDiv > div");
selenium.clickAt("css=div#autoSuggestDiv > div", "");
………
browserMob.endStep();

[Example: 1d]

Running a validation for the script above threw the error “Element css=div#autoSuggestDiv > div not found”. What was the reason? Look at the waterfall chart for united.com Home page below:








The ‘waitForPageToLoad’ at Step 1 captured the traffic only until the document was complete. But as the waterfall chart above shows, several images, JavaScript files and iframes loaded asynchronously after the document was complete, especially the JavaScript file ‘airportSuggestion.js’ which was required for the autosuggestion at Step 2. Since the script simulated entering of the airport code ‘SAN’ even before the airportSuggestion.js was loaded, the autosuggest did not work as expected, resulting in a script failure.

Here is the modified script that works as expected:

browserMob.beginStep("Step 1: United Airlines Home ");
selenium.open("http://www.united.com/");
browserMob.waitForNetworkTrafficToStop(2000,timeout);
selenium.verifyTextPresent("Travel information");
browserMob.endStep();
 
browserMob.beginStep("Step 2: Choose Flights using AutoSuggest ");
selenium.click("id=shop_from0_temp");
selenium.typeKeys("id=shop_from0_temp", "SAN ");
// wait for the network traffic to stop and the autosuggest to populate
browserMob.waitForNetworkTrafficToStop(2000,timeout);
selenium.mouseOver("css=div#autoSuggestDiv > div");
selenium.clickAt("css=div#autoSuggestDiv > div", "");
browserMob.endStep();

[Example: 1e]

I’d like to highlight one more important aspect of the ‘waitForPageToLoad/waitFor[Text/Element]Present’ and ‘waitForPageToLoad/verify[Text/Element]Present’ combination. While both work fine it is recommended to use the combination waitForPageToLoad/verify[Text/Element]Present for load testing. Why?

Consider the snippets below:

var timeout = 60000;
selenium.setTimeout(timeout);
 
browserMob.beginStep("Open CNN Home Page");
selenium.open("http://www.cnn.com/");
selenium.waitForPageToLoad(timeout);
selenium.verifyTextPresent("Latest news");
browserMob.endStep();

[Example: 1f]

var timeout = 60000;
selenium.setTimeout(timeout);
 
browserMob.beginStep("Open CNN Home Page");
selenium.open("http://www.cnn.com/");
selenium.waitForPageToLoad(timeout);
selenium.waitForTextPresent("Latest news");
browserMob.endStep();

[Example: 1g]

Both would exhibit the same behavior when the page loads successfully. However, in the instance of an error, say an ‘Internal Server’ error or a ‘Connection Reset’ error, the script would declare an error immediately after the error page loads with script 1f while it’ll wait for 60 more seconds to declare the same error with script 1g as the ‘waitForTextPresent(“Latest news”) is going to wait for 60 more seconds, which is the timeout we’ve set in the script, to declare the same content error, irrespective of the page’s status code. Besides,  script 1f will help simulate more transactions in the same amount of time than script 1g, especially in the event of errors.

3. Setting Proper Timeouts

The default selenium timeout is 30 seconds (30000 milliseconds) and the default TimeUnit in use by BrowserMob is milliseconds. It must be remembered that most of the ‘waitFor’ commands like the ‘waitForElementpresent’, ‘waitForVisible’, ‘waitFortextPresent’ etc wait for however long this default selenium timeout is set to. Timeouts can be set the following way in the script:

var timeout = 60000;
selenium.setTimeout(timeout);

It is recommended to set the timeout value before the actual transaction begins in the script. Studies show that the ideal page load time is 5 seconds. Any amount of load time greater than this is considered unacceptable for real users. However, 5 seconds is too small of a timeout threshold for load testing as load times are expected to rise with increase in user load. Hence, 60 seconds to 120 seconds would be considered ideal. The upper limit on the timeout threshold set by BrowserMob is 120 seconds.

Occasionally, you may have the necessity to set a different timeout value for different page loads which can be accomplished by the solution proposed in this article. The article on waitForCondition may also make an interesting read to learn more about advanced handling of timeouts.

4. Optimization of Locators

Selenium provides several ways of locating elements on pages. It may be worthwhile to read through this before proceeding further: http://release.seleniumhq.org/selenium-remote-control/0.9.2/doc/dotnet/Selenium.html

The most commonly used locator is the xpath. FireFox plugins like xpath finder, firebug may be used to identify the correct xpath for the appropriate elements. Below are a couple of useful articles on this topic:

http://blog.browsermob.com/2008/12/using-firebug-for-load-testing/

http://blog.browsermob.com/2009/04/test-your-selenium-xpath-easily-with-firebug/



Optimization of the locators used is extremely important for script robustness and re-usability. In the screenshot below, the IDE has captured the xpath ‘link=Arrested GI had gunpowder, ammo’ when clicking on the first article under “Latest News”. Obviously, the script will work only until this article is available on the Home Page. The xpath shown by firefbug for the same is ‘/html/body/div[5]/div/div[4]/div[3]/div/div[3]/div/ul[2]/li/a’ . Again, this might work temporarily but the moment the site is updated and the DOM changes, the script could break because of this complex xpath as the hierarchy of the elements could have been altered. ‘//ul[@class='cnn_bulletbin']/li[1]/a’ would be a much more optimized and a less brittle xpath for this example. Learn more about locators and pattern matching here.

5. Pause Times

Inserting pause times in between steps is essential in order to simulate real user actions more closely. A range of pause times is recommended over a fixed time as different users browse through a web site at different paces. Both BrowserMob and Selenium API’s pause method can be used to insert think/pause times in between steps.

6. Randomization

In the context of load testing, I want to emphasize the extreme importance of having large datasets available for testing. Many important bugs simply do not surface unless you deal with very large entities such thousands of users, thousands of mail server mailboxes etc. For scenarios involving product checkout, browsing through product pages etc, consider randomizing the product being purchased or browsed through, from a list of several products in order to simulate real user behavior more realistically as not all users are going to be purchasing the same product at the same time. Randomization and parameterization can be achieved in a few ways which are explained in depth here and here.

7. Use of quickStop() and stopExpected()

The BrowserMob API has two methods that come in handy when you want to pause a test that has exceptionally lengthy scripts running, such as loops that repeat certain actions several times over the course of the script execution.

By default, when a test is paused, BrowserMob will wait for the actively running scripts to finish executing and complete the current transaction. Long running scripts with loops or scripts testing a streaming video will not be interrupted until they complete. So in order to abort such transactions immediately on pausing, it is important to include the method browserMob.quickStop() at the top of the script.

However, forcefully aborting the scripts could result in an increased error rate, skewing the results. This can be avoided by using quickStop() in conjunction with stopExpected(). This instructs BrowserMob that the script is expecting to be stopped at any time and if that happens, the sudden stop should not be considered an error.

Here is a complete script, incorporating all of the above:

browserMob.quickStop();
browserMob.stopExpected();
 
var selenium = browserMob.openBrowser();
var c = browserMob.getActiveHttpClient();
 
//  Set the timeout threshold
//  Set a different timeout threshold for local validation as it has a limit on the
// transaction length, capped at 2 minutes.
if (browserMob.isValidation()){
 	var timeout = 30000;
}
else
{
 	var timeout = 120000;
}
 
selenium.setTimeout(timeout);
 
//*************** Randomize think time *********************//
function randomXToY(minVal,maxVal)
{
 	var randVal = minVal+(Math.random()*(maxVal-minVal));
 	return Math.floor(randVal);
}
 
function sleep(x,y)
{
 	if(!browserMob.isValidation()){
  		browserMob.pause(randomXToY(x,y)*1000);
 }
 	else{
  		browserMob.pause(randomXToY(1,2)*1000);
  }
}
 
//********** Randomly pick an articleId, SearchString combination *************//
function randOption()
{
	var list = "1,PERMALINK;2,Related Links;3,Related Articles;4,Most Popular";
	var myArray = list.split(";");
	//pick a random combination from the array every time
	var position = Math.floor(Math.random()*myArray.length);
	var myArray1 = myArray[position];
	return myArray1;
}
 
var selected = randOption().split(",");
var id = selected[0];
var string = selected[1];
 
// Logging the variables being set provides a good means of troubleshooting
browserMob.log(id);
browserMob.log(string);
 
// Begin Transaction
transaction = browserMob.beginTransaction();
 
browserMob.beginStep("Open CNN Home");
selenium.open("http://www.cnn.com/");
selenium.waitForPageToLoad(timeout);
selenium.verifyTextPresent("Latest news");
browserMob.endStep();
 
sleep(2,5);
 
browserMob.beginStep("Click on a random article");
// Randomly clicking on an article with every script run
selenium.click("//ul[@class='cnn_bulletbin']/li["+id+"]/a");
selenium.waitForPageToLoad(timeout);
selenium.verifyTextPresent(string);
browserMob.endStep();
 
browserMob.endTransaction();

As web sites get complex, the scenarios for the load testing tend to get complex and so does the scripting and other load testing requirements such as configuration and set up. Our team of trained load testing experts is waiting to assist you with all your complex load testing requirements. Contact us right away and get all your load testing needs fulfilled!!

Hope you enjoyed reading this article as much as I enjoyed writing it. Coming up next in this Load Testing Best Practices blog series is “Test Execution” so be sure to stay tuned.

Happy Testing!!

More articlesIn the Driver’s Seat Series:
Part One: Planning
Part Two: Configuration

Occasionally, users may want to conduct a load test using cached pages. BrowserMob offers this ability through scripting using RBUs. Typical behavior of a RBU is to the run the script once, close the browser, clear the cache, and start over. To prevent the browser from closing and the cache from being cleared, the steps of the script are placed in a loop.  The loop encases everything between browserMob.beginTransaction(); and browserMob.endTransaction(); which leaves the browser open and prevents the cache from being cleared until the final iteration of the loop and the end of the script.

The first step in setting up a cached load test is to determine the percentage of transactions to run cached. For example, 10 loops will produce a 90% cache rate, 4 loops will produce 75% because the first iteration of the loop will not be cached; there would be 9/10 or 3/4 cached loops respectively.

Once the number of loops is determined, define a variable for the number of loops:

var loops = 10; // 90% cache rate

For script validation, ensure the number of loops has been reduced using the command browserMob.isValidation(); If the loops are not reduced for validation, an invalid script may result with an error message of: “Script took too long and reached timeout limit.”

if (browserMob.isValidation()) {
  loops = 1;
}

Next, enclose the steps of the script inside of a “for” loop, the size of the script does not matter, but make sure the proper syntax is followed and brackets are opened and closed properly. Also, make sure browserMob.beginTransaction(); and browserMob.endTransaction(); are inside the loop. Otherwise an inaccurate count of the number of transactions will result:

for (var i = 0; i < loops; i++) {
  browserMob.beginTransaction();
  // rest of script here...
  // and here...
  // and here...
  // etc...
  browserMob.endTransaction();
}

Finally, the commands browserMob.quickStop(); and browserMob.stopExpected(); need to be somewhere in the script. Normally when a load test is paused or the load level drops to fewer concurrent users, BrowserMob allows the actively running scripts to finish executing and complete the current transaction. Sometimes, especially for very long running scripts, it is desirable to interrupt the script immediately. Calls to browserMob.quickStop(); tell BrowserMob that when a “stop” event (such as a pause, ramp down, or end of a test) occur, the script should attempt to stop executing as quickly as possible. The browserMob.stopExpected(); command tells BrowserMob that the script is expecting that it could be stopped at any time and that if that happens, it should not be considered an error.

Here are all the components put together:

browserMob.quickStop();
browserMob.stopExpected();
 
var timeout = 60000;
selenium.setTimeout(timeout);
 
var selenium = browserMob.openBrowser();
 
var loops = 10; // 90% cache rate
 
if (browserMob.isValidation()) {
  loops = 1;
}
for (var i = 0; i < loops; i++) {
  browserMob.beginTransaction();
  browserMob.beginStep("Step 1");
  // rest of script here...
  browserMob.endStep();
  browserMob.beginStep("Step 2");
  // and here...
  // and here...
  // etc...
  browserMob.endStep();
  browserMob.endTransaction();
}

As you build out your script and plan for a successful load test, it’s important to design the script that incorporates timeout values. BrowserMob has a few types of timeouts that you can use: HTTP-level, Selenium page-level, and Script-level timeouts.

HTTP-level timeouts are controlled by the HTTP client, usually the “c” variable. var c = browserMob.getActiveHttpClient(); and there are 3 commands relating to HTTP timeouts: setConnectionTimeout, setRequestTimeout, and setSocketOperationTimeout.  For more information, check out our API documentation which outlines each of these commands.

Selenium page-level timeouts are controlled via selenium.setTimeout() and affect waitForXyz commands and open commands. The default value is 30 seconds and can be set up to 120 seconds. However, if set higher; the timeout will cause an error when you run a load test.

If the page-level timeouts are changed the HTTP-level timeouts setConnectionTimeout and setSocketOperationTimeout are automatically updated to two times the Selenium timeout (setRequestTimeout is not changed because it is -1 by default, which is infinity).

Selenium commands timeouts are how long Selenium waits for a command to return. This is hardcoded to 135 seconds (purposely designed to be more than the 120 second Selenium timeout). If a command does not return in this time frame, we assume the transaction is broken, shut it down, and throw out the results. This is how we detect crashed browsers and other errors.

Script-level timeout is the time a script has to complete the value. Typically, this is set to infinity for load testing, two minutes for validation, and three minutes for monitoring. Most transactions can complete within two minutes, but some scripts may take longer. The most common reason is the addition of long pauses using browserMob.pause();.  In these cases, include  logic to make those long pauses shorter during validation using the isValidation(); command. See example:

if (browserMob.isValidation()) {
  browserMob.pause(1000);
}
else {
  browserMob.pause(10000);
}

In a load test, simulating think time is important, but during validation it may not be as important. However, if your script still does not validate, try validating it locally using the local-validation tool and then check “bypass validation” when saving the script. Next,  run a small load test (between five to ten users) before running a larger one to confirm the script works as expected.  If there are still issues, feel free to contact support@browsermob.com as you may need to get an  override code to resolve the issue.

selenium.waitForElementPresent(“myelement”);

The waitForElementPresent commands polls the page waiting for the specified element to load. If you are familiar with Webmetrics/WATIR scripting, the command is very similar to the wait_until() command. The waitForElementPresent command is most often used on sites containing dynamic content such as Ajax, JavaScript, etc. Scripts are designed to execute as quickly as possible and Selenium tends to fire steps faster than the DOM can render changes, causing issues/errors with the script playback.

There are a few ways to determine if you need this command:

The first is to check the playback in SeleniumIDE:

This type of error may indicate a timing problem, i.e., the element specified by a locator in your command wasn’t fully loaded when the command was executed. Try putting a pause 5000 before the command to determine whether the problem is indeed related to timing. If so, investigate using an appropriate waitFor* or *AndWait command before the failing command(s).

If playback is successful in the IDE but not on the local-validator or in BrowserMob try adding the command to the script and see if the script passes. For example, you are clicking on a link; if SeleniumIDE picked-up the element as myFooElement when you recorded the script just add the waitForElementPresent command before the click command:

selenium.waitForElementPresent(“myFooElement”);

selenium.click(“myFooElement”);

A third way to check is to examine the validation.txt (generate by the local-validation service) or click the “View Log” link in BrowserMob for the script. If the validation.txt or the log shows an error like the one below try adding the wait command, especially if you are certain the element exists on the page.

***** ERROR DETECTED *****

Error Message: Wrapped com.thoughtworks.selenium.SeleniumException: ERROR: Element myFooElement not found

Error Line Num: 25

***** ERROR DETECTED *****

The above error indicates the element could not be found. This may be an indication the script executed the command before the site had completely loaded.

Setting up your load tests with a blend of RBUs and VUs can be ideal but it does depend on the type of scripts you are running.  Does your script require the user to fill out a form or does the site use AJAX? For those instances, RBUs work best. If your script is about browsing the site as a read-only, VUs work just as well; plus saves you money. VUs is simply HTTP and may be harder to script (but 1/10th the price!).

Hello Readers, Welcome Back !!

As the first part of this Selenium Tips series, this article intends to summarize the different ways to interact with dynamic sites that refresh content asynchronously wherein the script will have to wait for certain elements to appear or disappear before proceeding further. As you are already aware of, Selenium has several ‘waitFor’ commands that fulfill this purpose. waitForCondition is one among them. ‘waitForCondition’ basically takes in two arguments, a JavaScript snippet and a timeout period in milliseconds. The snippet is executed either until it returns true or until the timeout period, after which the command will return an error. Now enough of theory and let’s get into action.

Example:
Variation 1:
Consider the following example where there are two drop-downs, one for the states and the other for counties.
The county drop-down is populated based on the state selection.

(Example courtesy of bitrepository.com)

You can see in the site above, when the state is selected, a spinner shows up for a few seconds before the county drop-down is populated. Using waitForCondition and some JavaScript, we wait until display style attribute changes to ‘none’ in the snippet below.

selenium.open("http://www.bitrepository.com/apps/viewDemo/?originalPost=http://www.bitrepository.com/dynamic-dependant-dropdown-list-us-states-counties.html&demoPage=http://www.bitrepository.com/demo/dynamic-dependent-dropdown-list/");
selenium.selectFrame("mainFrame");
selenium.select("state", "label=California");
selenium.waitForCondition("var value = selenium.browserbot.findElementOrNull('loading_county_drop_down'); value.style.display == 'none'", "10000");
selenium.select("county", "label=Amador County");

Variation 2:
You may also use logical operators to add additional logic.

selenium.waitForCondition("var value = selenium.browserbot.findElementOrNull('loading_county_drop_down'); value.style.display == 'none' || selenium.browserbot.getUserWindow().document.form.county.options[0].text == 'Select a county of California';", "10000");

Variation 3:
The same can also be accomplished by simply using waitForNotVisible and/or waitForSelectOptions.

selenium.waitForNotVisible("loading_county_drop_down");
selenium.waitForSelectOptions("county","regexpi:Amador County");

You may wonder why you need to go into the trouble of using complex JavaScript with waitForCondition while you can accomplish the same with one of the other selenium waitFor commands. The additional advantage of using waitForCondition is that you are able to declare a timeout value explicitly whereas the other waitFor commands simply honor the default selenium timeout value.

Other useful variations:
You may have seen many AJAX intensive sites displaying a ‘Loading..’ or ‘Saving..’ or ‘Processing…’ message when a button or a link is clicked. In such cases you may find this to be helpful to wait for the status text to change or disappear.

Assuming that a ‘Saving’ message is displayed within a div tag when a fictional ‘Save’ button is clicked in the snippet below:

selenium.waitForCondition("var value = selenium.getText(\"//div[@class='save-text']\");value != \"Saving\"", "20000");

In general, .Net sites that use the web form controls, panel controls etc, trigger an asynchronous postback everytime a form control is filled or a .Net tab or panel control is clicked etc. In such cases, I’ve found the following to be very helpful in waiting for the postback to complete.

selenium.waitForCondition("selenium.browserbot.getUserWindow().Sys.WebForms.PageRequestManager.getInstance().get_isInAsyncPostBack() == false;", "10000");

Now for the killer part, for sites that use jQuery, if all you need is to confirm there aren’t any active asynchronous requests, then the following does the trick:

selenium.waitForCondition("selenium.browserbot.getUserWindow().$.active == 0", "10000");

As you can see, there are several possible variations to a single command. The possibilities are endless with Selenium.

Happy Testing !!

While we support Selenium as our core API and Selenium is a popular functional testing toolkit, we do not currently support running tests written in languages like Java/C#/Ruby/Python/etc. There are two reasons we don’t support general unit test-style tests:

First, Often these kinds of tests contain references to resources that aren’t accessible when running the test in the cloud. For example, we’ve seen tests that have hard coded URLs (ie: http://localhost:8080) and reference databases and web services to verify backend functionality.

We run 1000′s of Selenium instances in the cloud and so it is a hard requirement that the tests run “nearby” the browsers to be able to capture the best performance data. As such, these tests are pretty much impossible for us to allow to run from behind your firewall as if they were simple functional tests.

Secondly, and more importantly, functional tests are rarely written to work well as load tests or performance tests. A good performance test has two unique properties:

• Data parameterization – A good load test does not send the same data to the server repeatedly. Instead, it mixes the data up to reflect real world usage. Most functional tests have fixed data inputs specifically to test certain use cases, so these goals are at odds with each other. You can read more about parameterizations here and here.
• Performance timings – A good performance test also breaks up the functional interactions in to “steps” that are aggregated and charted, helping testers easily understand what pages and interactions are slow. A typical functional test has no concern (or even API!) to group parts of the test together.

Because of these reasons, we require that tests be written in a JavaScript. We still permit access to the Selenium API, but we also offer additional APIs that allow for things like data parameterization and capturing performance timings. We chose JavaScript because it’s a neutral language that just about everyone knows and understands.

We are continuing to evaluate whether we can support more traditional functional tests written in Java/C#/Ruby/Python/etc and we may change are policy on this matter in the future. But for now we encourage you to base new performance tests based on existing functional tests, but plan on making customizations to ensure you’re running a load test following all our recommended best practices!

Hello Readers, Welcome back!!

This article intends to explain a few ways of using regular expressions with BrowserMob and Selenium. As we all know already, regular expressions are extremely helpful in scripting dynamic web sites, especially when you run into situations such as picking the first product from a dynamic list of products, clicking on the last link of a dynamic drop-down etc.

To begin with, let’s see a couple of examples for using regular expressions with BrowserMob’s VU (Virtual User) scripts. Not sure what VU is? Check it out here or contact BroserMob Support at anytime.

Example 1:
Single-line Regex – In this example we are trying to find a match against a piece of content spanning across a single line, which is made easy with the BrowserMob method findRegexMatches

In the snippet below, we are trying to parse the first item under the ‘News’ section on yahoo.com using single-line regex.

browserMob.beginStep('Yahoo Home');
var response = c.get('http://www.yahoo.com/',200);
// single-line regex
// getBody() returns the body of the HTTP response
var matches = browserMob.findRegexMatches(response.getBody(), "a class=\"small\" href=\"(.*?)\"");
// logging for troubleshooting purposes
browserMob.log(matches[0]);
var item = matches[0];
browserMob.endStep();
 
browserMob.beginStep('Follow the First News Item');
// go to the first new item parsed from the previous step
var response = c.get(item,200);
browserMob.endStep();

Example 2:
Multi-line Regex – In this example we are trying to find a match against a piece of content spanning across multiple lines. The Javascript ‘RegExp’ object comes handy in this instance.

In the snippet below, we are trying to parse the first hyperlink from html that spans across multiple lines as indicated below. As you can see, there are few newlines separating the ‘ul’ and ‘li’.

<ul class="menu">
 
 
 
 
<li><a href="/website-load-testing">

browserMob.beginStep('BM Home');
var response = c.get('http://browsermob.com/performance-testing',200);
// multi-line regexp
// The regular expression uses '\s' which is any whitespace, including newline, OR \S which is anything NOT a white space
var re = new RegExp(/<ul class="menu">[\s|\S]*?<li>&lt;a href=\&quot;(.*?)\&quot;/i);
var content = response.getBody();
myArray = re.exec(content);
var item = myArray[1];
browserMob.endStep();
 
browserMob.beginStep('Load Testing Home');
// 'item' is the url parsed from the previous step
var response = c.get('http://browsermob.com/'+item,200);
browserMob.endStep();

That was easy !! Now let’s look at a few ways of pattern matching with Selenium for BrowserMob’s RBU scripts.

Selenium supports a few methods that help match text patterns. However, selenium locators don’t accept regular expressions. Only patterns or values accept them.

Globbing:

selenium.click("link=glob:*Gifts"); // Clicks on any link with text suffixed with 'Gifts'
selenium.verifyTextPresent("glob:*Gifts*");

Regular Expressions:[regexp, regexpi]

selenium.click("link=regexpi:^Over \\$[0-9]+$");  //matches links such as 'Over $75', 'Over $85' etc

Contains:

selenium.highlight("//div[contains(@class,'cnn_sectbin')]");  //highlights the first div with class attribute that contains 'cnn_sectbin'
selenium.highlight("css=div#cat_description:contains(\"to last\")");  //locating a div containing the text 'to last' using css selector

Starts-with:

selenium.click("//img[starts-with(@id,'cat_prod_image')]");  //clicks on the first image that has an id attribute that starts with 'cat_prod_image'
selenium.click("//div[starts-with(@id,'tab_dropdown')]/a[last()]");  //clicks on the last link within the div that has a class attribute starting with 'tab_dropdown'
selenium.click("//div[starts-with(@id,'tab_dropdown')]/a[position()=2]"); //clicks on the second link within the div that has a class attribute starting with 'tab_dropdown'
selenium.highlight("css=div[class^='samples']"); //highlights div with class that starts with 'samples'

Ends-with:

selenium.highlight("css=div[class$='fabrics']"); //highlights div with class that ends with 'fabrics'
selenium.click("//img[ends-with(@id,'cat_prod_image')]"); //clicks on the first image that has an id attribute that ends with 'cat_prod_image'

[Note: ends-with is supported only by Xpath 2.0. FF 3 might throw an error for this.]

Happy Testing !!