We are happy to announce that we recently added support for Firefox 7 to our monitoring and load testing products! You can now explicitly specify the Firefox version your script will run against.  Currently we support both Firefox 3.6 and Firefox 7.

For example, the following script will open the Firefox 7 browser and navigate to a url:

var selenium = browserMob.openBrowser("FF7");
browserMob.beginTransaction();
browserMob.beginStep("Home Page");
selenium.open("http://seleniumhq.org/");
browserMob.endStep();
browserMob.endTransaction();

The same is possible when using the Selenium 2.0 API (WebDriver):

var webDriver = browserMob.openBrowserWebDriver("FF7");
browserMob.beginTransaction();
browserMob.beginStep("Home Page");
webDriver.get("http://seleniumhq.org/");
browserMob.endStep();
browserMob.endTransaction();

Firefox 3.6

Existing monitors and load tests will continue to use Firefox 3 by default.  New monitors will default to using Firefox 7, so you if you want to create a new monitor and have it run against Firefox 3.6, specify “FF3″ as the browser type:

var selenium = browserMob.openBrowser("FF3");

Future Versions

In the future, we will add support for more Firefox versions. If you want to keep up with the latest version of Firefox installed on our systems, you will need to  specify “FF”:

var selenium = browserMob.openBrowser("FF");

When we upgrade to a later version of Firefox, the script will automatically use the new version.

Local Validator

Local validator will use the Firefox version you have installed by default. You can override the locations it looks in for the different browsers in a property file. The file is located in your home directory on Unix and under your user folder on Windows.

$HOME/.browserMob/config.properties

For example, if you have Firefox 7 installed in the default path and Firefox 3 under a different directory,  you can use the following config.properties to point local validator to the Firefox executables:

FF3 = C:/Program Files/Mozilla Firefox 3/firefox.exe
FF7 = C:/Program Files/Mozilla Firefox/firefox.exe

A LinkedIn user recently posed the question: “What are the best practices for monitoring, collecting and analyzing traffic data for a large content site?” Ian White, lead engineer at Neustar, jumped into to tell readers why active monitoring is just as important as analytics. Website monitoring ensures your site is functioning properly anytime from anywhere.

Below we’ve outlined a few of his key points that will help you monitor your site so you can sleep well at night knowing your website is always up and running.

Monitor everything. You need to be alerted whenever a critical system process goes offline. This includes web servers, cache, servers, databases, etc. Understand that you may not know everything you should monitor until it breaks for the first time, but that only serves as lessons learned for additional metrics to monitor and make for a more failsafe website over time.

How you monitor matters. Whether the purpose of your website is to log in a user or add an item to a shopping cart, your customers depend on each click of the mouse to be fully functional. These interactions require monitoring that automates real browser interactions—like sending headers, setting cookies or executing JavaScript—and alerts you if something unexpected occurs. This provides a realistic view of the customer experience.

Monitor from multiple locations. To ensure your website is fully functional from every coordinate on the globe, you need to monitor your site from different geographic locations. There is no “magic number” for how many locations you should monitor from, but we recommend at least three. For example, if your company is based in the U.S., we suggest you monitor from two locations in the States and at least one in Europe or Asia.

Implement Real User Monitoring (RUM). In addition to monitoring your website uptime, you should also test the load times for your website visitors. Similar to Google Analytics, RUM involves placing a JavaScript beacon on your site. Every time a user visits your site, the beacon records the load time and you are alerted whenever this time goes above a specified threshold.

Establish tiered alerting. If a tree falls down in a forest, does it make a sound? The same thought process behind this popular riddle goes into a tiered alerting system. If the individual responsible for fixing a downtime issue doesn’t check his or her email at 3:00am regularly, with a simple email alert system he or she will not hear if/when your site goes offline. Services like PagerDuty ensure that alerts run up the chain of command with continual forms of alerts until someone responds via email, text message or phone call.

For more information on Ian’s response or to join in on the discussion, click here.

It’s almost that time of year again. Holiday shoppers looking to stay at arm’s length from the seasonal mall madness will undoubtedly turn to the Internet to make their gift purchases. In fact, ShopperTrak predicts national retail sales will rise 3% during November and December this year as compared to the same time period last year.

In order to keep up with this expected spike in website traffic, there are a few steps ecommerce companies should take to ready their site. In order to help, we’ve rounded up the top eight load testing and website monitoring tips to help ecommerce sites stay on their “A-game” this holiday season.

1. Start load testing now. Procrastination can be a worrisome habit. Put off today what you can do tomorrow, and you may find your ecommerce site has crashed. According to IBM report, Act, Don’t React: A Proactive Business Continuity Solution Protects your Revenues and Reputation, the average revenue loss per hour of downtime is $1.01 million—a price tag your business cannot afford.

2. Set objectives. Look at industry trends and establish what is “acceptable performance” for your organization. Example Key Performance Indicators (KPI’s) before load testing can include:

• 90% of pages should load in 4 seconds or less
• All business transactions should take less than 1 minute to complete
• Category search should take no more than 6 seconds to complete

3. Use real browsers. Thanks to cloud computing, it is now cost effective to load test and monitor a website using real browsers. Real browsers let you create more life-like site traffic, unlike virtual browsers that simply mimic a browser’s http request / response sequence. You’ll come away with a clearer view of your end user’s experience.

4. Test beyond the firewall. The most frequently visited parts of your site are the most important to test: applications supporting your home page, browsing, product selection, checkout and exit. Without external testing, you’re only getting half the picture.

5. Revisit frequency: Consider increasing your monitoring frequency to make sure nothing goes unnoticed. You need to be notified immediately if there is any change in your website’s behavior.

6. Collaborate. Advertising and special promotions will affect traffic to your website. Coordinate with your marketing department to ensure your website can handle traffic upticks caused by marketing campaigns.

7. Document. Establish a holiday preparation process and document all of your load testing and monitoring activities. That way you’ll have all the records you need to perform a post-mortem and improve for next year.

8. Choose the Solution that Works for Best You. When working with load testing providers, you typically have two choices: (a) on-demand services that let you run tests 24/7 or (b) full-service testing, which normally includes a dedicated engineer. If you have the resources and expertise on staff, and require greater flexibility for when you test, on-demand is a good choice (Tip: See if you can get started with a free trial.)  If you could use additional expertise with testing and analysis, including professional recommendations and reporting, then full-service load testing is the way to go.

Visit http://www.browsermob.com for more information about our on-demand load testing solutions or http://www.webmetrics.com for more information on our performance monitoring and full-service load testing solutions.

After a long night of reviewing the data logs and trying to recover what we lost on 04/21, we anticipate that we can recover about 9 hours of missing data during yesterday’s outage. Obviously, we were hoping to be more successful and looking to recover 100% of the data. But for now, we are able to ascertain and recover the data lost between these time stamps:

• 04/21/2011 at 12:47AM PT to 2:36AM PT
• 04/21/2011 at 8:36AM PT to 3:47PM PT

We do know that Amazon Web Services posted a message on their status page at around  8AM PT notifying us that was a severe issue:

8:12 AM PDT Despite the continued effort from the team to resolve the issue we have not made any meaningful progress for the affected database instances since the last update. Create and Restore requests for RDS database instances are not succeeding in US-EAST-1 region.

So, here’s our next steps:

We will be putting resources – from engineering development to operational processes – in place to prevent this from happening again. In the next few days, we will be sifting through the recovered data and will be getting your reports and charts updated accordingly.

Again, we appreciate your patience!

Thanks and Happy Friday

We are happy to inform you that our monitoring services are now back online and our website is now stable.

Here’s the latest update:

• Washington DC is now enabled for load tests and monitoring.
• Access to web service APIs is now available.
• Charts and reports are now accessible.

However, from 1:00AM PT – 4:00PM PT today (04/21/2011), all reports, charts, and data relating to your monitoring services were disrupted. If you log in to your account now and view your reports or charts, you will notice a gap in your data. We want to ensure you that during this period, monitoring and alerting was not affected. You would have continued to receive downtime alerts.  In an effort to retrieve this data loss, we will be proactively sorting our raw logs and will do our best to backfill the reports and charts so it can be viewed in your account. We know that this information is imperative to your day-to-day activities, so it will be our #1 priority to get this done in the next few days.

We appreciate your support and patience with us today! In the meantime, if you have questions, please contact us so we can address your concerns immediately.

We have been experiencing a disruption in our monitoring services, due to an issue we’ve been wrestling with all morning: Amazon EC2 is having a severe service disruption. You can follow the progress here:  http://status.aws.amazon.com/.

We are working through these issues in conjunction with Amazon EC2’s updates. In the meantime, here’s the latest on our situation:

• Our website has some stability issues.
• We have disabled running load tests from Washington DC.
• At this time, we know that some monitoring data has been lost but still limited on the scope of the loss. We will do our best to retain what we can but our primary focus is to ensure that we get downtime alerts out to you.
• Monitoring is still continuing and alerts are still going out.
• You can still schedule and run load tests in other regions.
• You may have experienced validating your scripts for about a couple hours this morning.  Retry editing and saving your script, and it should validate.
• Our web services API for monitoring data is currently offline.

We will continue to update you as we know it. We sincerely apologize for the inconvenience. A monitoring system is only as good as the confidence you have in it, and do hope that you will continue to have confidence in BrowserMob monitoring as time goes on.

If you have concerns or questions, please feel free to contact us.

Hello Readers, Welcome Back !!

As the first part of this Selenium Tips series, this article intends to summarize the different ways to interact with dynamic sites that refresh content asynchronously wherein the script will have to wait for certain elements to appear or disappear before proceeding further. As you are already aware of, Selenium has several ‘waitFor’ commands that fulfill this purpose. waitForCondition is one among them. ‘waitForCondition’ basically takes in two arguments, a JavaScript snippet and a timeout period in milliseconds. The snippet is executed either until it returns true or until the timeout period, after which the command will return an error. Now enough of theory and let’s get into action.

Example:
Variation 1:
Consider the following example where there are two drop-downs, one for the states and the other for counties.
The county drop-down is populated based on the state selection.

(Example courtesy of bitrepository.com)

You can see in the site above, when the state is selected, a spinner shows up for a few seconds before the county drop-down is populated. Using waitForCondition and some JavaScript, we wait until display style attribute changes to ‘none’ in the snippet below.

selenium.open("http://www.bitrepository.com/apps/viewDemo/?originalPost=http://www.bitrepository.com/dynamic-dependant-dropdown-list-us-states-counties.html&demoPage=http://www.bitrepository.com/demo/dynamic-dependent-dropdown-list/");
selenium.selectFrame("mainFrame");
selenium.select("state", "label=California");
selenium.waitForCondition("var value = selenium.browserbot.findElementOrNull('loading_county_drop_down'); value.style.display == 'none'", "10000");
selenium.select("county", "label=Amador County");

Variation 2:
You may also use logical operators to add additional logic.

selenium.waitForCondition("var value = selenium.browserbot.findElementOrNull('loading_county_drop_down'); value.style.display == 'none' || selenium.browserbot.getUserWindow().document.form.county.options[0].text == 'Select a county of California';", "10000");

Variation 3:
The same can also be accomplished by simply using waitForNotVisible and/or waitForSelectOptions.

selenium.waitForNotVisible("loading_county_drop_down");
selenium.waitForSelectOptions("county","regexpi:Amador County");

You may wonder why you need to go into the trouble of using complex JavaScript with waitForCondition while you can accomplish the same with one of the other selenium waitFor commands. The additional advantage of using waitForCondition is that you are able to declare a timeout value explicitly whereas the other waitFor commands simply honor the default selenium timeout value.

Other useful variations:
You may have seen many AJAX intensive sites displaying a ‘Loading..’ or ‘Saving..’ or ‘Processing…’ message when a button or a link is clicked. In such cases you may find this to be helpful to wait for the status text to change or disappear.

Assuming that a ‘Saving’ message is displayed within a div tag when a fictional ‘Save’ button is clicked in the snippet below:

selenium.waitForCondition("var value = selenium.getText(\"//div[@class='save-text']\");value != \"Saving\"", "20000");

In general, .Net sites that use the web form controls, panel controls etc, trigger an asynchronous postback everytime a form control is filled or a .Net tab or panel control is clicked etc. In such cases, I’ve found the following to be very helpful in waiting for the postback to complete.

selenium.waitForCondition("selenium.browserbot.getUserWindow().Sys.WebForms.PageRequestManager.getInstance().get_isInAsyncPostBack() == false;", "10000");

Now for the killer part, for sites that use jQuery, if all you need is to confirm there aren’t any active asynchronous requests, then the following does the trick:

selenium.waitForCondition("selenium.browserbot.getUserWindow().$.active == 0", "10000");

As you can see, there are several possible variations to a single command. The possibilities are endless with Selenium.

Happy Testing !!

After hearing your suggestions on monitoring alerts, we’ve streamlined the number of alerts you get due to intermittent or localized problems. Before, whenever a site encountered an error condition while taking a sample, that sample may have been logged as an error or strike, causing an alert to be generated.  Now, you’ll receive an email alert only after we take a sample three times and still find an error. We’ll report the 1st and 2nd failures in your charts–and they’ll show up as additional checks to your normal monitoring–but we won’t clog up your inbox with extraneous alerts. Thanks much to all who provided feedback. You’ve helped us serve you more efficiently!

Hello Readers, Welcome back!!

This article intends to explain a few ways of using regular expressions with BrowserMob and Selenium. As we all know already, regular expressions are extremely helpful in scripting dynamic web sites, especially when you run into situations such as picking the first product from a dynamic list of products, clicking on the last link of a dynamic drop-down etc.

To begin with, let’s see a couple of examples for using regular expressions with BrowserMob’s VU (Virtual User) scripts. Not sure what VU is? Check it out here or contact BroserMob Support at anytime.

Example 1:
Single-line Regex – In this example we are trying to find a match against a piece of content spanning across a single line, which is made easy with the BrowserMob method findRegexMatches

In the snippet below, we are trying to parse the first item under the ‘News’ section on yahoo.com using single-line regex.

browserMob.beginStep('Yahoo Home');
var response = c.get('http://www.yahoo.com/',200);
// single-line regex
// getBody() returns the body of the HTTP response
var matches = browserMob.findRegexMatches(response.getBody(), "a class=\"small\" href=\"(.*?)\"");
// logging for troubleshooting purposes
browserMob.log(matches[0]);
var item = matches[0];
browserMob.endStep();
 
browserMob.beginStep('Follow the First News Item');
// go to the first new item parsed from the previous step
var response = c.get(item,200);
browserMob.endStep();

Example 2:
Multi-line Regex – In this example we are trying to find a match against a piece of content spanning across multiple lines. The Javascript ‘RegExp’ object comes handy in this instance.

In the snippet below, we are trying to parse the first hyperlink from html that spans across multiple lines as indicated below. As you can see, there are few newlines separating the ‘ul’ and ‘li’.

<ul class="menu">
 
 
 
 
<li><a href="/website-load-testing">

browserMob.beginStep('BM Home');
var response = c.get('http://browsermob.com/performance-testing',200);
// multi-line regexp
// The regular expression uses '\s' which is any whitespace, including newline, OR \S which is anything NOT a white space
var re = new RegExp(/<ul class="menu">[\s|\S]*?<li>&lt;a href=\&quot;(.*?)\&quot;/i);
var content = response.getBody();
myArray = re.exec(content);
var item = myArray[1];
browserMob.endStep();
 
browserMob.beginStep('Load Testing Home');
// 'item' is the url parsed from the previous step
var response = c.get('http://browsermob.com/'+item,200);
browserMob.endStep();

That was easy !! Now let’s look at a few ways of pattern matching with Selenium for BrowserMob’s RBU scripts.

Selenium supports a few methods that help match text patterns. However, selenium locators don’t accept regular expressions. Only patterns or values accept them.

Globbing:

selenium.click("link=glob:*Gifts"); // Clicks on any link with text suffixed with 'Gifts'
selenium.verifyTextPresent("glob:*Gifts*");

Regular Expressions:[regexp, regexpi]

selenium.click("link=regexpi:^Over \\$[0-9]+$");  //matches links such as 'Over $75', 'Over $85' etc

Contains:

selenium.highlight("//div[contains(@class,'cnn_sectbin')]");  //highlights the first div with class attribute that contains 'cnn_sectbin'
selenium.highlight("css=div#cat_description:contains(\"to last\")");  //locating a div containing the text 'to last' using css selector

Starts-with:

selenium.click("//img[starts-with(@id,'cat_prod_image')]");  //clicks on the first image that has an id attribute that starts with 'cat_prod_image'
selenium.click("//div[starts-with(@id,'tab_dropdown')]/a[last()]");  //clicks on the last link within the div that has a class attribute starting with 'tab_dropdown'
selenium.click("//div[starts-with(@id,'tab_dropdown')]/a[position()=2]"); //clicks on the second link within the div that has a class attribute starting with 'tab_dropdown'
selenium.highlight("css=div[class^='samples']"); //highlights div with class that starts with 'samples'

Ends-with:

selenium.highlight("css=div[class$='fabrics']"); //highlights div with class that ends with 'fabrics'
selenium.click("//img[ends-with(@id,'cat_prod_image')]"); //clicks on the first image that has an id attribute that ends with 'cat_prod_image'

[Note: ends-with is supported only by Xpath 2.0. FF 3 might throw an error for this.]

Happy Testing !!

When setting up monitoring jobs, there are often predictable time periods in which you want to change the behavior of a script or prevent it from running at all, without having to manually stop/start the monitoring job each time. For instance, you might want to prevent errors and alert emails during routine maintenance windows, or perhaps you’re only interested in site performance on weekdays during regular business hours. We’ve come up with solutions to a few common situations that will help jump start your scripts.

Overview
At the core of all these examples is a small configuration object that sits at the top of the script for quick changes, and a single function that returns true or false depending on whether or not the current time is within a blackout period. The following is some pseudo-code outlining one way in which you might choose to implement these examples in your own scripts. Don’t forget to add in the appropriate maintenance object and isInMaintenance() function from the examples below to make this work.

// The configuration object for the maintenance window
var maintenance = {...}
 
// The maintenance function (plug in the appropriate function)
var isInMaintenance = function() {
    ...
}
 
var selenium = browserMob.openBrowser();
 
if(!browserMob.isValidation() && isInMaintenance()) {
    // Active maintenance window
    browserMob.log("Is in maintenance!");
} else {
    // Run normal script
    browserMob.beginTransaction();
 
    browserMob.beginStep("Step 1");
    selenium.open("http://google.com/search?q=website+monitoring");
    ...
    browserMob.endStep();
 
    browserMob.endTransaction();
}

Web Page Check
The most straight forward way to check for a maintenance window is to have Selenium load a web page and look for an element that’s only present during scheduled down time. You’ll want to be careful not to choose something that is also present when the site is really in trouble or you will never get your alerts!

var maintenance = {
    check: {
        url: "http://example.com/home",
        element: "//img[@title=\"maintenance\"]"
    }
}
 
var isInMaintenance = function() {
    if(maintenance.check && maintenance.check.url != null && maintenance.check.element != null) {
        selenium.open(maintenance.url);
        return selenium.isElementPresent(maintenance.element);
    }
 
    return false;
}

Multipe Date Ranges
One of the most common requests we get is for a way to handle scheduled maintenance windows. These windows tend to be one-off date ranges decided well in advance of the actual event. Given a set of start/end dates, you could prevent monitoring errors and email alerts during the blackout periods. When setting the dates, don’t forget to include the time zone abbreviation in the argument string or the script will use the local time of the machine it’s running on.

var maintenance = {
    dates: [
        {start: new Date("05/12/2010 23:00 PDT"), end: new Date("05/13/2010 02:30 PDT")},
        {start: new Date("05/24/2010 23:00 PDT"), end: new Date("05/25/2010 02:30 PDT")}
    ]
}
 
var isInMaintenance = function() {
	var now = new Date(),
	    nowTime = now.getTime(),
	    maintenanceWindows = [];
 
    // Check for maintenance date ranges
    if(maintenance.dates && maintenance.dates.length) {
        maintenanceWindows = maintenance.dates;
    }
 
    for(var i in maintenanceWindows) {
        var m = maintenanceWindows[i];
 
        if(nowTime >= m.start.getTime() && nowTime < m.end.getTime()) {
            return true;
        }
    }
 
    return false;
}

Days of the Week
Here’s an example for blacking out specific days of the week. This will allow you to create a script that, for example, monitors on Mondays, Wednesdays, and Fridays, or only on business days. Simply provide a time zone offset, which is the number of minutes your time zone deviates from coordinated universal time (a.k.a. UTC, GMT, Zulu time), and the days of the week you don’t want to run the script (from 0-6, where 0 is Sunday, 1 is Monday, etc.).

var maintenance = {
    timezoneOffset: 240, // UTC offset in minutes (eg: NYC (GMT-4) is 4 * 60
    days: [0, 6]    // Sunday = 0, Saturday = 6
}
 
var isInMaintenance = function(){
    var now = new Date(),
        nowTime = now.getTime(),
        timezoneOffset = maintenance.timezoneOffset || 0,
        localTimezoneOffset = now.getTimezoneOffset(),
        timezoneDiff = localTimezoneOffset - timezoneOffset;
 
    // Check for full blackout days (such as weekends)
    if(maintenance.days != null && maintenance.days.length) {
        var targetDate = new Date(nowTime + (timezoneDiff * 60 * 1000)),
            targetDay = targetDate.getDay();
 
        for(var i in maintenance.days) {
            if(maintenance.days[i] == targetDay) {
                return true;
            }
        }
    }
 
    return false;
}

Time of Day
This example shows how you can have a daily blackout range. This code will parse the start/end strings into a daily time range adjusted to the target time zone. Use this if you only want to monitor during banker’s hours, or if you want to prevent a script from running while your database is backing up.

var maintenance = {
    timezoneOffset: 240, // UTC offset in minutes (eg: NYC (GMT-4) is 4 * 60
    hours: {start: "23:30", end: "07:30"}
}
 
var isInMaintenance = function(startTime, endTime, offset) {
    var offset = maintenance.timezoneOffset || 0;
    var now = new Date();
    var nowTime = now.getTime();
    var maintenanceWindows = [];
 
    var toUtc = function(localTime) {
        var chunks = localTime.toString().split(":");
        var h = parseInt(chunks[0], 10);
        var m = parseInt(chunks[1], 10) || 0;
 
        var d = new Date();
        d.setHours(h);
        d.setMinutes(m + offset);
 
        return d.getHours() + ":" + d.getMinutes();
    }
 
    var sH, eH, sM, eM;
 
    startTime = toUtc(maintenance.hours.start);
    var chunks = startTime.toString().split(":");
    sH = parseInt(chunks[0], 10);
    sM = parseInt(chunks[1], 10) || 0;
 
    endTime = toUtc(maintenance.hours.end);
    chunks = endTime.toString().split(":");
    eH = parseInt(chunks[0], 10);
    eM = parseInt(chunks[1], 10) || 0;
 
    var startDate = new Date(now.getFullYear(), now.getMonth(), now.getDate(), sH, sM, 0, 0);
    var endDate = new Date(now.getFullYear(), now.getMonth(), now.getDate(), eH, eM, 0, 0);
 
    if(startDate.getTime() <= endDate.getTime()) {
        // Single range mid-day
        maintenanceWindows.push({start: startDate, end: endDate});
    } else {
        // Date wraps, which means two blackout ranges in a single day
        var morning = new Date(now.getFullYear(), now.getMonth(), now.getDate(), 0, 0, 0, 0),
            midnight = new Date(morning.getTime() + (24 * 60 * 60 * 1000));
 
        maintenanceWindows.push({
            start: morning,
            end: endDate
        });
 
        maintenanceWindows.push({
            start: startDate,
            end: midnight
        });
    }
 
    for(var i in maintenanceWindows) {
        var m = maintenanceWindows[i];
        if(nowTime >= m.start.getTime() && nowTime < m.end.getTime()) {
            return true;
        }
    }
 
    return false;
}

Now, a few of you might have noticed that the above examples are compatible with one another; you can mix-and-match them as you see fit. Just combine the various maintenance object fields into one config object, combine the contents of all the maintenance methods you want to use (just watch not to overwrite the variable declarations), and you can create something that fits just about any situation.