Today we received an excellent question from a customer of ours:

We were wondering if you all have any information that says “X Unique visitors per day translates into Y simultaneous users at any given time.”

Essentially, we’re looking for a way to determine how many simultaneous users we should load test with if we know the sites normal daily traffic.

While every site is different, we recommend following this line of reasoning to help you find the answer. Suppose your site gets 100K unique visitors per day, with peak traffic in the mornings and afternoons. Assume that 40% of the traffic comes between 7AM and 11AM, 40% at 4PM to 9PM, and 20% at other times. This means during your peak hours (7AM to 11AM and 4PM to 9PM) you’ll get ~10% of your unique visitors per hour, or 10K uniques in our example.

Now that you know how many unique visitors you’ll get in an hour, you can start turning that in to concurrent users. To do that, it’s important to understand that a unique visitor is roughly equivalent of a transaction. So really you want to figure out how many users you need to reach 10K transactions in an hour.

Suppose your script (or scripts) take an average of 2 minutes to complete. That means a single user will execute 30 transactions in an hour. So to reach 10K transactions, you’d need 334 users (10K divided by 30). If you decide you want to create realistic scripts that include human think time, then the scripts will take that much longer and you’ll need that many more concurrent users. For example, if the script gets 5X think time added and now takes 10 minutes to run, then you’ll need 1,667 users (10K divided by 6).

Of course, this calculation will only get you the load on a typical day (assuming a single hour sees 10% of traffic). Your traffic patterns may vary, or you may want to prepare for a larger surge. For example, if you want to test what happens when 60% of the daily traffic visits in an hour, then you’d need 2,000 users (60K divided by 30).

Tweet This Post 

No tags

We recently had a customer from a large clothing retailer ask us if there was any way to ensure that data, such as a username/password combination, could be restricted such that it was “checked out” and available only for a specific concurrent user. This is very common with logins, where systems often will prevent concurrent logins from multiple IP addresses.

While BrowserMob does not have a concept in which data rows can be “checked out”, some simple scripting can achieve the same results. The key is in creative use of the browserMob.getUserNum() and browserMob.getTxCount() APIs. You can learn more about them by reading up on the BrowserMob APIs.

The getUserNum function returns 0, 1, 2, etc based on the concurrent user in your load test. So if you have a 100 user test, getUserNum will return between 0 and 99. It’s important to understand that it will return the same value for the same user throughout the test.

The getTxCount function returns 1, 2, 3, etc based on the number of cycles for that specific user. This number will effectively be a counter of the unique number of transactions that that particular user has executed. So user 1 and user 100 will both have a getTxCount of 1 returned, but by the time user 100 sees it, user 1 might already be on transaction 50.

Now suppose you want to run a 1000 user test in which you never concurrently log in with the same user. All you need to do is pre-create 1000 user accounts and then write your script like so:

var userId = browserMob.getUserNum();
var username = "test-" + userId;
var password = "password";
selenium.type("username", username);
selenium.type("password", password);

This works great, but what if you want to use more than 1000 logins? Suppose you want to use up to 10,000 logins among the 1000 user test? This is where the getTxCount function comes in to:

var loginsPerUser = 10;
var userNum = browserMob.getUserNum();
var txCount = browserMob.getTxCount();
var userId = userNum * loginsPerUser + txCount % loginsPerUser;
var username = "test-" + userId;
var password = "password";
selenium.type("username", username);
selenium.type("password", password);

What this does is allocate 10 logins per concurrent user. So user 1 will get usernames test-0, test-1, …, test-9 while user 8 will get usernames test-80, test-81, …, test-89, etc. Because of the mod call (%) the ten usernames will simply wrap around once they’ve been used.

Tweet This Post 

API · Scripting

Because both our load testing and website monitoring services are based on Selenium, we have a unique ability to measure the performance of things like page load times, AJAX timings, and other in-browser interactions.

Selenium has both a setTimeout command and a waitForPageToLoad command. Both can be given a timeout value, which will control how long Selenium waits for a given page to load or element to appear. When it comes to using our services, most people stick with the default time of 30 seconds. If the timeout is reached, an error is thrown, the script aborts, and the transaction is recorded.

However, sometimes people want to know when pages take more than X seconds to load, but don’t want to necessarily interrupt the flow of the script. In fact, just last week we got this request from a customer:

Ability to trigger an alert based on a set threshold (by the user) – not using the timeout. This basically came from the performance issues we are experiencing. Here’s the scenario:

• We have a specific page that takes 2+ minutes to load.
• The page load timeout in the Selenium script was set to 60s.
• BMob properly reported the “timeout” error but when this error happens, BMob quits the script.
• This is not ideal for me since I want to still be able to see how long the page takes to load.
• Increasing the page load timeout for the page in question works, but now I don’t have a way (that I know of) to still trigger an alert after 60s.

I should be able to set a threshold for any page I choose that would then send a notification alert.

In other words, this customer wanted for a way to still report a transaction as a failure and receive an alert, but also still allow the script to continue. Fortunately, our support for JavaScript as a scripting language provides the answer:

var timeout = 90000;
...
// start of script
...
var start = new Date().getTime();
selenium.waitForPageToLoad(timeout);
var end = new Date().getTime();
...
// rest of script
...
if ((end - start) > 45000) {
    throw "An important page took longer than 45 seconds to load";
}

What this script does is sets the timeout to a very long amount (1.5 minutes) but will still report an error if a specific page takes longer than 45 seconds. This allows the remainder of the script to execute even when the page takes more than a specified 45-second threshold.

The only problem with this script is that if the page takes longer than 90 seconds, then the rest of the script will still not run because waitForPageToLoad will throw an exception. You can solve that too with a little code:

var start = new Date().getTime();
try {
    selenium.waitForPageToLoad(timeout);
} catch (e) {
    // this will happen after 90 seconds
    // todo: recover and send the browser to the the next URL
}
var end = new Date().getTime();

The only thing that is important to remember with this use of try/catch is that you’ll need to properly recover from the error. Simply catching the error and trying to continue may not work. For example, if the next Selenium command requires clicking on a button that should have loaded from the last page, there may be no way to recover. However, if the next step is simply visiting a new URL, you could possibly get away with a simple open() command in the catch block.

Tweet This Post 

Scripting · Selenium · Timeouts