Scheduling a Test

Note: Before you can schedule a test, you must have one or more working Selenium scripts already recorded and verified. Be absolutely sure that you’ve verified the script before scheduling your test. If you haven’t, you’ll still be charged for the test even if it doesn’t behave as you expected.

Scheduling a test is a three step process:

1. Provide basic information, such as name of the test and desired start time.
2. Define the load profile in terms of virtual users and time.
3. Upload Selenium scripts and allocate frequency.

Step 1: Basic Information

Here you fill out the name of the load test, the desired start time, and the geographic location you wish the load to come from. The name of the test can be anything you want and is simply used to help you keep track of your different tests. We recommend something descriptive, possibly including site being tested, the number of virtual users, and the date/time of the test.

The desired start date and time must be provided in 24 hour time (0-23 hours) and is in your local timezone, which is identified on the page. As you type a value in, BrowserMob will inform you when the test is schedule to run in relative terms. This is especially useful if you’re planning to run a test in the next few hours and want to be sure you’ve got the right time.

The geographic location is the physical loation in which the browsers or virtual users will run from. Select a location that most closely resembles where your users will come from. However, keep in mind that no load test will ever truly mimic your user population perfectly, so just select the best choice. Remember that with load testing, you often are more concerned with relative performance (ie: site performance at 10 RBUs vs 100 RBUs) than absolute performance numbers.

Tip: BrowserMob requires that you schedule tests at least 15 minutes in advance. If you enter a time that is 15 minutes from now, there is a good chance you won’t fill out the rest of the form in time to make the cutoff. To avoid an error message, we recommend scheduling the test a couple extra minutes out, giving you time to finish this form.

Step 2: Define Load Test

This section is where you define the actual load profile you wish to schedule. You can make your test as long or as short as you want, though each step must be between 5 and 60 minutes. This means if you want a three hour test, you need at least three steps.

You can add new steps by clicking on the “Add” button. Clicking on the “Delete” button will remove the last step in the test.

For each step in the test, you must provide a duration, maximum users, and step type. The duration is the length of that step in minutes. The maximum users is the target concurrent browsers that you want executing your script(s) at the end of that step. Depending on the step type, the number of concurrent browsers during the test will either stay flat (type: CONSTANT) or ramp up/down to the defined number of users (type: RAMP).

To help you visualize the concurrency and load test definition, BrowserMob provides a preview of the concurrent users (Y axis) over time in minutes (X axis). This chart changes based on the values you enter, so try out different values until you get the load profile you want to run.

Tip: Remember that BrowserMob charges for tests by maximum virtual users per hour, rounded up to the nearest hour. This means that a 5 minute test of 100 users costs the same as a 60 minute test of 100 users. So while you are free to schedule shorter tests, you can maximize your value by scheduling hour-long tests and then pausing or stopping the test to control the test while it’s running.

Step 3: Select Scripts

This section is where you select existing test scripts you have previously created and validated inside of BrowserMob. You are also asked to select a script and an allocation percentage. Each script name in the selection box will be identified with either an “RBU” or “VU” notation. This tells you what type of script you are selecting.

Please note that a single load test must consist of 100% RBUs or 100% VUs and that mixing them is currently not supported. If you wish to simultaneously run RBUs and VUs, we recommend running multiple load tests that are synchronized to run at the same time.

The allocation percentage tells BrowserMob how often you want that script to run. Choose an allocation that realistically simulates the behaviors of your real users. For example, if you know that 90% of your traffic browses the site, 5% make purchases, and 5% do searches, then enter those values for the three transactions you’ve recorded.

Tip: Unsure what the allocations should be for your scripts? If this is an existing site, the web server traffic logs may be able to help. If your site is running any analytics software, such as Google Analytics, you may also be able to determine it there. If this is a new site, work with other team members, such as product managers and the marketing team, to determine realistic estimates.

Scheduling the Test

Once you’ve completed all three steps, you can try to schedule it by clicking the “Create” button. If there are any problems with the test, such as a corrupt script, invalid start date/time, or incorrect load profile, you’ll be asked to correct the error.

If there are no errors, you’ll may be asked to pay for the load test. If you have existing credits in your account, the test will be scheduled immediately. If you do not, you can pay for the test right now or come back and pay for it later. If you simply want to know how much the test would cost, simply click “Create” and you’ll be told what the cost is. You can then delete the test or pay for it later.

However, if you haven’t paid for the test at 15 minutes before the test is scheduled to start, it will immediately fail due to lack of funds. You can learn more about pricing, credits, and the payment process here.

Scheduling Tips

Load testing with BrowserMob is as simple as it can be, but there are still some things to keep in mind when planning out your test. We strongly encourage you to browse this blog for tips and tricks and other frequently asked questions.

However, there are a few tips you should be aware of right away when scheduling a test that we’d like you to be aware of:

• Tests are charged on a per-hour basis. Plan for longer tests and then take advantage of the pause and stop features of BrowserMob. This gives you more “bang for your buck”.
• Run a smaller load test before running a larger one. Even if you’ve validated your scripts by running them in Selenium IDE, there are still chances it might not work correctly the first time in BrowserMob. For only a few dollars, you can run a handful of virtual users to confirm that it works. Only after you’ve done that should you schedule the larger test.
• Add verification commands to your scripts. Advanced scripting techniques are covered in our blog, but verification commands are something unique to BrowserMob. They can validate very specific parts of the pages in your scripts, such as whether a table contains the expected data or an AJAX call completed successfully.

Tweet This Post 

No comments yet.