Traditional load testing has always involved simulating user traffic by replaying a series of HTTP requests. This traffic is designed to appear just like a the traffic a real user would cause and is considered a virtual user.

However, achieving realistic traffic can sometimes be very tricky, especially with new technologies like AJAX and things like .NET’s view state. The upside of this approach is that it requires a fraction of the hardware that would be required to run real browsers. For this reason, load testing tools have opted to simulate HTTP traffic for years.

Benefits of Real Browser Users over Virtual Users

That all changed when BrowserMob launched. With our service, you could leverage the power of the cloud to emulate real user behavior with real web browsers. Although this approach does indeed require much more hardware, we are able to make it lower cost than traditional load testing through the cost savings of cloud computing and open source.

Real browsers have many benefits:

• They generate load based on how the site is at runtime, rather than how the site was when the script was created.
• They automatically handle AJAX, .NET view state, JSF, and other technologies that are difficult to script in traditional load testing.
• They provide real screenshots of what a user would see in the event of an error.
• They are very easy to script, since you only have to worry about user interaction with the browser and not proper HTTP traffic simulation.

For as little as $1 per concurrent browser per hour, you could use real web browsers in your load test. But as great as real browsers are, sometimes they are overkill and there are lower-cost alternatives.

Using the Right Tool for the Job

But what about the times when you want to test something simple, such stressing your web server with a bunch of HTTP requests for the home page? Or suppose 90% of your users navigate the site but never fill out any forms or interact with complex AJAX components? Do you really need to use the overhead of a full browser in these cases?

The answer is: of course not. That’s why we have introduced a secondary service that is even lower cost. If you can script the HTTP traffic ahead of time, you can save 90% on your load test using our virtual user technology. BrowserMob even provides tools to allow you to convert a real browser user (RBU) script to a virtual user (VU) script, making it easy to pick the right tool for the job.

A common approach our customers use is to schedule two load tests at the same time: one smaller one using real web browsers and one larger one using virtual users. For example, suppose you wanted to test an e-Commerce website. While 10% of your traffic involves users logging in, adding items to shopping carts, and checking out purchases, 90% of your traffic might be doing simple searches and browsing your online catalog.

By using a mixture of approaches you can save money, reach the traffic levels you need to test, and still get the high fidelity/real browser playback for the use cases for the revenue-generating use cases.

Comparing the Differences

When deciding whether to use real browsers or virtual users, consider the following differences in features and pricing:

Consider the different requirements for your test by asking yourself the following questions:

• What kind of bandwidth do you hope to generate? How many IP addresses do you want to use for your test?
• Can you verify if a transaction is successful by simply looking for a text match in an HTTP request, or do you require more complex validation, such as checking whether a particular element on the page is visible?
• Is it enough to know the HTTP response code when an error happens, or do you wish to see a screenshot of the error?
• Can you sacrifice a bit of fidelity in order to reduce the price of the test?

When in doubt, we recommend using real browsers initially and contacting our support team for some additional advice.

Tweet This Post 

No tags