This morning I will be presenting at with Jinesh Varia, an Amazon Web Services evangelist, at FutureTest in New York City. The conference is centered around the idea of exploring the future directions to take testing, QA, and software development. Jinesh Varia wrote up a nice summary of the slides we presented.

In our talk, we will demonstrate how cloud computing is primed to drastically change the way both automated and manual testing is done. Using service like Amazon EC2 and Amazon Mechanical Turk, it has never been easier to tap in to the power of the cloud. The key point of our presentation is that these new services allow for massive parallelization.

When running a load test, there is usually a ton of data that gets generated. You get charts for response times and error rates. You usually also end up with large amounts of web server logs, which may have interesting data embedded in them. If you’re profiling your servers for their CPU, memory, and disk utilization, you’ll likely have additional charts to look at. All of these are necessary to piece together a complete picture of the results of your load test.

The Throughput Chart

Contegix, an amazing internet hosting provider, recently ran some tests to determine the performance differences between various configurations of Apache and Tomcat. In the process of doing their testing, they used Pylot, an open source Python-based load testing tool, and BrowserMob. You can read their entire findings here.

What’s interesting is the use of both internal (Pylot) and external load testing (BrowserMob). This is something we strongly encourage our customers to do. In talking with Mark Rogers at Contegix, it was clear he saw a big difference between the type of traffic generated by Pylot and BrowserMob.

A couple weeks ago we were approached by Tim Davis from MGX Lab to help with a load test for one of their clients who was going to be running an ad in the Super Bowl. They were working closely with their ISP and had been doing some internal testing up to that point, but they were looking for a much larger external test that could validate that the client website could withstand the huge volume of traffic sent it’s way during the big game.

That’s where BrowserMob’s web load testing system comes in. Here’s what Tim had to say about the experience:

Traditional load testing has always involved simulating user traffic by replaying a series of HTTP requests. This traffic is designed to appear just like a the traffic a real user would cause and is considered a virtual user.

However, achieving realistic traffic can sometimes be very tricky, especially with new technologies like AJAX and things like .NET’s view state. The upside of this approach is that it requires a fraction of the hardware that would be required to run real browsers. For this reason, load testing tools have opted to simulate HTTP traffic for years.

As of today, BrowserMob now supports both concurrent browsers and simulated virtual users. The term virtual user was adopted by early load test providers as a way to describe the technique of simulating the HTTP traffic that a real user would cause when visiting a website. While BrowserMob’s core technology sidesteps the concept of virtual users and actually drives real browsers, we felt it was important to also provide a second option for our customers.

For any moderate-to-large size website, a load balancer is almost always used to spread web traffic across multiple web servers. When it comes to load testing an environment with load balancers, it’s important to understand how your network infrastructure and software behavior is configured to handle the incoming load. It’s also equally important to understand how your load testing tool generates it’s load. Only by knowing both of these things is it possible to run an accurate load test that yields useful results.

Types of Load Balancers

Load balancers often have three types of logic for distributing incoming HTTP requests. They are:

© 2012 The BrowserMob Blog Suffusion theme by Sayontan Sinha