Tips for Testing with Load Balancers

For any moderate-to-large size website, a load balancer is almost always used to spread web traffic across multiple web servers. When it comes to load testing an environment with load balancers, it’s important to understand how your network infrastructure and software behavior is configured to handle the incoming load. It’s also equally important to understand how your load testing tool generates it’s load. Only by knowing both of these things is it possible to run an accurate load test that yields useful results.

Types of Load Balancers

Load balancers often have three types of logic for distributing incoming HTTP requests. They are:

• Round-robin: incoming traffic is spread evenly across all the web servers, resulting in a single user’s browser session to likely hit some or all of the web servers. If a server goes down, incoming traffic is simply routed to the the other servers.
• Sticky sessions (cookie-based): incoming traffic is “stuck” to a single web server based on a cookie value that is added by either the web server or the load balancer. If a server goes down, the sessions that were tied to it are distributed to other servers.
• Sticky sessions (IP-based): incoming traffic is “stuck” to a single web server based on the IP address of the requesting client. If a server goes down, the sessions that were tried to it are distributed to other servers.

Depending on the load balancer logic used and how stateful your web application is, you may also need to configure your web server (IIS, Apache, Tomcat, etc) to share backend session state among the servers using a clustered memory technology. There are also other types of load balancing logic (ie: Cisco’s Aeropoint technology), but they are usually derivatives of one of the three outlined here.

Common Mistakes with Load Testing

One of the most common mistakes made during a load test is not understanding the mechanics in which traffic is generated from your load testing tool. Let’s use the following deployment architecture as an example:

In this example, 300 virtual users are being generated from one physical machine/IP address. Because the load balancer is configured to use sticky IP addresses, all the load is being sent to web server A, leaving web servers B and C entirely unused.

This scenario is very common during load tests. In fact, it’s one of the first things to check when your load testing tool reports increased response times and/or error rates even when manually browsing the site shows no such degradation (also known as the “it works for me” observation).

The reason it works well for individual browsers but is slow for the load testing tool is that most load balancers are smart to route new IP addresses to web servers B and C. They do this because they are seeing that web server A is being inundated with requests and is much busier.

Spreading the Traffic Around

There are multiple ways to address this problem. You could change the load balancer logic, bypass the load balancer and directly test just one web server, or temporarily remove enough web servers from the cluster to ensure that the ones remaining receive an even distribution of load.

Or you could simply spread the traffic being generated from your load test across more physical machines. Consider the following diagram:

In this example, 300 virtual users are now being generated from 15 different IP addresses. Now the traffic is spread evenly among the three web servers. By configuring your load test to spread the traffic among multiple sources, you not only solve the “it works for me” problem, but you also make your load test that much more realistic to the real world scenario of one IP address = one real user.

This is why it’s important to know how your traffic is being generated. If you’re using a self-hosted tool (Apache JMeter, OpenSTA, Mercury Load Runner, Borland SilkPerformer, etc), try acquire enough machines to spread the load appropriately. If you’re using a hosted service (Keynote, AlertSite, WebMetrics, etc), ask them about the number of IP addresses and machines being used in your load test. Doing this will help prepare you for any surprises when it comes time to test.

Getting Even More Distribution

BrowserMob is also a hosted load testing service, so it’s important to know what our machine distribution is as well. We offer two types of load tests:

• Ultra low-cost load tests using HTTP browser simulation.
• Low-cost load tests using real web browsers to drive traffic.

In both cases, our service uses a very low IP-to-VU ratio – much better than the rest of the industry. In the case of our simulated browsers, we use no more than 50 VUs per IP address. This means that if you run a 300 VU test you’ll be getting traffic from six different IP addresses (just as you saw in the last diagram). For our real web browsers, we do even better and use no more than eight RBUs per IP address. That is, for a 300 RBU test you would see load coming from at least 35 different IP addresses!

It may not be practical or cost-effective to simulate an exact 1:1 ratio of IPs to RBU/VUs, as IP addresses are a limited quantity and can be costly to acquire in large volumes for this type of load testing. However, the lower you can keep that ratio the more realistic your test will likely be.