A common question we get here at BrowserMob and elsewhere in the open source and QA communities we’re involved in is: “what’s the difference between stress testing, load testing, and performance testing?”
The three types of tests are often used interchangeably, though they do actually have somewhat different meanings. Fortunately, a friend of ours, Grig Gheorghiu, wrote about this very topic a few years ago in two blog posts:
Selenium is a popular open source, cross browser functional testing tool. You can learn more about it here. If you’re looking for functional testing solutions that use real web browsers, we strongly encourage you to take a look at Selenium.
We have a close relationship with Selenium: BrowserMob’s founder, Patrick Lightbody, was the original creator of the most popular version of Selenium, Selenium Remote Control. So it’s not surprising that BrowserMob utilizes Selenium technologies for both the record and playback of web traffic.
Because Selenium is a functional testing tool, it isn’t really meant to be used as a load product. There are two reasons for that:
There is no hard or fast rule on how long your tests should be, since it all depends on what you’re trying to do. However, we can provide some general guidance as well as some tips to optimize your value with BrowserMob.
First, it helps to know a bit about how the application works, since that may very well determine how long you should run your test for.
There is no clear cut answer to this, but we’re happy to provide guidance.
Your business liaison (usually a product manager, project manager, or analyst) may know the answer right away, since there may be specific business goals associated with the project.
The system architect may also have a good idea of what the capacity of the overall system is, so that might be another place to start asking.
You should also read our FAQ on hits/second vs. concurrent users, as that may help you better think about how many virtual users you need in a load test.
Like most aspects of load testing, there is no easy or simple answer to this question. The answer often depends on what your goals are, how the application logic works, and what parts of the application you’re trying to test.
But in general, it’s important to clearly define various terms, which will help communication amongst your team and help you get to a clear goal before starting testing.
Some common words or phrases that are worth clarifying are:
BrowserMob can support hundreds and thousands of concurrent browsers hitting your website. Because we use real browsers (vs simulating traffic only) this consumes far more resources than any other load testing solution out there. As such our top-end capacity is not necessarily as high as other offerings, though we feel that we can provide enough load for most needs.
See our pricing page for details of our current capacity. If you’re not sure if we can support the capacity you need, try scheduling a test on your own. If we can support it without any human interaction (which is often the case), the test will be scheduled without any warning.
People sometimes wonder how we can charge so much less, especially considering that we’re running real browsers for every virtual user, which costs significantly more than any other load testing technique.
The answer is that we have built an innovative technology that utilizes open source and cloud computing, which allows us to keep our costs very low, despite our higher overhead per virtual user.
What this means in terms of physical location of our browsers is that it’s determined by our cloud computing provider, Amazon. Those locations are:
BrowserMob is an external load testing service, meaning traffic originates from outside your firewall. We believe this is the most realistic way to test your web applications, as it properly shows how your site will be perceived by actual end users using real web browsers from external locations, such as their homes or offices. The topic of whether you should test on a production or production-like staging environment is discussed here, so please read that as well.
There are three common approaches if your application is only available internally:
Last Updated: November 18, 2011
Some of our customers want to know the IP addresses of our browsers so that they can allow them inside their firewall for the purpose of testing internally-hosted applications. Because we can support up to thousands of browsers, we can’t possibly list every individual IP address. However, we can share with you the range of IP addresses.
www.browsermob.com
For our Washington DC location, they are:
The concept of a “virtual user” is commonly used in load testing, but the truth is that the definition means different things to different people. Here we will explain what it means to BrowserMob and how you can use them in different ways to test capacity, load, or stress of your application.
In general, a virtual user is exactly that: a virtualize representation of a user, specifically designed to simulate the same behaviors and interactions with your website that a real user would. For example, if you knew that at peak you had 100 real users on your site last year, you could simulate that same behavior with 100 virtual users running one or more scripts that invoke similar navigation and interaction that your real users do.
Recent Comments