FAQ: Which matters more: hits per second or concurrent users?

Like most aspects of load testing, there is no easy or simple answer to this question. The answer often depends on what your goals are, how the application logic works, and what parts of the application you’re trying to test.

But in general, it’s important to clearly define various terms, which will help communication amongst your team and help you get to a clear goal before starting testing.

Some common words or phrases that are worth clarifying are:

• hits – A hit is most commonly known as an HTTP request to your website. It could be for a web page, an image, a style sheet, some JavaScript, a video file, etc. All hits are not equal: some may just need a simple image to be served, while others might cause complex application logic and database access.
• visits – A visit is usually referred to as the end-to-end interaction between a single user and your website.
• sessions – Sessions are similar to visits, but often have a more server-centric meaning. Most modern web application environments today have a concept of a server-side session, which is responsible for storing things like what is in the user’s shopping cart. Sessions often stay active for some time after the user’s visit is over, meaning additional memory is consumed until that session expires.
• virtual users – See this FAQ entry for more information on virtual users.
• concurrent users – Concurrent users can take on multiple meaning, but it most often represents how many virtual users are actively simulating a visit to your site.
• page views – A request to a web page by a browser. A page view always involves one hit (for the page itself) plus dozens of subsequent hits for images, JavaScript, CSS, etc that the page depends on.
• requests – An overloaded term that often means many things, such as a hit or a page view. We suggest using the term “hits” for raw HTTP requests and “requests” as a synonym for “page views”, or just avoiding the term “requests” entirely to avoid confusion.
• transactions – A single execution of the recorded script that the virtual user is supposed to run. A single virtual user will run as many transactions as it can (as long as the VU is active).

With these definitions out of the way, now you can ask yourself: what am I most concerned about testing?

If your application is mostly stateless, meaning it likely doesn’t consume much or any session state in memory, you are probably more interested in discovering what your peak hits/second, page views/second, or transactions/minute is.

If your application is not stateless or otherwise has significant background processing and load that is due to new sessions, you’re probably more interested in concurrent users.

Often you care about both, so thinking about these different terms and setting expected goals for all of them is important. Find out how many hits you expect for a particular visit or transaction. Determine how many total sessions you expect in a peak time period, but also how many concurrent users you expect at the same time. Then rank the metrics from most important to least important in terms of which you’d like to simulate closely.

With this information, you’ll be better prepared to schedule the appropriate load tests. Or, if you’re still unsure, feel free to share the information you have already gathered with our support team. We’d be happy to help advise you on how to proceed or get you in touch with one of our partners who can provide dedicated, hands-on support.