There is no hard or fast rule on how long your tests should be, since it all depends on what you’re trying to do. However, we can provide some general guidance as well as some tips to optimize your value with BrowserMob.
First, it helps to know a bit about how the application works, since that may very well determine how long you should run your test for.
For example, suppose that your application took a request at the end of a user transaction and put it in to an in-memory queue for processing later. This would mean that if your load test executes more transactions than the queue can process, memory will get consumed, eventually resulting in a failure. Therefore, it might be a good idea to run your test long enough to determine when that failure occurs.
On the other hand, if your application doesn’t have any characteristic that would require running a test for hours on end, a shorter test of 10-15 minutes of sustained load might be all you need. Talk with the developers and architects of the software to get a feeling for what type of testing is important.
You should also know early on what type of test you want to run. There are different types of tests that you might be interested in, such as a multi-day “burn in” test, a short “burst” test that only lasts a few minutes, or an hour long test designed to simulate real load during a peak hour in the day. Different types of tests have different lengths, so we encourage you to discuss this with your team.
Finally, it also helps to optimize your costs with BrowserMob. Currently, BrowserMob charges on a per-hour basis, rounding up to the nearest hour. This means that a 100 user test for 5 minutes costs the same as a 100 user test for 60 minutes. Therefore, we recommend that even if you need only a small 5 or 10 minute test, you schedule a longer one and take advantage of the “pause” feature.
By scheduling a longer test and/or configuring multiple phases at different load levels, you can save money. And if you need to stop the load activity for a few minutes while you configure services or reset the database, you can pause the active test first, do your work, and then resume the test activity.
If you have questions about how long your test should be or how many users you might need, please don’t hesitate to contact us.
Cool site, love the info.