Real-time Charts

BrowserMob provides you with four types of charts:

• One test plan chart for the whole test
• One throughput chart for the whole test
• One response time chart for each script
• One error rate chart for each script

A description of what each chart type provides can be found at the end of this document.

Every chart has a common set of capabilities, including the ability to drill into individual data points, the option to hide lines from displaying, and the ability to dynamically update in real-time as a test is running.

Drilling Down in to Data Points

Once a chart is enlarged, it becomes easier to see the Y axis and the individual data points in the chart. By hovering your mouse over a data point, a small window will appear telling you the exact numerical value for that X/Y coordinate. By clicking on that data point, you can initiate an ad-hoc SQL query that will pull up relevant data about that point.

For example, clicking on a point in the throughput chart will generate a SQL query for you that displays all the transactions for that particular script, during that specific minute of time, selecting out the total bytes transfered for each script. Those total bytes pulled out of the detailed data represent the underlying data point and in fact add up to the exact value of the point clicked.

To learn more about the SQL query interface, including the rules for when you can drill deeper down in to the data, click here.

Hiding Lines from Displaying

An enlarged chart looks like the following:

As you can see, for each line there is a color-coded checkbox on the right-hand side. By un-checking a box, the line will be removed from the chart. This is a very handy way to focus in on specific steps or scripts that you think may be the root cause of performance issues.

Real-Time Charting Capabilities

Every test contains basic test info at the top of the test results page. This info includes the name of the test, the start and end times of the test, and the current state of the test.

A test can be in the following states:

• SCHEDULED – the test is scheduled and will begin STAGING shortly before the scheduled time
• STAGING – the test is getting ready to run
• RUNNING – the test is currently running
• CLEANING_UP – the test just recently completed and is finishing collecting the data
• FINISHED – the test is completely finished
• FAILED – the test failed, likely because the user aborted it

When tests are in the RUNNING state, the charts on the page will automatically update their displays every 10 seconds. Because the charts all have an X axis of one minute, this means that you may see the most recent data point “move” up or down as the real-time results come in.

Because of this, we recommend that you don’t worry about an individual data point until the next minute has started and you are sure the data point isn’t going to change anymore.

Tip: sometimes real-time chart updates can get “out of sync” with the true underlying data. This happens usually due to a slow or unreliable internet connection from your browser. If you think this is happening, simply reload to resolve it.

Chart Type: Test Plan

The test plan chart simply shows two lines: the planned virtual users (browsers), as determined by the user who scheduled the test, and the actual number of active users on a per-minute basis.

It’s important to remember that BrowserMob always ensures that the exact number of browsers are alive and running transactions that were requested. However, because there is a small amount of time that must be spent either initiating or finalizing a transaction, not all browsers will be actively running a transaction at the same time.

This important point explains why you will often see that the two lines in this chart do not perfectly sync up. The other reason the lines wouldn’t sync up is if someone paused the browsers during the test. This would cause the active users to drop sharply to zero until the test was resumed.

Chart Type: Throughput

The throughput chart is the best way to know that you’ve likely reached the point in your load test where additional users begins to produce diminishing returns. By showing how many bytes were received by every browser in terms of the total test and by each script, you can quickly tell when the system began to “max out”.

For example, in this chart, you can see that the test plateaued at about 17:10, even as BrowserMob was still adding concurrent users. This means that up until this point, the response time was likely holding steady or at least growing at a rate slower than the added users.

However, after 17:10, this was no longer true and the system became so overloaded that it began to actually send back less data over time. This often happens when there are so many users that the CPU and RAM on the web servers and database servers end up spending more time trying to handle different requests that they spend less time actually sending data back.

Chart Type: Response Time

The response time chart is provided for every script in the test. The first line (always the top one) represents the average response time per minute of all transactions that were executing this script.

The additional lines represent the average response time of each step in the transaction. A step is identified as a new page in the browser, often due to either an “open” command, which opens a new page in the browser, or by a “clickAndWait” (or any other xxxAndWait) command, which tells the browser to click on an object and then wait for the page to load.

Often there is one step in the transaction that is slower than the others. This chart can help pinpoint which step it is.

It’s also interesting to note that this example chart comes from the same test where we found the throughput chart. Note how the response time for the transaction stayed flat until about 17:10 and then got progressively worse. This tells us that the two charts are very closely related: as response time decreases, less data is being send back to the browser over time.

Chart Type: Error Rate

The error rate chart is also provided for every script in the test. The first line (always the top one) shows the total transactions (successes or failures) that ran during each minute of the test for this script.

The other lines represent the total number of failures for each step in the transaction. Because tests often have one or two “problem” steps, both in terms of response time and errors, this chart can help identify which steps are causing problems.

In this particular chart we can see that one step was failing fairly regularly (roughly 25% of the time), but none of the other steps had any problems until very late in the test. Clicking on any data point in this chart is very valuable, as it shows the individual transactions or steps that failed during that minute. This is a great way to drill down and see exactly what failed, both in terms of an error message and an actual screenshot of the browser.