BrowserMob founder and current Sr. Director of Product Management at Neustar, Patrick Lightbody, recently spoke at the Velocity Conference in Santa Clara, CA, where Neustar was a sponsor. Patrick shared his personal cloud computing experiences, highlighting techniques he used and lessons he learned when starting up BrowserMob.  Focusing on operational excellence, scaling, performance, and finance, he provided 4 lessons learned:

Lesson 1: Building Business Plan: The cloud provided Patrick with a safety net to launch the BrowserMob business. He tried to outsource as much as possible, and turned to the cloud for his business because it had an easy financial model, metered usage model and no upfront investment.

Lesson 2: Optimizing for the cloud: Patrick learned quickly that the cloud is not infinitely scalable. He worked with the infrastructure of the cloud to optimize to his business benefit. He notes that you can’t treat the cloud like a black box; you need to understand what you are working on technically, understand the architecture, and also work with the people that are at your cloud provider.

Lesson 3: Processing lots of data: The company that Patrick was building would need to process lots of data. He says it was definitely a hit and miss process as he discovered how exactly the cloud would work when processing data. He also learned that one size definitely does not fit all when it comes to data storage.

Lesson 4 – Sleeping well at night: According to Patrick this is the most critical lesson learned. The way to ensure that you aren’t attached to your computer at all hours is to make sure you are monitoring everything. Patrick said he set up voice alerts that would wake him, and other team members, if anything went down.

Listen to the full session to get more tips and details on Patrick’s lessons learned.  The Velocity Conference has posted the full presentation online or you can view it at the following URL: http://bit.ly/jhnuDx .

As you all know, BrowserMob utilizes Amazon Web Services (AWS) and open source software to power our services – in fact, BrowserMob couldn’t have been started without it.  We have put together a case study about our work with Amazon, highlighting the ways that it is helping us save you money.

Our approach to load testing depends on launching thousands of instances in short order, and AWS fits that model perfectly.

Leveraging Amazon EC2 On-Demand and Spot Instances, we start by launching a baseline of On-Demand instances to ensure the capacity will run for the entire duration of the test – always looking ahead 5 minutes to determine capacity needs. We’ll attempt to launch a Spot Instance first at a lower bid price, but if the instance doesn’t launch within the first 5-7 minutes, BrowserMob will attempt to run On-Demand.

Spot Instances provide two advantages: First, we get a lower price when the capacity is available, and second, we get extra capacity when the On-Demand market is saturated.

With the combination of Amazon EC2 On-Demand Instances and Spot Instances, BrowserMob is able to run approximately 1000 Amazon EC2 instances for less than one hour intervals twenty to thirty times every day. And of course, we are always able to further expand our Amazon EC2 usage when necessary to meet the needs of its customers.

For example, we actually ran 5,000 Firefox browsers on 2,300 CPU cores within Amazon EC2 in order to load test an online gaming Website that was promoting the launch of a major movie.

We’re also exploring options to create additional cost savings through Amazon EC2 Spot Instances by allowing customers the flexibility to determine their own service needs in relation to the current Spot Price.

Let us know if you have any specific questions about how we work with Amazon. The full case study can be found on the AWS website here: http://aws.amazon.com/solutions/case-studies/browsermob/?ref_=pe_8050_19833900

Hot off the press … Selenium 2.0b3 has just been released simultaneously for Java, .Net, Ruby and Python. This release focused on providing support for the next generation of browsers – specifically, IE 9 and Firefox 4.

For more information, check out the blog posting from the Official Selenium Blog.

BrowserMob uses almost every AWS service out there. I plan to start a blog series outlining how we use them all, but here’s the quick overview:

• EC2 – We routinely launch thousands of instances every day to generate websites load tests. This is our primary use in the cloud and BrowserMob could not exist with EC2.
• S3 – We store every individual bit of detail about every test we run in standard formats such as MySQL database dumps or HAR files.
• SQS – Every website monitoring check is managed through Amazon’s excellent queue technology.
• SimpleDB and RDS – We store a variety of data using Amazon’s “NoSQL”, schemaless database system (SimpleDB) as well as traditional MySQL databases managed by Amazon.
• FPS – We handle all our credit card processing using Amazon’s Flexible Payment System.
• SNS – We were a beta tester of SNS and are using it to coordinate the thousands of EC2 machines that we spin up in the cloud.
• CloudWatch – We monitoring some of our important EC2 instances using Amazon’s simple monitoring service.
• CloudFront – To make our website faster, we store all our images, CSS, JavaScript, etc across Amazon’s 16 servers, which are closer to our users than our web server is.

Today I want to briefly talk about CloudFront, since Amazon announced that it has gone from beta to GA, which includes a 99.9% availability SLA. It also now supports reading origin files from locations other than S3, which makes it even easier to quickly use it as a super low cost CDN alternative.

In fact, one of our users recently declared: Amazon CloudFront Rocks. Check out Renat’s post, which we’re pleased also had some nice things to say about BrowserMob too

We’re a few weeks late to the story, but we think it’s still important to call attention to: Watir and Selenium are joining forces. For those unfamiliar with Watir or Selenium, both of them are open source browser automation frameworks. For years they have been “competing” in the open source community, with Watir winning favor among the Ruby community and those who needed strong IE support, while Selenium won favor among the Java and C# crowds and those who really valued cross-browser support.

While both projects had their pluses and minuses, it’s great to see them finally working together. On behalf of BrowserMob (which builds on top of Selenium) and as a Selenium contributor, it pleases me to no end to know that the Selenium community will now gain two huge contributions: a first-class Ruby API and all the benefits of Watir’s fantastic IE support.

Note: this is not a merger. Watir will continue to run as an independent project and it will even still work with it’s own browser backend. The important thing here is that Watir will now offer an option to wrap around Selenium WebDriver, the core cross-browser automation library that all the other Selenium projects also wrap around. This means that while the projects will remain independent there will likely be a lot more cooperation moving forward.

Today we’re extremely excited to announce another industry-first in the world of load testing: video capture.

From day one, we always believed that cloud computing ushered in an era where the majority of website load tests could and should be done using real browsers, something no one else had considered before. Doing so made scripting painless, load tests more realistic, and provided better error reporting through the use of screenshots.

Today we’ve raised the bar by also capturing video for every unique failure in a load test. This means that you can now see exactly what led up to the failure and even get a feel for what the user experience was like as your site began to break down.

In addition we also now capture the source HTML (ie: DOM structure) of the web page in the event of a failure, as well as the entire set of request and response headers for the failed transaction. This gives unparalleled information that is simply not available in the traditional load testing approach that relies on headless/virtual users.

We have a lot more great features in store for both our website load testing and website monitoring products, but we’re especially excited about this one. It’s available today so please check it our for yourself and let us know what you think.

One of our customers, SignalFive, recently published a case study about a project they did with Al Gore’s Alliance for Climate Protection.

The project, called the Repower America Wall, involved a massive, Flash-based interface in which thousands of user-created images and videos could be scanned through at high speed.

In the case study, they describe how they calculated the estimated site traffic and decided to use cloud services, such as Amazon and YouTube, to help the site scale. However, before launch they still needed to prove to the customer that the site was ready:

While this stress-testing was valuable, and optimizations were made based on them, none of these techniques truly simulated real-world use. Typically, a testing environment such as Selenium IDE might be used. But since the UI of the wall was built in Flash, things got tricky.

Because of the rich Flash interface, traditional load testing services based on simple HTTP protocol simulation weren’t well-suited. For that reason, they came to BrowserMob. Using our real browser technology, we were able to load the entire UI, allow the background AJAX and AMF calls to take place, and even stream the various videos:

BrowserMob’s team helped us craft some tests which we ran on the dev and production servers. The information was invaluable — based on the test results, we decided to upgrade the production server, and doubled its memory.

Below is some of the load test charts SignalFive generated using our software:

We’re happy to report that Selenium IDE 1.0.4 has been released. You can download it here and you can find the release notes here.

While this release doesn’t have many new user-facing features, it does clean up several bugs. More importantly, however, is that Adam Goucher and Jérémy Hérault did some amazing work to lay the foundation for a plugin framework. This means that soon you’ll see Selenium IDE plugins that further expand the Selenium IDE capability.

Jérémy is working on one such plugin, called Helenium (see proof of concept in action), that will allow you to do text matching against images and PDF files using optical character recognition (OCR). We’re also working on a plugin that will make it easier to upload scripts from the IDE to BrowserMob.

If you’re interested in how to build your own plugin, I recommend reading Adam’s blog, which has a series of recent posts on how the plugin framework works.

Late last night Amazon released an early Christmas present: a third location for which to run Amazon Web Services. This is important news for BrowserMob customers for two reasons:

• Monitoring – it will be our fifth location from where checks will run from.
• Load testing – it will be our third location from where traffic can be generated from.

We are committed to moving at the “speed of the cloud” and as such expect to have both of these options available in the next week. We’re excited about Amazon’s commitment to continue to provide geographic diversity to their cloud offering, and we look forward to their expansion to Asia early next year.

I’m a little late to the party (dynaTrace released their product a couple weeks ago), but I wanted to still highlight their very important tool: dynaTrace AJAX Edition. It’s by far the best browser profiling tool out there – and it’s free!

If you’re familiar with Firebug, then that’ll give you a rough idea of what this product does. However, instead of being a “Swiss Army knife” like Firebug is, dynaTrace did a deep dive on JavaScript profiling.

This means that this tool can help identify hotspots and bottlenecks in your client-side JavaScript. And as applications continue to get more complex, this will continue to be a bigger and bigger area that developers need to keep an eye on. Ultimately, it doesn’t matter if your server is fast if the client application is getting bogged down due to inefficient JavaScript and DOM calls.

Rather than launching in to a full overview of what dynaTrace AJAX Edition does, I recommend the following three articles:

• A Step-by-Step Guide to dynaTrace Ajax Edition – by Andreas Grabner, technology strategist @ dynaTrace
• dynaTrace Ajax Edition: tracing JS performance – by Steve Souders, chief performance engineer @ Google
• Deep Tracing of Internet Explorer – by John Resig, creator of jQuery

If you’re working on performance optimizations for your site, especially if you are concerned about IE (and let’s be honest, who isn’t?), do yourself a favor and add this tool to your toolset. You won’t be disappointed.

Note: I should add that BrowserMob is a dynaTrace partner and we are continuing to explore ways in which BrowserMob, Selenium, and dynaTrace products can work in concert.