This morning I will be presenting at with Jinesh Varia, an Amazon Web Services evangelist, at FutureTest in New York City. The conference is centered around the idea of exploring the future directions to take testing, QA, and software development. Jinesh Varia wrote up a nice summary of the slides we presented.

In our talk, we will demonstrate how cloud computing is primed to drastically change the way both automated and manual testing is done. Using service like Amazon EC2 and Amazon Mechanical Turk, it has never been easier to tap in to the power of the cloud. The key point of our presentation is that these new services allow for massive parallelization.

BrowserMob’s load testing service is an example of massive parallelization used for testing from the client’s perspective. We’re doing something no one has ever done before: spinning up thousands of web browsers and sending them against your website, allowing you to simulate the most realistic load test possible.

This concept just isn’t possible for most companies. Consider this: a 2,000 real browser user (RBU) test from BrowserMob consumes more than 2.5TB of memory and over 2,500 CPU cores. Assuming each CPU core and 1GB of memory costs $500, you’d have to spend $1.25M just for the hardware alone. It’s clearly not scalable to own that hardware, but our cost effective load test pricing shows that it is definitely affordable to rent it.

But as much as we love BrowserMob, the main point of our presentation today is that services like ours is just the tip of the iceberg. For example, Amazon Mechanical Turk, which is a human-powered intelligence engine, could allow you to enroll thousands of testers, paying them for each test case, to temporarily increase your manual QA effort from two QA engineers to hundreds or thousands of additional eyes.

In the case of Amazon EC2, suppose you had a nightly integration test, which runs Selenium scripts against your application. If you were to parallelize both the client side (ex: BrowserMob) and the server side (ex: Rails, J2EE, .NET, etc), you could execute each test case individually and independently from the rest. Assuming you had 1,000 tests, each taking one minute to run, you could reduce your integration test time from over 16 hours to minutes.

Whether it’s using parallel computing or parallel human resources, the cloud now offers the tools necessary to revolutionize testing and, in turn, software engineering in general. Look for new innovations in the coming months and weeks from all parts of the software stack – from vendors such as BrowserMob to open source projects such as Ruby on Rails. Do you have ideas for how the cloud can accelerate or improve the world of software testing and software development? If so, please leave a comment and join the discussion!

Tweet This Post 

No tags