Flash and Flex automation options using Selenium

We get a lot of customer requests about Flash automation, Selenium, and BrowserMob. Because our load testing and website monitoring services uses real browsers, complete with Flash 10, we can do things no one else can do, like run a load test with hundreds of Flash runtime environments driving traffic on your site.

However, while our infrastructure allows for extremely unique Flash testing support, it’s not perfectly streamlined (yet). While we are hard at work on making Flash support for Selenium and BrowserMob significantly better, at this time some work is required by Selenium users before you can get started.

Specifically, the first question that must be answered is: what automation technique do I want to employ? There are two distinct ways to go:

• Native mouse & keyboard integration – good for situations where there is minimal Flash interaction required
• API-level integration – good for situations where there is a lot of Flash interaction required

Native mouse & keyboard integration

Let’s talk briefly about native integration. This is where a mouse movement, mouse click, or keyboard action is simulated at the operating system level. It reproduces the most realistic user simulation. However, it can be very brittle, as it has limited ways to “read” from the screen and validate that functionality worked correctly.

Presently there is no great Selenium-only solution here. There are commercial functional testing products such as eggPlant, but nothing on the load side of things. BrowserMob does have, however, the ability to interact with the operating system using native key events and coordinate-based mouse interaction.

This feature is perfect for those who want to fire off one or two simple interactions, such as clicking on a movie’s “play” button or interacting with a confirmation dialog. If this is all you need for your Flash or Flex integration, please contact our support team and we’ll gladly help you get your script working using this unique API.

API-level integration

For applications that have much more complex Flash/Flex user interfaces or applications that are 100% Flash-based, we don’t recommend native events as they can be brittle and difficult to confirm that the desired functionality worked. Instead, we recommend interacting with your Flash applications using API-level integration.

Important: to do this type of automation you must be able to recompile/modify the underlying Flash object(s). If you cannot do this and require full “black box” style automation, we suggest you consider native automation or you consult with the authors of the original Flash component.

How you proceed from here depends on whether you are using Flash or Flex. The reason is that Flex exposes some automation APIs that you will otherwise have to reproduce if you are using Flash-only.

Flash automation with ExternalInterface

If you are using low-level Flash, your automation options are a little more limited:

• Using ExternalInterface directly – no 3rd parties, but might requires the most programming
• Using FlashSelenium – an open source library to make ExternalInterface callbacks easier to call

We recommend you first start off with ExternalInterface directly, as FlashSelenium is really just a thin wrapper around ExternalInterface. While it’s convenient, we think doing it by yourself once is an important learning experience.

To get started, we recommend reading the ExternalInterface documentation by Adobe. The basic idea is that within your Flash component you execute this code:

ExternalInterface.addCallback("doSomething", doSomething);

This exposes the function doSomething such that it can be executed by JavaScript. Fortunately, because Selenium can execute JavaScript using the getEval, storeEval, assertEval, etc commands, we’re in luck:

var jsExpr = "window.document.getElementById('myFlashObjId').doSomething()";
var something = selenium.getEval(jsExpr);
if (something == 'blah') {
   // throw an error, we weren't expecting 'blah'!
}

Using this technique, you can begin to expose ActionScript functions that can be used to automate the Flash component and validate that the right behavior took place.

Flex automation with Selenium-Flex

If you’re using Flex, automation should be quite a bit easier. Thanks to Selenium-Flex, you don’t have to spend time exposing dozens of functions via ExternalInterface. Instead, just compile your Flex application using the Selenium-Flex library:

1. Copy the sfapi.swc file in to your project (ie: the lib directory)
2. In your IDE/build system, ensure that the library is included (ie: -include-libraries)

That’s it! Now the next time you build your Flex app it’ll automatically have a whole bunch of functions already created. Even better, you don’t have to do all that getEval mumbo-jumbo we did for Flash testing. Instead, thanks to another open source project called FlexUISelenium (which works in concert with Selenium-Flex), the getEval stuff is abstracted away so that your tests look like this:

flex.type("2").at("arg1");
flex.type("3").at("arg2");
flex.click("submit");

If you’re doing functional programming using Java or some other full-blown programming language, you can get started with FlexUISelenium today. If you’re looking to load test or do website monitoring with Selenium, we are in the middle of adding FlexUISelenium support to the BrowserMob API. We are looking for beta participants right now, so contact us if you’re interested.

A note about recording support

Currently, none of the open source solutions referenced here support recording interactions from within Selenium IDE. While not Selenium-compatible (yet), FlexMonkey is a very good open source Flex record and playback functional testing framework. We have plans to integrate Selenium and FlexMonkey in the future, including record support through Selenium IDE.

In the meantime, you will need to write each Selenium command by hand. This means that you’ll need to be aware of the IDs of all the elements you wish to interact with. Once you know those IDs, you can write the code that issues clicks or types on the component.