Wait for it...a deep dive into Espresso's Idling Resources

(A first draft of this blog post originally appeared on my personal blog: Wait for It…a Deep Dive in Espresso's Idling Resources)

One of the challenges developers have to face when writing UI tests is waiting for asynchronous computations or I/O operations to be completed. In this post I'll describe how I solved that problem using the Espresso testing framework and a few gotchas I learned. I assume you're already familiar with Espresso, so I won't describe the philosophy behind it but instead I'll just focus on how to solve that problem the Espresso way.

Idling Resource

Espresso introduces the concept of IdlingResource, which is a simple interface that:

Represents a resource of an application under test which can cause asynchronous background work to happen during test execution

The interface defines three methods:

  • getName(): must return a non-null string that identifies an idling resource.
  • isIdleNow(): returns the current idle state of the idling resource. If it returns true, the onTransitionToIdle() method on the registered ResourceCallback must have been previously called.
  • registerIdleTransitionCallback(IdlingResource.ResourceCallback callback): normally this method is used to store a reference to the callback to notify it of a change in the idle state.

For example, an implementation for an idling resource that waits for a page to be fully loaded in a WebView will look something like this:


public class WebViewIdlingResource extends WebChromeClient implements IdlingResource {

    private static final int FINISHED = 100;

    private WebView webView;
    private ResourceCallback callback;

    private WebViewIdlingResource(WebView webView) {
        this.webView = checkNotNull(webView,
                String.format("Trying to instantiate a \'%s\' with a null WebView", getName())));
        // Shall we save the original client? Atm it's not used though.
        this.webView.setWebChromeClient(this);
    }

    @Override public void onProgressChanged(WebView view, int newProgress) {
        if (newProgress == FINISHED && view.getTitle() != null && callback != null) {
            callback.onTransitionToIdle();
        }
    }

    @Override public void onReceivedTitle(WebView view, String title) {
        if (webView.getProgress() == FINISHED && callback != null) {
            callback.onTransitionToIdle();
        }
    }

    @Override public String getName() {
        return "WebView idling resource";
    }

    @Override public boolean isIdleNow() {
        // The webView hasn't been injected yet, so we're idling
        if (webView == null) return true;
        return webView.getProgress() == FINISHED && webView.getTitle() != null;
    }

    @Override public void registerIdleTransitionCallback(ResourceCallback resourceCallback) {
        this.callback = resourceCallback;
    }
}

After creating your own custom idling resource, it needs to be registered with Espresso by calling Espresso.registerIdlingResource(webViewIdlingResource).

Register a component tied to an Activity instance

Sometimes the component you need to wait for is tied to an activity instance - i.e. a page in a webview to fully load: in this case things can get tricky. To understand, why let's go back to the getName() method and have a closer look at the docs, at some point they state

it is used for logging and idempotency of registration

Here's a definition of "idempotence" taken from Wikipedia:

Idempotence is the property of certain operations in mathematics and computer science, that can be applied multiple times without changing the result beyond the initial application.

What this means is that after the first time Espresso.registerIdlingResource(webViewIdlingResource) is called, all subsequent calls for an idling resource with the same name won't have any effect (Espresso will simply log a warning). Generally this is no big deal, but it becomes an issue if an idling resource has a dependency to the current Context (i.e. the current activity). Let's take our WebViewIdlingResource, it needs a reference to a WebView in the current activity in order to do its job. A naive use of it would be something like this:


public class WebsiteActivityTestBase extends ActivityInstrumentationTestCase2<WebsiteActivity> {

    public WebsiteActivityTestBase() {
        super(WebsiteActivity.class);
    }

    @Override public void setUp() throws Exception {
        getActivity();
        Espresso.registerIdlingResource(new WebViewIdlingResource(getActivity().findViewById(R.id.webview)));
    }

    public void testSomething() {}

    public void testSomethingElse() {}
}

Because of idempotence, only the first idling resource registered will be remembered by Espesso and this is bad for at least two reasons:

  1. all test cases will rely on that idling resource, but its reference to the webview is not valid anymore because a new activity has been created
  2. the context of the first activity is being leaked

To solve these issues, we can introduce an idling resource that is tied to the activity lifecycle and that injects and clears the reference to a component when appropriate.


interface ActivityLifecycleIdlingResource<T> extends IdlingResource {

  void inject(T activityComponent);

  void clear();
}

Now let's modify our WebViewIdlingResource class to implement this interface


public class WebViewIdlingResource extends WebChromeClient implements ActivityLifecycleIdlingResource<WebView> {

    private WebView webView;

    // Same code as above

    @Override public void inject(WebView webView) {
        this.webView = checkNotNull(webView,
                String.format("Trying to instantiate a \'%s\' with a null WebView", getName())));
        // Save the original client if needed.
        this.webView.setWebChromeClient(this);
    }

    @Override public void clear() {
        // Free up the reference to the webView
        webView = null;
    }
}

The last step is to find a good place to inject and clear the webview reference, for this we can leverage the ActivityLifecycleCallback interface


class WebViewInjector implements ActivityLifecycleCallback {

    private final WebViewIdlingResource webViewIdlingResource;

    WebViewInjector(WebViewIdlingResource webViewIdlingResource) {
        this.webViewIdlingResource = webViewIdlingResource;
    }

    @Override public void onActivityLifecycleChanged(Activity activity, Stage stage) {
        ComponentName websiteActivityComponentName =
                new ComponentName(activity, WebsiteActivity.class.getName());
        if (!activity.getComponentName().equals(websiteActivityComponentName)) return;

        switch (stage) {
            case CREATED:
                // We need to wait for the activity to be created before getting a reference
                // to the webview
                Fragment webViewFragment =
                        activity.getFragmentManager().findFragmentByTag(WebsiteScreen.TAG);
                webViewIdlingResource.injectWebView((WebView) webViewFragment.getView());
                break;
            case STOPPED:
                // Clean up reference
                if (activity.isFinishing()) webViewIdlingResource.clearWebView();
                break;
            default: // NOP
        }
    }
}

It's not particularly concise and there's a lot of wiring involved but it does what we want.

Register for a component tied to an Application instance

Things are easier if you have to wait for a component tied to the application object, since it's around for the whole life of the app and typically you don't have to care about registering/unregistering components. These days 90% of the apps out there rely on an API to get data and need to handle asychronous I/O. On Android there are a few ways to do that, everyone with its pros and cons, but in any case you need to wait for the API call to return. If you're using AsyncTask Espresso has support for it out of the box, but there can be reasons why you'd want to not use the built-in Android components and use Java thread pools instead. In this case you'd need to define a custom idling resource that checks if the executor(s) used by the application are idle. Looking at the Espresso source code, with a small refactoring to the AsyncTaskPoolMonitor class (Espresso uses it to check if there is some tasks running on the AsyncTask thread pool executor) a general-purpose ThreadPoolIdlingResource can be implemented.

Wrapping up

There are a few edge cases that one need to keep in mind in order to use idling resources properly. I described the current solution I ended up implementing, another - probably less error-prone - solution would be to have an Espresso.unregisterIdlingResource(myIdlingResource) API (there is already a feature request to add it). As for registering idling resources, I ended up doing it inside the callApplicationOnCreate(app) method of a custom InstrumentationTestRunner, this way I am sure the registration happens only once when the test suite is created.


P.S.: Jimdo is looking for an Android developer to join our team in Hamburg! Feel free to contact me if you're interested.

Soundtrack

Stefano Dacchille | half man. half Android.

stefano@jimdo.com, @mod3rn0