How can I run a specific set of Android instrumentation tests with Gradle?

Question

I have multiple test packages: ``` com.mypackage.blackbox - Robotium UI tests com.mypackage.integration - REST integration tests com.mypackage.unit - low level unit tests ``` Our server team needs to be able to run just the integration tests on every push (they take a couple of minutes), but then run all tests every night (the black box UI tests take more than 10 minutes). This great answer provides a slightly hacky (but effective) way to do it by overloading an existing JUnit annotation like `@SmallTest` or `@LargeTest`. The Gradle documentation suggests that test filters are the way to do this, e.g. ``` ./gradlew connectedAndroidTestDevDebug --tests com.mypackage.integration.* ``` However, that fails with an `> Unknown command-line option '--tests'.` error (presumably because the Android Gradle plugin doesn't support everything that vanilla Gradle does?). The same documentation says in future they plan to support these alternatives: - Filtering based on custom annotations (future) -…

Accepted Answer

This is now possible with the addition of Android's recent Testing Support Library, you can now use `AndroidJUnitRunner` and filter the tests you run by your own custom annotations.

Filter test run to tests with a custom annotation (com.myapp.MyAnnotation in this example):

```
adb shell am \
  instrument -w -e annotation com.myapp.MyAnnotation \
  com.myapp/android.support.test.runner.AndroidJUnitRunner
```

Complete AndroidJUnitRunner Documentation

You'll need to annotate your test cases with your custom annotation to get this to work. Example test case:

```
import android.support.test.runner.AndroidJUnit4;
import com.myapp.MyAnnotation;

@RunWith(AndroidJUnit4.class)
public class CalculatorTest {

@MyAnnotation
    @Test
    public void testAddition() {
        //Do testing here
    }

}
```

Here is what your "MyAnnotation" would look like:

```
package com.myapp;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

/**
 * My custom Annotation to specify a type of tests to run.
 */
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD, ElementType.TYPE})
public @interface MyAnnotation {
}
```

How can I run a specific set of Android instrumentation tests with Gradle?

Solution

Problem

Related problems