How can I use a custom runner when using categories in Junit?
gradle, java, junit, junit-runner, unit-testing
Solution
The solution to this turned out to smaller and trickier than I thought. Gradle was using my custom test runner and correctly invoking the `filter` method. However, my runner reloads all test classes through its own classloader for Javaassist enhancements.
This lead to the issue that `SlowTest` annotation was loaded through the Gradle classloader but when passed to my custom runner, the runner checked if the class was annotated with that annotation. This check never resolved correctly as the equality of the `SlowTest` annotation loaded through two different classloaders was different.
--
Since I've already done the research, I'll just leave this here. After days of digging through the Gradle and the (cryptic) JUnit sources, here's what I got.
Gradle simply doesn't handle any advanced JUnit functionality except the test categorization. When you create a Gradle task with the include-categories or the exclude-categories conditions, it builds a CategoryFilter. If you don't know, a `Filter` is what JUnit gives to the test-runner to decide whether a test or a test method should be filtered out. The test runner must implement the `Filterable` interface.
JUnit comes with multiple runners, the `Categories` is just another one of them. It extends a family of test runners called `Suite`. These suite based runners are designed to run a "suite" of tests. A suite of tests could be built by annotation introspection, by explicitly defining tests in a suite or any other method that builds a suite of tests.
In the case of the `Categories` runner, JUnit has it's own `CategoryFilter` but Gradle doesn't use that, it uses it's own `CategoryFilter`. Both provide more or less the same functionality and are JUnit filters so that can be used by any suite that implements `Filterable`.
The actual class in the Gradle responsible for running the JUnit tests is called `JUnitTestClassExecuter`. Once it has parsed the command line options it requests JUnit to check the runner should be used for a test. This method is invoked for every test as seen here.
The rest is simply up to JUnit. Gradle just created a custom `RunNotifier` to generate the standard XML files representing test results.
I hope someone finds this useful and saved themselves countless hours of debugging.
TLDR: You can use any runner in Gradle. Gradle has no specifics pertaining to runners. It is JUnit that decided the runners. If you'd like to know what runner will be used for your test, you can debug this by calling `Request.aClass(testClass).getRunner()`. Hack this somewhere into your codebase and print it to the console. (I wasn't very successful in attaching a debugger to Gradle.)
Problem
I have a bunch of JUnit tests that extend my base test class called `BaseTest` which in turn extends `Assert`. Some of my tests have a `@Category(SlowTests.class)` annotation. My `BaseTest` class is annotated with the following annotation `@RunWith(MyJUnitRunner.class)`. I've set up a Gradle task that is expected to run only `SlowTests`. Here's my Gradle task: ``` task integrationTests(type: Test) { minHeapSize = "768m" maxHeapSize = "1024m" testLogging { events "passed", "skipped", "failed" outputs.upToDateWhen {false} } reports.junitXml.destination = "$buildDir/test-result" useJUnit { includeCategories 'testutils.SlowTests' } } ``` When I run the task, my tests aren't run. I've pinpointed this issue to be related to the custom runner `MyJUnitRunner` on the `BaseTest`. How can I set up my Gradle or test structure so that I can use a custom runner while using the `Suite`.