Unit tests sometimes failing, sometimes passing

jenkins, mocha.js, unit-testing

Solution

Tests can fail intermittently for a number of reasons and identifying why they fail is often revealing about your codebase and environment.

Here are a few possible causes:

• Shared objects - singletons that hold state can cause problems between tests if the test environment isn't reset to a well known state. if if your test runner executes tests in a non-deterministic order you may see random errors that are actually exposing corrupted state issues

• Environmental and external dependencies - any external object that can hold state can cause unpredictable results

• Timing - sometimes tests are written with timeouts or thread sleeps that are too specific. If the build server is operating under heavy load these timeouts may not be long enough

As general guidance, tests must be:

- isolated: tests focus on one unit at a time

- repeatable: produces the same results each time

- independent: the order in which tests are executed should not matter

Problem

Original source

Related problems

• How do I write a JUnit test case to test threads and events
• How should I unit test multithreaded code?
• Thread.sleep in JUnit test case
• Unit-testing multithreaded applications