@​RepeatedTest: why is value == failureThreshold a precondition violation?

[beatngu13]

#2925 introduced failureThreshold for @RepeatedTest as part of JUnit Jupiter 5.10.0, which (to my understanding) should also address #2119. Now, @RepeatedTest and JUnit Pioneer's @RetryingTest are becoming more and more similar (🎉).

What I don't understand: @RetryingTest(2) retries a (flaky) test one more time, if the first execution fails. How can I get the same behavior with @RepeatedTest? That is, at most two test executions, where the first one is allowed to fail.

1. value = 2, failureThreshold = 1 fails with "Failure threshold [1] exceeded".
2. value = 2, failureThreshold = 2 violates the precondition "'failureThreshold' […] less than the total number of repetitions".
3. value = 3, failureThreshold = 2 executes the test three times.

I wonder why value == failureThreshold a precondition violation?

Reproducer:

@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class RepeatedVsRetryingTest {

	Map<Method, Integer> testMethod2NumExecutions = new HashMap<>();

	@RepeatedTest(value = 2, failureThreshold = 1)
	void testRepeatedTest(TestInfo testInfo) {
		firstFailThenSucceed(testInfo);
	}

	@RetryingTest(2)
	void testRetryingTest(TestInfo testInfo) {
		firstFailThenSucceed(testInfo);
	}

	void firstFailThenSucceed(TestInfo testInfo) {
		Method testMethod = testInfo.getTestMethod().orElseThrow();
		int numExecutions = testMethod2NumExecutions.compute(testMethod, (k, v) -> v == null ? 0 : v + 1);
		if (numExecutions == 0) {
			fail();
		}
	}

}

[sbrannen]

How can I get the same behavior with @RepeatedTest? That is, at most two test executions, where the first one is allowed to fail.

The following works for me.

@RepeatedTest(2)

I wonder why value == failureThreshold a precondition violation?

The rationale is that there is no need to declare a threshold in that situation.

[beatngu13]

Ah, that was too easy. 😅 Thank you, Sam!

But – in contrast to @RetryingTest – it is not possible to stop repeating after the first successful execution?

[sbrannen]

But – in contrast to @RetryingTest – it is not possible to stop repeating after the first successful execution?

That's correct.