Detector suggestion: Expecting Error in unit tests

[ThrawnCA]

Tripped myself up recently when unit-testing some code that is meant to handle a subset of java.lang.Error (specifically, it's a scheduled job that shouldn't completely kill the scheduler if a particular invocation encounters out-of-memory errors).

The thing is, when writing the unit tests that threw a different (not to be handled) Error, I told TestNG to expect the test case to throw Error. Which meant that when I made a mistake in the test, the resulting AssertionError didn't cause a test failure...

Can a new TestNG/JUnit bug be added to detect cases where tests expect AssertionError or a superclass?

Sample:

// oops, this will swallow AssertionError!
@Test(expectedExceptions = Error.class)
public void shouldNotHandleFatalErrors() {
    when(dao.findAll()).thenThrow(new UnknownError("test error"));

    service.new ReportDeliveryTask(Frequency.DAILY);
    // oops, the exception won't be thrown until we actually "run" the task

    fail("Should have aborted due to fatal error");
}

(The fail line was triggering, but the test still passed...mea culpa)

[mebigfatguy]

so if you expect Error, then have any kind of Assert.xxx code, report that?

[ThrawnCA]

It's more "if you expect AssertionError or any superclass of it, report that." IMO you don't need to look for an actual assertion in the method body. AssertionError is pretty fundamental to the idea of a unit test, so it almost never makes sense to swallow it.