Problems With Asynchronous Assertions

[JasonBock]

Overall, I like TUnit, but one area I'm finding problematic is with its insistence that everything must be asynchronous. This is starting to become kind of a PITA in certain scenarios:

• Unsafe contexts. Since assertions have to be awaited, which makes the method async, you start to run into issues like CS4004 and CS9123.
• Span<> usage. If I have a Span<> in a test, and I want to read values from that Span<> within Assert.That(), I can't, because I'll get CS4007. I can get around this by capturing all the values before I start doing assertions, but this is kind of clumsy and unnecessary.
• Just in general, forcing asynchronous behavior on tests that are CPU-bound also feels kind of odd. I get that TUnit wants to do everything in parallel, but that doesn't mean that the test itself has to be asynchronous.

While unsafe code isn't common, it is valid. Furthermore, Span<> types are showing up more often, and eliminating the async boundary would make tests easier to write. I feel like there should be some consideration to having synchronous assertions. Curious to hear what others have to say.

[thomhurst]

One of the main reasons for this is that the await keyword actually triggers the execution of the assertion. Without awaiting it, you're just building a fluent chain of options.

Consider this:

Assert.That(number).IsDivisibleBy(2).Or.IsDivisibleBy(5)

If assertions executed immediately, without the await keyword, this may fail and throw immediately on the IsDivisibleBy(2). By lazily triggering the execution, we allow the full chain to be built first, which gives us more context and therefore control over what to do and how to handle the scenario.

Sure, I could have opted for an .Execute() method or something, but I think that's even worse.

I can't really think of a way around this with the current state of the language.

I think your options are:

• Creating your own extension method like .Execute() that calls .GetAwaiter().GetResult() - But accept that you'll be doing sync-over-async.
• Bring in your own other assertion library. TUnit is built in a modular way, so you can bring in TUnit.Engine as the test runner, and whatever other assertions library you'd like to use.

[CrispyDrone]

Out of curiosity, would it be possible for Or to capture the assertion failure in case it happens and then evaluate the other branch to see if that one succeeds? I have not looked at the code base so I have no idea how it's actually implemented yet or if this is an implementation that wouldn't cause issues elsewhere.

What about changing the syntax?

Assert.That(number).IsDivisibleBy(2, Or.IsDivisibleBy(5));
Assert.That(number).IsDivisibleBy(2.Or(5));

Alternatively, but unrealistically, the entire assertion library could be rewritten to use weaving meaning the actual assertions don't do anything and it's just a language expressing intent.

Finally, I think it's great that we can simply use another assertion library if we want to, so it's not exactly a blocking issue.

[JasonBock]

This isn't about chaining assertion calls, it's about valid C# coding scenarios. In the unsafe example, this code now fails:

var pValue = &value;

This isn't even in an assertion, but because the whole test now needs to be async, I get CS9123.

With a ref struct like Span<>, this code is now an issue with CS4007:

await Assert.That(data[0]).IsEqualTo(1);

That isn't chaining, just a simple equality check. But because I have to await, I have an issue.

[thomhurst]

I know - I'm just saying that the assertions library is built in a way that you build a chain (even if that chain is 1 element in your case) and then the await performs the execution.

[JasonBock]

So it sounds like the answer is, "use another assertion framework."

[kzrnm]

Would it make sense to keep the tasks and process them all at once with Task.WhenAll, as shown below?

#pragma warning disable TUnitAssertions0002
    [Test]
    public async Task SpanTest()
    {
        Span<int> span = [1, 2, 3];
        var list = new List<Task>();
        using (Assert.Multiple())
        {
            list.Add(Assert.That(span.Length).IsNotEqualTo(3).AssertAsync());
            list.Add(Assert.That(span[^1]).IsNotEqualTo(3).AssertAsync());
        }
        await Task.WhenAll(list);
    }

    [Test]
    public async Task SpanTestWithUsing()
    {
        Span<int> span = [1, 2, 3];
        await using (var multi = new MultipleAsync())
        {
            multi.Add(Assert.That(span.Length).IsNotEqualTo(3));
            multi.Add(Assert.That(span[^1]).IsNotEqualTo(3));
        }
    }

    class MultipleAsync : IAsyncDisposable
    {
        [System.Diagnostics.CodeAnalysis.SuppressMessage("Usage", "TUnitAssertions0004")]
        IDisposable AssertionScope = Assert.Multiple();
        List<Task> list = new();
        public void Add(IAssertion assertion)
        {
            list.Add(assertion.AssertAsync());
        }   
        public async ValueTask DisposeAsync()
        {
            AssertionScope.Dispose();
            await Task.WhenAll(list);
        }
    }
#pragma warning restore TUnitAssertions0002

[thomhurst]

If you use assert multiple this is kinda what it does