Ensure zig build test includes all tests

[hendriknielaender]

It has come to our attention that not all unit tests are being executed when the zig build test command is invoked. This command is the standard way to run tests in a zig project, and it's expected to compile and execute all test cases defined across the project. However, some tests appear to be omitted during this process.

Expected Behavior

The zig build test command should automatically find and run all unit tests in the project, regardless of their location within the codebase. This ensures a consistent and reliable testing process, which is crucial for maintaining code quality and stability.

Suggested Solution

I propose an adjustment to the build.zig file to ensure that all unit tests are included when running zig build test. This may involve reviewing the build configurations to guarantee that each test target is appropriately identified and included in the test run.

[jasperdunn]

I've seen this pattern used a bunch, though I'm sure Zig will eventually have a better solution for it:

// src/tests.zig

comptime {
    _ = @import("foo.zig");
    _ = @import("bar.zig");
    _ = @import("baz.zig");
}

// build.zig

const unit_tests = b.addTest(.{
        .root_source_file = .{ .path = "src/tests.zig" },
        .target = target,
        .optimize = optimize,
    });

[FObersteiner]

Another option is to handle tests like we handle the examples, i.e. via the build-script. Suppose you have 3 zig files with tests in direcory ./src, you would define their names like

const test_files = [_][]const u8{ "test_a", "test_b", "test_c" };

then call to run each of them in the build.zig with

const test_step = b.step("tests", "Run library tests");
for (test_files) |test_name| {
    const _test = b.addTest(.{
        .name = test_name,
        .root_source_file = .{ .path = b.fmt("src/{s}.zig", .{test_name}) },
        .target = target,
        .optimize = optimize,
    });
    const run_test = b.addRunArtifact(_test);
    test_step.dependOn(&run_test.step);
}

That seems quiet clear to me, just look at the build.zig an you know what's happening; no nested calls within source code files to other files etc.

---

Addendum, I tried to find a way to avoid caching for tests. One option is to completely "disable" caching by dumping the build cache to some temp directory. However, that is not what we want I think. I also set up a question on ziggit. For now, IMHO the best option is to keep benchmark in "tests", but compile those to binaries that you can run repeatedly if desired - or live with the fact that tests only run once, after you change the code that is tested. On the long run, maybe benchmarks can become a std lib feature :)

[hendriknielaender]

Close this issue since, now all tests should be included.

Opened a new issue regarding the caching issue for the benchmark test #26