Enable async testing in CI

[MichalStrehovsky]

With #121814 merged we should be able to start running these in the CI for native AOT.

Cc @dotnet/ilc-contrib

[dotnet-policy-service]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

[Copilot]

Pull Request Overview

This PR enables async testing in the CI pipeline for native AOT builds following the merge of prerequisite work in #121814. The changes selectively enable the async test suite for NativeAOT while keeping it disabled for other runtime flavors.

• Modified build configuration to allow async tests to run for NativeAOT
• Added async tests to the Checked configuration NativeAOT CI pipeline

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
src/tests/async/Directory.Build.targets	Updated DisableProjectBuild conditions to enable async tests specifically for NativeAOT (when TestBuildMode is 'nativeaot') while keeping them disabled for mono, wasm, and default CoreCLR builds
eng/pipelines/runtime.yml	Added "async;" to the test build arguments for the NativeAOT Checked configuration to include async tests in the CI test run

[MichalStrehovsky]

/azp run runtime-nativeaot-outerloop

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[MichalStrehovsky]

/azp run runtime-nativeaot-outerloop

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[MichalStrehovsky]

/azp run runtime-nativeaot-outerloop

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[MichalStrehovsky]

/azp run runtime-nativeaot-outerloop

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[am11]

@MichalStrehovsky, can we add a few representative runtime-async cases somewhere in smoketest as well?

[MichalStrehovsky]

@MichalStrehovsky, can we add a few representative runtime-async cases somewhere in smoketest as well?

We have been slowly moving away from adding AOT-specific testing for things that are not AOT specific. Adding a super basic test would be fine I guess (somewhere under SmokeTests/UnitTests) but at this point where only few architectures work it's extra hassle because we need to also duplicate (and keep up to date) the per-platform disabling.

[MichalStrehovsky]

/azp run runtime-nativeaot-outerloop

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[am11]

but at this point where only few architectures work it's extra hassle because we need to also duplicate (and keep up to date) the per-platform disabling.

Actually, that’s exactly my interest: being able to see how well the community platforms are holding up. Smoketest lowers the infrastructure overhead enough to make that feasible. Running the full test suite is still blocked (#115622 (comment)), so we’ve been using hacky workflows; building only the smoke tests and running them on-device and in QEMU VMs.

[MichalStrehovsky]

but at this point where only few architectures work it's extra hassle because we need to also duplicate (and keep up to date) the per-platform disabling.

Actually, that’s exactly my interest: being able to see how well the community platforms are holding up. Smoketest lowers the infrastructure overhead enough to make that feasible. Running the full test suite is still blocked (#115622 (comment)), so we’ve been using hacky workflows; building only the smoke tests and running them on-device and in QEMU VMs.

You could instead run the subset specified at

https://github.com/MichalStrehovsky/runtime/blob/d01347075010383bd6ad2b41d4daf27175469900/eng/pipelines/runtime.yml#L608

That's the "important subset" that we want to run on all PRs. It covers the important things that are not covered in SmokeTests and it's still cheaper to run than all of pri0 tests.

[am11]

Claude Sonnet 4.5 generated this main...am11:runtime:chore/smoketess-runtime-async, if we could get something like this in smoketests, that would be great. :)

MichalStrehovsky/runtime@d013470/eng/pipelines/runtime.yml#L608

Thanks, I’ll give them a try on linux-riscv64 once the infra is sorted out. At the moment, test builds/runs on community platforms are still rough due to infra issues.