Skip to content

Adding Offload Test into DXC build pipeline #8584

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
joaosaffran merged 5 commits into from
Jun 26, 2026
+193 −1
Merged

Adding Offload Test into DXC build pipeline #8584

Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
1cab50c
add vk to release pipeline
joaosaffran-zz Jun 23, 2026
fad2cb9
clean up
joaosaffran-zz Jun 24, 2026
aba9720
address comments, test changes
joaosaffran-zz Jun 25, 2026
418b47c
remove release-dxil from test
joaosaffran-zz Jun 25, 2026
5c25c24
add parameters
joaosaffran-zz Jun 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
164 changes: 163 additions & 1 deletion azure-pipelines.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,15 @@ stages:
VS2022_Release:
configuration: Release
spirvBuildFlag: -spirvtest
artifactName: dxc_spirv_release
VS2022_Debug:
configuration: Debug
spirvBuildFlag: -spirvtest
artifactName: ''
VS2022_Release_NoSPIRV:
configuration: Release
spirvBuildFlag: ''
artifactName: dxc_dxil_release

steps:
- checkout: self
Expand Down Expand Up @@ -76,7 +79,27 @@ stages:
call utils\hct\hcttest.cmd -$(configuration) compat-suite v1.8.2505.1
displayName: 'DXIL Compat Suite Tests (1.8 point release)'
condition: succeededOrFailed()

- script: |

@damyanp damyanp Jun 24, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For PR builds, I'm wondering if we should only do this if the PR has a particular label on it? I don't think we'll want to gate all PRs on this since it'll add significant time to the builds.

call utils\hct\hctstart.cmd %HLSL_SRC_DIR% %HLSL_BLD_DIR%
call utils\hct\hcttest.cmd -$(configuration) spirv
displayName: 'SPIR-V Tests'
condition: and(succeeded(), ne(variables['spirvBuildFlag'], ''))
- pwsh: |
$bin = "$(HLSL_BLD_DIR)\$(configuration)\bin"
$dst = "$(Build.ArtifactStagingDirectory)\dxc"
New-Item -ItemType Directory -Force -Path $dst | Out-Null
foreach ($f in @("dxc.exe","dxv.exe","dxcompiler.dll","dxil.dll")) {
if (Test-Path "$bin\$f") { Copy-Item "$bin\$f" $dst } else { Write-Warning "missing $f" }

@alsepkow alsepkow Jun 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of logging a warning, should we fail if one of the files doesn't exist?

}
displayName: 'Stage DXC binaries'
condition: and(succeeded(), ne(variables['artifactName'], ''))
- task: PublishPipelineArtifact@1
displayName: 'Publish DXC binaries'
condition: and(succeeded(), ne(variables['artifactName'], ''))
inputs:
targetPath: '$(Build.ArtifactStagingDirectory)\dxc'
artifact: '$(artifactName)'


- job: Nix
timeoutInMinutes: 165
Expand Down Expand Up @@ -177,3 +200,142 @@ stages:
testResultsFormat: 'JUnit'
testResultsFiles: '**/testresults.xunit.xml'
condition: succeededOrFailed()

- stage: OffloadTests
displayName: 'Offload tests'
dependsOn: Build
condition: succeededOrFailed()
variables:
LAVAPIPE_VERSION: '26.1.2'
WARP_VERSION: '1.0.19'

@alsepkow alsepkow Jun 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are hard coded versions intentional? I worry about needing to consistently bump these versions.

VS_DEV: 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvars64.bat'

@damyanp damyanp Jun 24, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shouldn't hardcode this string. We should be able to reuse some of the logic in the other scripts to find this.


jobs:
- job: Offload
timeoutInMinutes: 180
pool:
vmImage: windows-2025
workspace:
clean: all
strategy:
matrix:
dxil_release:
artifact: 'dxc_dxil_release'
runLavapipe: 'false'
spirv_release:
artifact: 'dxc_spirv_release'

@alsepkow alsepkow Jun 25, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whats the reasoning for this? If we just default this to true all the time then we could remove the branhcing logic and just do that every run instead?

@joaosaffran joaosaffran Jun 25, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not all emitted binaries support SPIR-V codegen; in those cases, we don't want to test on lavapipe.

@Icohedron Icohedron Jun 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This matrix only has one element: spirv_release. Is it planned to add more configurations to this matrix in the future? If not, we could fold that runLavapipe check into the steps instead of as part of the matrix.

@joaosaffran joaosaffran Jun 27, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have current plans to add more configurations, although I personally think we should. I personally think that leaving the matrix with one element helps clarify that this is a testing job and that we could add more binaries to be tests easily to that stage

runLavapipe: 'true'
steps:
- checkout: none

- task: DownloadPipelineArtifact@2
displayName: 'Download DXC binaries'
inputs:
artifact: '$(artifact)'
path: '$(Agent.TempDirectory)\dxc'

- pwsh: |
$dxc = Get-ChildItem "$(Agent.TempDirectory)\dxc" -Recurse -Filter dxc.exe | Select-Object -First 1
if (-not $dxc) { throw "dxc.exe not found in artifact $(artifact)" }
Write-Host "##vso[task.setvariable variable=DXC_DIR]$($dxc.Directory.FullName)"
Write-Host "DXC_DIR = $($dxc.Directory.FullName)"
displayName: 'Locate DXC_DIR'

- task: UsePythonVersion@0
inputs:
versionSpec: '3.11'
- script: choco install ninja -y

@Icohedron Icohedron Jun 25, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We install ninja using chocolatey as opposed to winget?

@joaosaffran joaosaffran Jun 25, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In previous jobs, we used hctstart, which is only available inside DXC repo. I am avoiding cloning that in this stage of the pipeline. Not sure how hctstart gets ninja installed

@Icohedron Icohedron Jun 26, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is Chocolatey a standard piece of software that's available on all the Windows VMs?
I was thinking WinGet would be more preferrable than Chocolatey, since Chocolatey is a third-party piece of software whereas WinGet is included with Windows.

@joaosaffran joaosaffran Jun 27, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chocolatey is available as part of the 1ES VM images, which we use in this build

displayName: 'Install Ninja'
- task: BatchScript@1
displayName: 'Set up MSVC environment'
inputs:
filename: '$(VS_DEV)'

@damyanp damyanp Jun 24, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does hctstart.cmd actually do this for you? Might be a way to avoid hard coding the script path?

@joaosaffran joaosaffran Jun 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would require cloning DXC repo in the offload-tests worker which I was avoiding doing... I've modified the script to find MSVC variables instead.

modifyEnvironment: true

- pwsh: |
$exe = "$(Agent.TempDirectory)\VulkanSDK.exe"
Invoke-WebRequest "https://sdk.lunarg.com/sdk/download/latest/windows/vulkan_sdk.exe?Human=true" -OutFile $exe
Start-Process $exe -ArgumentList "--accept-licenses","--default-answer","--confirm-command","install" -Wait
$sdk = (Get-ChildItem "C:\VulkanSDK" -Directory | Sort-Object Name -Descending | Select-Object -First 1).FullName
if (-not $sdk) { throw "Vulkan SDK install dir not found under C:\VulkanSDK" }
Write-Host "##vso[task.setvariable variable=VULKAN_SDK]$sdk"
Write-Host "##vso[task.prependpath]$sdk\Bin"
displayName: 'Install Vulkan SDK'
condition: and(succeeded(), eq(variables['runLavapipe'], 'true'))

- pwsh: |
$ver = "$(LAVAPIPE_VERSION)"
$url = "https://github.com/pal1000/mesa-dist-win/releases/download/$ver/mesa3d-$ver-release-msvc.7z"
Invoke-WebRequest $url -OutFile "$(Agent.TempDirectory)\mesa.7z"
7z x "$(Agent.TempDirectory)\mesa.7z" -o"$(Agent.TempDirectory)\mesa" -y | Out-Null
$lvp = (Get-ChildItem "$(Agent.TempDirectory)\mesa" -Recurse -Filter "lvp_icd.x86_64.json" | Select-Object -First 1).FullName
if (-not $lvp) { throw "lavapipe ICD (lvp_icd.x86_64.json) not found in Mesa package" }
reg add "HKLM\SOFTWARE\Khronos\Vulkan\Drivers" /v "$lvp" /t REG_DWORD /d 0 /f /reg:64
Write-Host "##vso[task.setvariable variable=MESA_BIN]$(Split-Path $lvp)"
Write-Host "Registered lavapipe ICD: $lvp"
displayName: 'Install lavapipe (Mesa) and register the ICD'
condition: and(succeeded(), eq(variables['runLavapipe'], 'true'))

- pwsh: |
git clone --depth 1 https://github.com/llvm/offload-test-suite OffloadTest
git clone --depth 1 https://github.com/llvm/llvm-project llvm-project
git clone --depth 1 https://github.com/llvm/offload-golden-images golden-images
python -m pip install -r OffloadTest/test/requirements.txt
$py = (python -c "import sys; print(sys.executable)").Trim() -replace '\\','/'
Write-Host "##vso[task.setvariable variable=PYTHON_EXE]$py"
displayName: 'Checkout offload-test-suite + LLVM'
workingDirectory: '$(Agent.TempDirectory)'

- script: >
cmake -G Ninja -B llvm-project/build
-DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=On
-DLLVM_EXTERNAL_PROJECTS=OffloadTest
-C llvm-project/clang/cmake/caches/HLSL.cmake
-DPython3_EXECUTABLE="$(PYTHON_EXE)"
-DDXC_DIR="$(DXC_DIR)"
-DLLVM_EXTERNAL_OFFLOADTEST_SOURCE_DIR=$(Agent.TempDirectory)/OffloadTest
-DGOLDENIMAGE_DIR=$(Agent.TempDirectory)/golden-images
-DOFFLOADTEST_TEST_CLANG=Off
-DOFFLOADTEST_ENABLE_DEBUG=Off
-DWARP_VERSION=$(WARP_VERSION)
-DLLVM_LIT_ARGS="--xunit-xml-output=testresults.xunit.xml -v"
llvm-project/llvm
displayName: 'Configure LLVM + OffloadTest against DXC'
workingDirectory: '$(Agent.TempDirectory)'

- pwsh: |
$line = (Select-String -Path "llvm-project/build/CMakeCache.txt" -Pattern '^Python3_EXECUTABLE:').Line
$py = ($line -split '=', 2)[1]
if (-not $py) { throw "Python3_EXECUTABLE not found in CMakeCache.txt" }
Write-Host "lit Python: $py"
& "$py" -m pip install -r OffloadTest/test/requirements.txt
displayName: 'Install offload requirements into lit Python'
workingDirectory: '$(Agent.TempDirectory)'

- script: cmake --build llvm-project/build --target hlsl-test-depends
displayName: 'Build offload test dependencies'
workingDirectory: '$(Agent.TempDirectory)'

- pwsh: |
$out = "$(Agent.TempDirectory)/results-d3d12-warp.xml" -replace '\\','/'
$env:LIT_OPTS = "--xunit-xml-output=$out"
cmake --build llvm-project/build --target check-hlsl-warp-d3d12
displayName: 'DirectX offload tests on WARP'
condition: succeededOrFailed()
workingDirectory: '$(Agent.TempDirectory)'

- pwsh: |
$env:PATH = "$(MESA_BIN);$env:PATH"
$env:VK_LOADER_DEBUG = "all"
$out = "$(Agent.TempDirectory)/results-vk-lavapipe.xml" -replace '\\','/'
$env:LIT_OPTS = "--xunit-xml-output=$out"
cmake --build llvm-project/build --target check-hlsl-vk
displayName: 'Vulkan offload tests on lavapipe'
condition: and(succeededOrFailed(), eq(variables['runLavapipe'], 'true'))
workingDirectory: '$(Agent.TempDirectory)'

- task: PublishTestResults@2
condition: succeededOrFailed()
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: '$(Agent.TempDirectory)/results-*.xml'
Loading