[Tests] Diagnose Windows cert creation failures

[paulmedynski]

Diagnostic changes for #3223. We're now emitting the PowerShell script content used to generate a self-signed cert and add it to the Windows system cert store. This may help figure out why the cert operations fail intermittently.

Created this draft PR to run the CI pipelines where the failures sometimes occur and acquire the PowerShell content for further investigation.

[edwardneal]

I think I introduced this in #3034, which used PowerShell to generate certificates more frequently. Apologies if so!

The only difference in the PS invocation which I can see between my version and the preceding one is that we no longer load the user profile. Does setting ProcessStartInfo.LoadUserProfile to true fix this?

[paulmedynski]

I think I introduced this in #3034, which used PowerShell to generate certificates more frequently. Apologies if so!

The only difference in the PS invocation which I can see between my version and the preceding one is that we no longer load the user profile. Does setting ProcessStartInfo.LoadUserProfile to true fix this?

LoadUserProfile is already being set to true, so likely not the issue. I've run some manual tests in a Windows Server container, and the PS script works fine. Since the failures are intermittent, I suspect a flaky test environment.

We're having some ADO CI pipeline issues that are preventing me from running CI reliably right now, so I will circle back once the overall CI stuff settles down.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 72.82%. Comparing base (1247ca4) to head (1336f73).
Report is 8 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3237      +/-   ##
==========================================
+ Coverage   72.69%   72.82%   +0.12%     
==========================================
  Files         303      298       -5     
  Lines       59718    59612     -106     
==========================================
- Hits        43414    43412       -2     
+ Misses      16304    16200     -104     

Flag	Coverage Δ
addons	92.58% <ø> (ø)
netcore	75.27% <ø> (+0.17%)	⬆️
netfx	71.41% <ø> (+0.03%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[paulmedynski]

The latest commit adds a retry within the CertificateFixtureBase.CreateCertificate that seems to work around the ACCESS DENIED issue. We now see this in the test logs:

2025-03-31T17:22:11.0473456Z   PowerShell command failed with exit code 1 on attempt 1 of 3; retrying in 5 seconds...
...
2025-03-31T17:22:19.7322237Z     Passed Microsoft.Data.SqlClient.Tests.AlwaysEncryptedTests.ExceptionsAlgorithmErrors.TestInvalidCipherText [44 ms]

There are no warnings or errors for the CI runs.

[Copilot]

Pull Request Overview

This PR updates various preprocessor directives from NET9_0 to NET9_0_OR_GREATER and introduces enhanced diagnostics for Windows certificate creation failures via retry logic and additional output details. Key changes include updating build conditions in multiple certificate and SQL components, adding retry logic with delay and logging for PowerShell command execution in certificate creation, and updating a pipeline URL to a newer dotnet-install.ps1 script location.

Reviewed Changes

Copilot reviewed 9 out of 10 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
ColumnEncryptionCertificateFixture.cs	Updated preprocessor condition to reflect newer target frameworks
CertificateFixtureBase.cs	Replaced NET9_0 with NET9_0_OR_GREATER and added retry logic with delay and diagnostic output for certificate generation
CertificateTestWithTdsServer.cs	Updated preprocessor condition for certificate loading
CertificateUtility.cs	Updated preprocessor condition for certificate loading and RSA decryption/signature verification
SqlDbTypeExtensions.cs	Updated preprocessor condition for exposing the Json SqlDbType constant
VirtualSecureModeEnclaveProvider.cs	Updated preprocessor condition for loading a certificate from binary payload
SNICommon.cs	Updated preprocessor condition for validating SSL server certificates
ci-run-tests-job.yml	Updated the URL to the latest dotnet-install.ps1 script

Files not reviewed (1)

• src/Microsoft.Data.SqlClient/netfx/src/Microsoft.Data.SqlClient.csproj: Language not supported

Comments suppressed due to low confidence (1)

src/Microsoft.Data.SqlClient/tests/tools/Microsoft.Data.SqlClient.TestUtilities/Fixtures/CertificateFixtureBase.cs:173

• Consider adding tests to verify the retry logic and ensure correct behavior on repeated PowerShell failures.

for (int attempt = 1; attempt <= retries; ++attempt)

[paulmedynski]

Ready for review, with my comments added for context.

[paulmedynski]

This avoids Invoke-WebRequest failures due to some dead IPs in the DNS results for dot.net, and is the proper canonical DNS name for the dotnet scripts.

[paulmedynski]

Now we're future-proofed!

The docs for the preprocessor symbols were missing NET9_0_OR_GREATER, so I notified the owners and they will be updated accordingly. The symbol itself has been present since .NET 9.0 release though.

[paulmedynski]

We don't need to explicitly define NETFRAMEWORK. It has been a built-in symbol for ages.

[paulmedynski]

These seem like reasonable values. I observed 2 retries across all of the jobs, so it's not adding significant delay.