Add FIPS build tooling #17907

beanuwave · 2025-04-11T16:49:36Z

Description

Provides the additional build tooling to test & build OS in FIPS JVM with -Pcrypto.standard=FIPS-140-3or the (deprecated) -Ptests.fips.enabled=true parameter. Code changes required to run successfully in FIPS mode are not included in this commit, they will be addressed in separate upcoming PRs.

Note:
The codecov report may appear inaccurate. Tests for the new Randomness logic exist but are only executed when the build is run with the new FIPS parameters enabled.

Related Issues

RFC

Check List

Functionality includes testing.
API changes companion pull request created, if applicable.
Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2025-04-11T16:56:00Z

❌ Gradle check result for 0bd8f13: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-04-28T15:00:34Z

❌ Gradle check result for e128e8a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-04-28T16:28:30Z

❌ Gradle check result for 3aae4ce: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-04-28T18:29:07Z

❌ Gradle check result for abfb83a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-04-29T08:34:39Z

✅ Gradle check result for 11cf500: SUCCESS

codecov · 2025-04-29T08:35:06Z

Codecov Report

Attention: Patch coverage is 55.55556% with 28 lines in your changes missing coverage. Please review.

Project coverage is 72.70%. Comparing base (d52cefa) to head (278f287).
Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
...rc/main/java/org/opensearch/common/Randomness.java	34.78%	14 Missing and 1 partial ⚠️
.../main/java/org/opensearch/bootstrap/Bootstrap.java	0.00%	4 Missing ⚠️
...rg/opensearch/gradle/OpenSearchTestBasePlugin.java	0.00%	3 Missing ⚠️
...opensearch/gradle/testclusters/OpenSearchNode.java	0.00%	2 Missing ⚠️
.../main/java/org/opensearch/common/ssl/PemUtils.java	80.00%	0 Missing and 2 partials ⚠️
.../opensearch/gradle/info/GlobalBuildInfoPlugin.java	66.66%	1 Missing ⚠️
.../opensearch/bootstrap/SecurityProviderManager.java	88.88%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #17907      +/-   ##
============================================
- Coverage     72.74%   72.70%   -0.04%     
+ Complexity    67767    67722      -45     
============================================
  Files          5497     5499       +2     
  Lines        311815   311864      +49     
  Branches      45261    45271      +10     
============================================
- Hits         226822   226733      -89     
- Misses        66504    66618     +114     
- Partials      18489    18513      +24

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

github-actions · 2025-05-06T12:11:29Z

❌ Gradle check result for 81729ad: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-05-06T13:25:25Z

❌ Gradle check result for c3cd834: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-05-06T14:46:48Z

✅ Gradle check result for df45869: SUCCESS

beanuwave · 2025-05-07T16:34:04Z

To include FIPS test coverage in the build, an additional test suite is required. Thankfully, @cwperks has already provided a skeleton for it - which we’ll extend and integrate shortly.

buildSrc/src/main/java/org/opensearch/gradle/OpenSearchTestBasePlugin.java

buildSrc/src/main/java/org/opensearch/gradle/info/FipsBuildParams.java

client/rest/src/test/java/org/opensearch/client/RestClientBuilderIntegTests.java

distribution/build.gradle

...ools/plugin-cli/src/test/java/org/opensearch/tools/cli/plugin/InstallPluginCommandTests.java

modules/opensearch-dashboards/build.gradle

server/src/main/java/org/opensearch/bootstrap/SecurityProviderManager.java

prakashsatpathy · 2025-05-12T18:17:16Z

Hi @beanuwave ,
As mentioned in the PR description, there are still code change required to run in FIPS enable mode.
I believe the code changes required are in this PR, #14912.
Can you please correct me on this?
For FIPS mode, we need to convert the keystore and truststore to BCFKS. And all other changes are there.
And is there any plan to progress the 14912 PR?

beanuwave · 2025-05-13T16:35:49Z

Hi @beanuwave , As mentioned in the PR description, there are still code change required to run in FIPS enable mode. I believe the code changes required are in this PR, #14912. Can you please correct me on this? For FIPS mode, we need to convert the keystore and truststore to BCFKS. And all other changes are there. And is there any plan to progress the 14912 PR?

Hi @prakashsatpathy you are absolutely right - the changes from this PR need to be merged into #14912 . Although it's stale and won't be reviewed anymore, we’re keeping it open as a backreference and to serve as a code diff for upcoming PRs.

github-actions · 2025-05-26T19:58:13Z

❌ Gradle check result for 634cc83: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-05-27T08:26:11Z

✅ Gradle check result for 1b16b34: SUCCESS

beanuwave · 2025-05-27T09:00:08Z

Thank you @beanuwave , a few minor cleanups and we should be good to go!

@reta Sounds good! I've addressed two of your comments in the latest change — please take a look.

github-actions · 2025-05-28T09:06:35Z

✅ Gradle check result for 3a74481: SUCCESS

test/framework/src/main/java/org/opensearch/bootstrap/BootstrapForTesting.java

server/src/main/java/org/opensearch/bootstrap/SecurityProviderManager.java

Signed-off-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]> # Conflicts: # CHANGELOG.md # Conflicts: # CHANGELOG.md

…ls/cli/plugin/InstallPluginCommandTests.java Co-authored-by: Andriy Redko <[email protected]> Signed-off-by: Iwan Igonin <[email protected]>

…Manager.java Co-authored-by: Andriy Redko <[email protected]> Signed-off-by: Iwan Igonin <[email protected]>

…SSWORD when running docker tests. Signed-off-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]>

Signed-off-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]>

Signed-off-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]> # Conflicts: # server/src/main/java/org/opensearch/common/Randomness.java

github-actions · 2025-06-02T09:22:21Z

❌ Gradle check result for 36da087: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-06-02T10:11:08Z

❌ Gradle check result for 9452133: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

…er#removeNonCompliantFipsProviders Signed-off-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]>

github-actions · 2025-06-02T11:37:21Z

✅ Gradle check result for 278f287: SUCCESS

beanuwave force-pushed the fips_build_tooling branch from 0bd8f13 to e128e8a Compare April 28, 2025 14:48

beanuwave force-pushed the fips_build_tooling branch 2 times, most recently from cabd094 to 3aae4ce Compare April 28, 2025 16:03

beanuwave force-pushed the fips_build_tooling branch from 3aae4ce to abfb83a Compare April 28, 2025 17:48

opensearch-ci-bot mentioned this pull request Apr 28, 2025

[AUTOCUT] Gradle Check Flaky Test Report for ReactorNetty4StreamingStressIT #15840

Closed

beanuwave force-pushed the fips_build_tooling branch from abfb83a to 11cf500 Compare April 29, 2025 07:31

beanuwave force-pushed the fips_build_tooling branch from 11cf500 to 81729ad Compare May 6, 2025 11:58

beanuwave force-pushed the fips_build_tooling branch from 81729ad to c3cd834 Compare May 6, 2025 13:11

beanuwave force-pushed the fips_build_tooling branch from c3cd834 to df45869 Compare May 6, 2025 13:37

beanuwave marked this pull request as ready for review May 7, 2025 16:16

beanuwave requested a review from a team as a code owner May 7, 2025 16:16

reta reviewed May 7, 2025

View reviewed changes

buildSrc/src/main/java/org/opensearch/gradle/OpenSearchTestBasePlugin.java Show resolved Hide resolved

reta reviewed May 7, 2025

View reviewed changes

buildSrc/src/main/java/org/opensearch/gradle/info/FipsBuildParams.java Show resolved Hide resolved

reta reviewed May 7, 2025

View reviewed changes

client/rest/src/test/java/org/opensearch/client/RestClientBuilderIntegTests.java Outdated Show resolved Hide resolved

reta reviewed May 8, 2025

View reviewed changes

distribution/build.gradle Outdated Show resolved Hide resolved

reta reviewed May 8, 2025

View reviewed changes

...ools/plugin-cli/src/test/java/org/opensearch/tools/cli/plugin/InstallPluginCommandTests.java Outdated Show resolved Hide resolved

reta reviewed May 8, 2025

View reviewed changes

modules/opensearch-dashboards/build.gradle Outdated Show resolved Hide resolved

reta reviewed May 8, 2025

View reviewed changes

server/src/main/java/org/opensearch/bootstrap/SecurityProviderManager.java Outdated Show resolved Hide resolved

beanuwave force-pushed the fips_build_tooling branch from 634cc83 to 1b16b34 Compare May 27, 2025 07:19

beanuwave force-pushed the fips_build_tooling branch from 1b16b34 to 3a74481 Compare May 28, 2025 08:00

beanuwave mentioned this pull request May 28, 2025

Fips smoke test #18376

Draft

3 tasks

opensearch-ci-bot mentioned this pull request May 28, 2025

[AUTOCUT] Gradle Check Flaky Test Report for SearchIT #18129

Closed

reta reviewed May 31, 2025

View reviewed changes

test/framework/src/main/java/org/opensearch/bootstrap/BootstrapForTesting.java Outdated Show resolved Hide resolved

reta reviewed May 31, 2025

View reviewed changes

server/src/main/java/org/opensearch/bootstrap/SecurityProviderManager.java Outdated Show resolved Hide resolved

iigonin and others added 11 commits June 2, 2025 10:47

Add FIPS build-tooling

d8c02ee

Signed-off-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]> # Conflicts: # CHANGELOG.md # Conflicts: # CHANGELOG.md

Update distribution/tools/plugin-cli/src/test/java/org/opensearch/too…

5986e59

…ls/cli/plugin/InstallPluginCommandTests.java Co-authored-by: Andriy Redko <[email protected]> Signed-off-by: Iwan Igonin <[email protected]>

Update server/src/main/java/org/opensearch/bootstrap/SecurityProvider…

2b3e41e

…Manager.java Co-authored-by: Andriy Redko <[email protected]> Signed-off-by: Iwan Igonin <[email protected]>

prevent bc deps propagation outside their modules

48fe437

Signed-off-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]>

change TestUtilsTests to test for cryptographic equality

0fca0d6

Signed-off-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]>

use 'fipsOnly' for all BC declarations

cfc239a

Signed-off-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]>

fix: do not allow other values than 'FIPS-140-3' for 'crypto.standard'

01ad29c

Signed-off-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]>

exclude BC from client & server compile scope

8b26c3e

Signed-off-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]>

Refactor BootstrapForTesting & Randomness

82f6024

Signed-off-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]>

beanuwave force-pushed the fips_build_tooling branch from 3a74481 to 36da087 Compare June 2, 2025 08:48

beanuwave force-pushed the fips_build_tooling branch from 36da087 to 9452133 Compare June 2, 2025 09:43

beanuwave force-pushed the fips_build_tooling branch from 9452133 to 278f287 Compare June 2, 2025 10:32

reta approved these changes Jun 2, 2025

View reviewed changes

reta merged commit 32cd2e6 into opensearch-project:main Jun 2, 2025
29 of 30 checks passed

Add FIPS build tooling #17907

Add FIPS build tooling #17907

Uh oh!

Conversation

beanuwave commented Apr 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Check List

Uh oh!

github-actions bot commented Apr 11, 2025

Uh oh!

github-actions bot commented Apr 28, 2025

Uh oh!

github-actions bot commented Apr 28, 2025

Uh oh!

github-actions bot commented Apr 28, 2025

Uh oh!

github-actions bot commented Apr 29, 2025

Uh oh!

codecov bot commented Apr 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions bot commented May 6, 2025

Uh oh!

github-actions bot commented May 6, 2025

Uh oh!

github-actions bot commented May 6, 2025

Uh oh!

beanuwave commented May 7, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

prakashsatpathy commented May 12, 2025

Uh oh!

beanuwave commented May 13, 2025

Uh oh!

github-actions bot commented May 26, 2025

Uh oh!

github-actions bot commented May 27, 2025

Uh oh!

beanuwave commented May 27, 2025

Uh oh!

github-actions bot commented May 28, 2025

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Jun 2, 2025

Uh oh!

github-actions bot commented Jun 2, 2025

Uh oh!

github-actions bot commented Jun 2, 2025

Uh oh!

Uh oh!

Uh oh!

beanuwave commented Apr 11, 2025 •

edited

Loading

codecov bot commented Apr 29, 2025 •

edited

Loading