Bump Go version to 1.25.1 #10156

ycombinator · 2025-09-25T17:46:23Z

This PR bumps up the Golang version to 1.25.1. It also:

removes the ms_tls13kdf Golang build tag when building in FIPS mode because this tag was only needed with Golang versions 1.24.x.
sets ths GODEBUG=tlsmlkem=0 environment variable when running FIPS140-only unit tests. This prevents errors like so: Failed to connect: crypto/ecdh: use of X25519 is not allowed in FIPS 140-only mode.

elasticmachine · 2025-09-25T17:46:27Z

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

ycombinator · 2025-09-25T19:12:54Z

The fips140=only unit tests are failing like so:

crypto/ecdh: use of X25519 is not allowed in FIPS 140-only mode

These appear to be golang/go#75148, which should be fixable when golang/go#74630 is implemented. However, in order to upgrade to Go 1.25.1 now, we'll need to find a workaround.

dev-tools/mage/otel/deps_test.go

ycombinator · 2025-09-25T22:43:15Z

The fips140=only unit tests are failing like so:
crypto/ecdh: use of X25519 is not allowed in FIPS 140-only mode
These appear to be golang/go#75148, which should be fixable when golang/go#74630 is implemented. However, in order to upgrade to Go 1.25.1 now, we'll need to find a workaround.

These errors are coming from Go downloading dependencies before executing the tests. The errors can be simulated like so:

GODEBUG=fips140=only go mod download -x
# get https://proxy.golang.org/github.com/opencontainers/image-spec/@v/v1.1.1.info
# get https://proxy.golang.org/github.com/opencontainers/image-spec/@v/v1.1.1.info: Get "https://proxy.golang.org/github.com/opencontainers/image-spec/@v/v1.1.1.info": crypto/ecdh: use of X25519 is not allowed in FIPS 140-only mode
...

So we probably just need to download the dependencies explicitly, ensuring that GODEBUG=fips140=only is not set for this step.

ycombinator · 2025-09-26T05:23:10Z

So we probably just need to download the dependencies explicitly, ensuring that GODEBUG=fips140=only is not set for this step.

I've implemented this approach in this PR and it has helped. However, now CI is failing with this odd error which seems unrelated to FIPS in any way.

https://buildkite.com/elastic/elastic-agent/builds/27575#0199842f-d9dd-4481-9d46-37baa5c789b1/155-822

=== FAIL: dev-tools/mage TestGoTest_CaptureOutput/capture_panic (1.61s)
--
  | >> go test: asserts Testing
  | >> ARGS: asserts Command: gotestsum --no-color --junitfile-hide-skipped-tests -f standard-quiet -- -test.run TestGoTest_Helper_WithPanic .
  | 2025/09/26 04:35:17 exec: gotestsum --no-color --junitfile-hide-skipped-tests -f standard-quiet -- -test.run TestGoTest_Helper_WithPanic .
  | exec: gotestsum --no-color --junitfile-hide-skipped-tests -f standard-quiet -- -test.run TestGoTest_Helper_WithPanic .
  | gotest_test.go:120: GoTest output mismatch:
  | want:
  | (?sm:
  | === FAIL: dev-tools/mage TestGoTest_Helper_WithPanic.*
  | panic: Kaputt. \[recovered\].*
  | panic: Kaputt.*
  | )
  |  
  | got:
  | FAIL	github.com/elastic/elastic-agent/dev-tools/mage	0.021s
  |  
  | === Failed
  | === FAIL: dev-tools/mage TestGoTest_Helper_WithPanic (0.00s)
  | panic: Kaputt. [recovered, repanicked]
  |  
  | goroutine 21 [running]:
  | testing.tRunner.func1.2({0xcd39e0, 0xf923f0})
  | /opt/buildkite-agent/.asdf/installs/golang/1.25.1/go/src/testing/testing.go:1872 +0x237
  | testing.tRunner.func1()
  | /opt/buildkite-agent/.asdf/installs/golang/1.25.1/go/src/testing/testing.go:1875 +0x35b
  | panic({0xcd39e0?, 0xf923f0?})
  | /opt/buildkite-agent/.asdf/installs/golang/1.25.1/go/src/runtime/panic.go:783 +0x132
  | github.com/elastic/elastic-agent/dev-tools/mage.TestGoTest_Helper_WithPanic(0xc000103880?)
  | /opt/buildkite-agent/builds/bk-agent-prod-gcp-1758859361500881345/elastic/elastic-agent/dev-tools/mage/gotest_test.go:329 +0x30
  | testing.tRunner(0xc000103880, 0xea4750)
  | /opt/buildkite-agent/.asdf/installs/golang/1.25.1/go/src/testing/testing.go:1934 +0xea
  | created by testing.(*T).Run in goroutine 1
  | /opt/buildkite-agent/.asdf/installs/golang/1.25.1/go/src/testing/testing.go:1997 +0x465
  |  
  | DONE 1 tests, 1 failure in 0.022s
  |  
  | === FAIL: dev-tools/mage TestGoTest_CaptureOutput (26.02s)

ycombinator · 2025-09-26T10:33:21Z

So we probably just need to download the dependencies explicitly, ensuring that GODEBUG=fips140=only is not set for this step.

I've implemented this approach in this PR and it has helped. However, now CI is failing with this odd error which seems unrelated to FIPS in any way.

https://buildkite.com/elastic/elastic-agent/builds/27575#0199842f-d9dd-4481-9d46-37baa5c789b1/155-822

=== FAIL: dev-tools/mage TestGoTest_CaptureOutput/capture_panic (1.61s)
--
  | >> go test: asserts Testing
  | >> ARGS: asserts Command: gotestsum --no-color --junitfile-hide-skipped-tests -f standard-quiet -- -test.run TestGoTest_Helper_WithPanic .
  | 2025/09/26 04:35:17 exec: gotestsum --no-color --junitfile-hide-skipped-tests -f standard-quiet -- -test.run TestGoTest_Helper_WithPanic .
  | exec: gotestsum --no-color --junitfile-hide-skipped-tests -f standard-quiet -- -test.run TestGoTest_Helper_WithPanic .
  | gotest_test.go:120: GoTest output mismatch:
  | want:
  | (?sm:
  | === FAIL: dev-tools/mage TestGoTest_Helper_WithPanic.*
  | panic: Kaputt. \[recovered\].*
  | panic: Kaputt.*
  | )
  |  
  | got:
  | FAIL	github.com/elastic/elastic-agent/dev-tools/mage	0.021s
  |  
  | === Failed
  | === FAIL: dev-tools/mage TestGoTest_Helper_WithPanic (0.00s)
  | panic: Kaputt. [recovered, repanicked]
  |  
  | goroutine 21 [running]:
  | testing.tRunner.func1.2({0xcd39e0, 0xf923f0})
  | /opt/buildkite-agent/.asdf/installs/golang/1.25.1/go/src/testing/testing.go:1872 +0x237
  | testing.tRunner.func1()
  | /opt/buildkite-agent/.asdf/installs/golang/1.25.1/go/src/testing/testing.go:1875 +0x35b
  | panic({0xcd39e0?, 0xf923f0?})
  | /opt/buildkite-agent/.asdf/installs/golang/1.25.1/go/src/runtime/panic.go:783 +0x132
  | github.com/elastic/elastic-agent/dev-tools/mage.TestGoTest_Helper_WithPanic(0xc000103880?)
  | /opt/buildkite-agent/builds/bk-agent-prod-gcp-1758859361500881345/elastic/elastic-agent/dev-tools/mage/gotest_test.go:329 +0x30
  | testing.tRunner(0xc000103880, 0xea4750)
  | /opt/buildkite-agent/.asdf/installs/golang/1.25.1/go/src/testing/testing.go:1934 +0xea
  | created by testing.(*T).Run in goroutine 1
  | /opt/buildkite-agent/.asdf/installs/golang/1.25.1/go/src/testing/testing.go:1997 +0x465
  |  
  | DONE 1 tests, 1 failure in 0.022s
  |  
  | === FAIL: dev-tools/mage TestGoTest_CaptureOutput (26.02s)

Turns out this is a change in behavior in Go 1.25: https://tip.golang.org/doc/go1.25#change-to-unhandled-panic-output. Addressed in 46cc036.

magefile.go

pchila · 2025-09-29T06:06:22Z

Looking at the latest build I see a couple of strange things (maybe some of those were already there and didn't notice until now)

windows 2016 unit tests have some failures which seem to be related to keeping files open/locked during cleanups
for example https://buildkite.com/elastic/elastic-agent/builds/27653#019988ea-47f4-49b3-8bfc-0710be9ad7ee/1293-1296
Locking/keeping files open seem to appear a lot also in Windows Integration tests, for example there are a few failures to remove agent executable in a row:
https://buildkite.com/elastic/elastic-agent/builds/27653#019988ea-f1b2-4aea-86d9-94c432db7258/1025-1918
https://buildkite.com/elastic/elastic-agent/builds/27653#019988ea-f1b2-4aea-86d9-94c432db7258/1025-1965
https://buildkite.com/elastic/elastic-agent/builds/27653#019988ea-f1b2-4aea-86d9-94c432db7258/1025-2042
https://buildkite.com/elastic/elastic-agent/builds/27653#019988ea-f1b2-4aea-86d9-94c432db7258/1025-2094

ycombinator · 2025-09-30T01:36:16Z

Windows build steps are failing in CI on this PR. See a lot of Access is denied errors. 🤔

elastic-sonarqube · 2025-09-30T04:44:16Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

Co-authored-by: Panos Koutsovasilis <[email protected]>

… run

…mmand is run" This reverts commit 079e74c.

elasticmachine · 2025-10-01T14:43:05Z

💔 Build Failed

Buildkite Build
Commit: b6b1a81

Failed CI Steps

History

💔 Build #27773 failed 1ceaae8
💔 Build #27764 failed 4db5a4a
💔 Build #27653 failed 67096f0
💔 Build #27649 failed d52ffe0

cc @ycombinator

mergify · 2025-10-03T14:42:09Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b bump-golang-1.25.1 upstream/bump-golang-1.25.1
git merge upstream/main
git push upstream bump-golang-1.25.1

ycombinator requested review from a team as code owners September 25, 2025 17:46

ycombinator requested review from pkoutsovasilis and pchila September 25, 2025 17:46

ycombinator added Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team backport-active-all Automated backport with mergify to all the active branches labels Sep 25, 2025

mergify bot assigned ycombinator Sep 25, 2025

pkoutsovasilis reviewed Sep 25, 2025

View reviewed changes

dev-tools/mage/otel/deps_test.go Outdated Show resolved Hide resolved

This was referenced Sep 26, 2025

Bump Go version to 1.25.1 elastic/beats#46793

Draft

Bump Go version to 1.25.1 elastic/fleet-server#5562

Merged

ycombinator force-pushed the bump-golang-1.25.1 branch from d8c500d to 0b69f33 Compare September 26, 2025 10:20

pchila reviewed Sep 26, 2025

View reviewed changes

magefile.go Outdated Show resolved Hide resolved

ycombinator force-pushed the bump-golang-1.25.1 branch from 4db5a4a to 1ceaae8 Compare September 30, 2025 01:36

ycombinator and others added 9 commits October 1, 2025 07:35

Bump Go version to 1.25.1

1ed9237

Update CHANGELOG entry

7b02292

Bump the version of golangci-lint

2efec20

Update dev-tools/mage/otel/deps_test.go

83f1171

Co-authored-by: Panos Koutsovasilis <[email protected]>

Remove references to the ms_tls13kdf build tag

b3aa7f9

Use mage target specifically intended for fips140=only unit testing

c3da90a

Download go module dependencies before GODEBUG=fips140=only is set

ba6de0f

Fix target name

dc2c700

[Debugging] Print go test command environment right before command is…

3a0732a

… run

ycombinator added 11 commits October 1, 2025 07:35

Revert "[Debugging] Print go test command environment right before co…

6a697f2

…mmand is run" This reverts commit 079e74c.

Fix regular expression for matching panicking output

c3617ce

Run dependencies first

c9b23d8

Append requirefips build tag for fips140=only unit tests

0540748

Fix policy change handler unit tests

19ad24c

Fix TestDownloadVersion test

d6cfca0

Set GODEBUG=tlsmlkem=0 for test

3111831

No longer necessary

f310eb7

Formatting fixes

0f8fb4b

Remove replace and running mage notice

401c2d8

Remove pre-downloading of go modules

6c9b4c7

ycombinator force-pushed the bump-golang-1.25.1 branch from 1ceaae8 to 52d2f67 Compare October 1, 2025 14:35

Revert unit test pipeline changes

b6b1a81

ycombinator force-pushed the bump-golang-1.25.1 branch from 52d2f67 to b6b1a81 Compare October 1, 2025 14:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Bump Go version to 1.25.1 #10156

Bump Go version to 1.25.1 #10156

Uh oh!

ycombinator commented Sep 25, 2025 •

edited

Loading

Uh oh!

elasticmachine commented Sep 25, 2025

Uh oh!

ycombinator commented Sep 25, 2025 •

edited

Loading

Uh oh!

Uh oh!

ycombinator commented Sep 25, 2025 •

edited

Loading

Uh oh!

ycombinator commented Sep 26, 2025

Uh oh!

ycombinator commented Sep 26, 2025

Uh oh!

Uh oh!

pchila commented Sep 29, 2025 •

edited

Loading

Uh oh!

ycombinator commented Sep 30, 2025

Uh oh!

elastic-sonarqube bot commented Sep 30, 2025

Uh oh!

elasticmachine commented Oct 1, 2025 •

edited

Loading

Uh oh!

mergify bot commented Oct 3, 2025

Uh oh!

Uh oh!

Bump Go version to 1.25.1 #10156

Are you sure you want to change the base?

Bump Go version to 1.25.1 #10156

Uh oh!

Conversation

ycombinator commented Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

elasticmachine commented Sep 25, 2025

Uh oh!

ycombinator commented Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

ycombinator commented Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ycombinator commented Sep 26, 2025

Uh oh!

ycombinator commented Sep 26, 2025

Uh oh!

Uh oh!

pchila commented Sep 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ycombinator commented Sep 30, 2025

Uh oh!

elastic-sonarqube bot commented Sep 30, 2025

Quality Gate passed

Uh oh!

elasticmachine commented Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💔 Build Failed

Failed CI Steps

History

Uh oh!

mergify bot commented Oct 3, 2025

Uh oh!

Uh oh!

ycombinator commented Sep 25, 2025 •

edited

Loading

ycombinator commented Sep 25, 2025 •

edited

Loading

ycombinator commented Sep 25, 2025 •

edited

Loading

pchila commented Sep 29, 2025 •

edited

Loading

elasticmachine commented Oct 1, 2025 •

edited

Loading