Skip to content

Conversation

igor-sirotin
Copy link
Collaborator

@igor-sirotin igor-sirotin commented Sep 24, 2025

Description

Stop vendoring Go dependencies. Delete vendor/ directory.

Reasoning

As for today, vendoring dependencies in Go is more of a burden than real need.
Major Go projects like go-ethereum don't vendor.

The only reason for vendoring is to have all dependencies on hands in case one of them is suddenly becomes unavailable
(deleted of removed from public). On the other hand, it makes the repo heavier and PRs less clear, which we believe is more important. This becomes more obvious with the usage of go tool (e.g. in PRs like #6933).

nwaku

Currently approach with cloning nwaku repository doesn't work without vendoring, as it requires sudo rights.

This PR is currently based on a branch where nwaku is a submodule, but I am looking into better solutions right now.

Currently the PR uses a libwaku included in the waku-go-bindings repo: waku-org/waku-go-bindings#92

@status-im-auto
Copy link
Member

status-im-auto commented Sep 24, 2025

Jenkins Builds

Click to see older builds (48)
Commit #️⃣ Finished (UTC) Duration Platform Result
772aa31 #1 2025-09-24 18:54:23 ~1 min linux/nwaku 📄log
772aa31 #1 2025-09-24 18:54:37 ~1 min windows/status-go 📄log
✔️ 772aa31 #1 2025-09-24 18:56:36 ~3 min linux/status-go 📦zip
✔️ 772aa31 #1 2025-09-24 18:59:38 ~6 min macos/status-go 📦zip
✔️ 772aa31 #1 2025-09-24 19:04:39 ~11 min tests-rpc 📄log
✔️ 772aa31 #1 2025-09-24 19:19:06 ~25 min tests 📄log
bcc7833 #2 2025-09-25 08:54:24 ~1 min linux/nwaku 📄log
✔️ bcc7833 #2 2025-09-25 08:55:44 ~3 min linux/status-go 📦zip
✔️ bcc7833 #2 2025-09-25 08:56:14 ~3 min macos/status-go 📦zip
✔️ bcc7833 #2 2025-09-25 08:58:20 ~5 min windows/status-go 📦zip
✔️ bcc7833 #2 2025-09-25 09:04:17 ~11 min tests-rpc 📄log
✔️ bcc7833 #2 2025-09-25 09:18:21 ~25 min tests 📄log
✔️ 485d724 #3 2025-09-26 16:06:33 ~3 min linux/status-go 📦zip
✔️ 485d724 #3 2025-09-26 16:06:57 ~3 min macos/status-go 📦zip
✔️ 485d724 #3 2025-09-26 16:09:30 ~5 min windows/status-go 📦zip
✔️ 485d724 #4 2025-09-26 16:10:43 ~3 min macos/status-go 📦zip
✔️ 485d724 #4 2025-09-26 16:11:04 ~4 min linux/status-go 📦zip
✔️ 485d724 #3 2025-09-26 16:13:52 ~10 min linux/nwaku 📦zip
✔️ 485d724 #3 2025-09-26 16:16:58 ~13 min tests-rpc 📄log
✔️ 485d724 #4 2025-09-26 16:24:44 ~10 min linux/nwaku 📦zip
✖️ 485d724 #4 2025-09-26 16:28:00 ~10 min tests-rpc 📄log
✔️ 485d724 #3 2025-09-26 16:31:33 ~27 min tests 📄log
✔️ 485d724 #5 2025-09-26 16:40:21 ~8 min tests-rpc 📄log
✔️ b288939 #5 2025-09-28 18:36:14 ~3 min linux/status-go 📦zip
✔️ b288939 #5 2025-09-28 18:36:29 ~3 min macos/status-go 📦zip
✔️ b288939 #4 2025-09-28 18:39:54 ~7 min windows/status-go 📦zip
✔️ b288939 #6 2025-09-28 18:41:57 ~9 min tests-rpc 📄log
✔️ b288939 #5 2025-09-28 18:43:05 ~10 min linux/nwaku 📦zip
✔️ b288939 #4 2025-09-28 18:58:38 ~25 min tests 📄log
✔️ e59e021 #6 2025-09-28 19:13:28 ~3 min macos/status-go 📦zip
✔️ e59e021 #6 2025-09-28 19:13:47 ~4 min linux/status-go 📦zip
✔️ e59e021 #5 2025-09-28 19:16:27 ~6 min windows/status-go 📦zip
✔️ e59e021 #7 2025-09-28 19:20:53 ~11 min tests-rpc 📄log
✔️ e59e021 #6 2025-09-28 19:21:09 ~11 min linux/nwaku 📦zip
✔️ e59e021 #5 2025-09-28 19:37:29 ~27 min tests 📄log
✔️ dde9a6b #7 2025-09-28 19:55:31 ~3 min linux/status-go 📦zip
✔️ dde9a6b #7 2025-09-28 19:55:46 ~3 min macos/status-go 📦zip
✔️ dde9a6b #6 2025-09-28 19:58:41 ~6 min windows/status-go 📦zip
✔️ dde9a6b #8 2025-09-28 20:01:06 ~9 min tests-rpc 📄log
✔️ dde9a6b #7 2025-09-28 20:02:22 ~10 min linux/nwaku 📦zip
✔️ dde9a6b #6 2025-09-28 20:18:06 ~26 min tests 📄log
✔️ dde9a6b #8 2025-09-29 21:46:59 ~3 min macos/status-go 📦zip
✔️ dde9a6b #8 2025-09-29 21:47:20 ~3 min linux/status-go 📦zip
✔️ dde9a6b #7 2025-09-29 21:50:08 ~6 min windows/status-go 📦zip
✖️ dde9a6b #9 2025-09-29 21:53:24 ~9 min tests-rpc 📄log
✔️ dde9a6b #8 2025-09-29 21:54:22 ~11 min linux/nwaku 📦zip
✖️ dde9a6b #10 2025-09-29 22:07:10 ~7 min tests-rpc 📄log
✔️ dde9a6b #7 2025-09-29 22:09:44 ~26 min tests 📄log
Commit #️⃣ Finished (UTC) Duration Platform Result
✖️ c61255e #8 2025-09-30 13:01:21 ~2 min tests 📄log
c61255e #9 2025-09-30 13:01:54 ~3 min linux/nwaku 📄log
✔️ c61255e #9 2025-09-30 13:02:16 ~3 min linux/status-go 📦zip
✔️ c61255e #9 2025-09-30 13:02:34 ~3 min macos/status-go 📦zip
✔️ c61255e #8 2025-09-30 13:03:55 ~4 min windows/status-go 📦zip
✖️ c61255e #11 2025-09-30 13:08:31 ~9 min tests-rpc 📄log
b4cc062 #10 2025-09-30 13:27:27 ~3 min linux/nwaku 📄log
✔️ b4cc062 #10 2025-09-30 13:27:42 ~3 min linux/status-go 📦zip
✔️ b4cc062 #10 2025-09-30 13:28:07 ~3 min macos/status-go 📦zip
✔️ b4cc062 #9 2025-09-30 13:30:22 ~6 min windows/status-go 📦zip
✔️ b4cc062 #12 2025-09-30 13:33:44 ~9 min tests-rpc 📄log
✔️ b4cc062 #9 2025-09-30 13:50:24 ~26 min tests 📄log

Copy link

codecov bot commented Sep 25, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 59.40%. Comparing base (b000470) to head (b4cc062).

Additional details and impacted files
@@                      Coverage Diff                       @@
##           feat/bump-waku-go-bindings    #6951      +/-   ##
==============================================================
- Coverage                       59.46%   59.40%   -0.06%     
==============================================================
  Files                             810      810              
  Lines                          119476   119476              
==============================================================
- Hits                            71047    70979      -68     
- Misses                          41083    41131      +48     
- Partials                         7346     7366      +20     
Flag Coverage Δ
functional 31.77% <ø> (-0.14%) ⬇️
unit 55.47% <ø> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 32 files with indirect coverage changes

@igor-sirotin
Copy link
Collaborator Author

@status-im/devops guys can you please assist me with -nix builds?
I tried to understand what's going on there, but couldn't get it 🤔

@igor-sirotin igor-sirotin changed the base branch from chore/nwaku-submodule to feat/bump-waku-go-bindings September 29, 2025 21:43
Copy link

Thank you for opening this pull request!

We require pull request titles and commits to follow the Conventional Commits specification and it looks like your PR needs to be adjusted.

Details:

According to the conventional-commits specification, some of the commit messages are not valid.

@jakubgs
Copy link
Member

jakubgs commented Sep 30, 2025

The trick is actually that vendorHash cannot be null if there's no vendor folder:

      # The SRI hash of the vendored dependencies.
      # If `vendorHash` is `null`, no dependencies are fetched and
      # the build relies on the vendor folder within the source.
      vendorHash ? throw (
        if args ? vendorSha256 then
          "buildGoModule: Expect vendorHash instead of vendorSha256"
        else
          "buildGoModule: vendorHash is missing"
      ),

https://github.com/NixOS/nixpkgs/blob/7ff837017c3b82bd3671932599a119d7bc672ff0/pkgs/build-support/go/module.nix#L30-L38

So from now on we need to specify a real hash. Which means it will have to be updated every time dependencies change.

@igor-sirotin
Copy link
Collaborator Author

The trick is actually that vendorHash cannot be null if there's no vendor folder:

Thanks for fixing this.
I was looking into this as well, but couldn't figure it out.

So from now on we need to specify a real hash. Which means it will have to be updated every time dependencies change.

@jakubgs is there any chance we can avoid this? It's gonna be quite an annoying procedure 😄
Perhaps smth like lib.fakeHash, or smth similar possible?

@jakubgs
Copy link
Member

jakubgs commented Sep 30, 2025

I know of no built-in way of doing this. The purpose of fakeHash is to just be a placeholder for when you have no value yet.
https://dmarcoux.com/posts/hash-in-nix-packages/

We'd have to write some kind of script to generate the hash for us, but it would have to be committed anyway by the devs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Blocked
Development

Successfully merging this pull request may close these issues.

3 participants