feat(plugin-npm): add npm provenance support

[GauBen]

What's the problem this PR addresses?

Hi! I added support for provenance to yarn npm publish.

Closes #5430

How did you fix it?

Adapted code from npm to produce a provenance signature in supported CI environment.

Checklist

• I have read the Contributing Guide.

• I have set the packages that need to be released for my changes to be effective.

• I will check that all automated PR checks pass before the PR gets reviewed.

Next steps

• Update https://github.com/npm/documentation/blob/c2efb649816e27d37b37da2b21200e4c9ade0d17/content/packages-and-modules/securing-your-code/generating-provenance-statements.mdx?plain=1#L124

[GauBen]

Confirmed to work for GitHub Actions!

[arcanis]

Thanks @GauBen for looking into this! I'm interested in supporting provenance, however I'm concerned about the number of dependencies this brings in. We try not to rely on the npm libraries, and here we're adding a bunch of them for a small feature.

Is sigstore well-speced enough that we can afford to just call whatever endpoints are needed ourselves? Or is the sdk a black box we can't avoid?

[GauBen]

Hi Arcanis, thanks for taking the time to review this quickly!

I'm mildly annoyed too by the fact that we need 90+ packages to sign a .tgz archive.

After a quick investigation, most of these libraries are pulled by make-fetch-happen.

To continue, I'd suggest opening a PR on tuf-js to remove make-fetch-happen from the dependencies and make it a BYOF (bring your own fetch) library. We can speed up the process by depending on the forked version until it is merged.

Edit: Node maintainer Antoine du Hamel opened theupdateframework/tuf-js#849 two months ago

[GauBen]

Ok I did something interesting:

• Patch tuf-js to use fetch
• Resolve make-fetch-happen to is-obj to remove all the dependency tree

It might not be the best way to do it, but we can continue from here, what do you think?

[abatalov-walletteam]

@GauBen, btw, thank you very much for this brilliant PR.

[GauBen]

I measured the weight of the feature:

• Master: 2,767,079 B
• Without tuf-js patch: 3,747,211 B (+35%)
• With tuf-js patch: 2,998,829 B (+8.4%)

If the hack is ok (or can be improved), we can go with sigstore

Is sigstore well-speced enough that we can afford to just call whatever endpoints are needed ourselves? Or is the sdk a black box we can't avoid?

I don't know, and I'm not really motivated to go through it. However, if @lsd-cat agrees, we can adapt their "webcat" project to fit our needs. Does it support the in-toto statements used by npm provenance?

Update: I'm too tired for that, there are more packages than tuf-js that requires make-fetch-happen in the sigstore chain...

[lsd-cat]

The project is MIT licensed and I'm happy with any use, though it is really rudimentary with the purpose to support just a basic set of Sigstore functionalities. TUF is a bit battle tested, as it was adapted through multiple root updates. Sigstore has just a few signing/hash algorithms tested, but adding support there should be trivial. It expects a bundle, and verifies the signing cert, the SCT, the signature over the artifact and the Rekor promise, but not the inclusion proof if I recall correctly. I'm happy to provide support in case of problems, but I can't promise it. There is no support for in-toto statements. None of them has really any kind of automated testing yet.

It is likely that we will continue the development of WEBCAT in the next months, so if there is community interests in reusing the sigstore/tuf implementations there, perhaps we could collaborate on them? Our main requirements would be to keep them browser native and without runtime dependencies.

[lsd-cat]

A reasonable test would be to run both:

• https://github.com/theupdateframework/tuf-conformance
• https://github.com/sigstore/sigstore-conformance

And see what works and what does not. I did not use tuf-conformance before because the tests check for files, and in the browser environment I'm using a different type of storage, but we can probably make that a build time preference and see if we pass a reasonable amount of tests or what are the issues?

[GauBen]

Thanks a lot for your detailed answer!

Our main requirements would be to keep them browser native and without runtime dependencies.

It would be nice to make it runtime-agnostic (for Node.js consumers too), but that might be out of the scope of your project.

if there is community interests in reusing the sigstore/tuf implementations there, perhaps we could collaborate on them?

I have no idea if there is interest past this PR, IIRC pnpm uses npm to publish, so all JS package managers would use sigstore under the hood. There is nothing the prevents Yarn to revisit the provenance implementation once WEBCAT is ready though.

---

A good night of sleep gave me a less dumb idea, let's try that instead

Edit: Indeed, we have provenance support with only 10 new packages! (down from 90 new packages 👀)

[radist2s]

@GauBen, I don't know about the Yarn command, but for me personally, it's a pretty important functionality 💜

Publishing workspace to NPM with Yarn is a true pleasure - just one line:

yarn workspaces foreach --recursive npm publish

and I wouldn't want to change anything about this elegance, except to add the --provenance option.

I've tried a combination of yarn pack and external npm publish --provenance, but then you have to maintain both .yarnrc and .npmrc, which is just not reasonable in my opinion.

I've been looking forward to the Yarn provenance releases, and it would be great if it worked out!

Thanks for your contribution!

[GauBen]

Thank you @abatalov-walletteam and @radist2s for your kind comments (you seem to be only one person but I don't want te be rude)

Any chance you try this PR on your repo and confirm it works? Run yarn set version from sources --repository https://github.com/GauBen/berry.git --branch feat/provenance to update Yarn to this version

yarn workspaces foreach --recursive npm publish --provenance should work exactly as you expect

Thank you again

---

Edit: confirmed to work on GitHub for the latest commit

[radist2s]

Hey @GauBen, just wanted to confirm — everything works great! ✅

I tested publishing with --provenance to our private registry, and it went smoothly both locally and in CI.
You can see the published package on NPM with the provenance-test tag here:

The corresponding workflow run handled the release.

Thanks again for implementing this — really appreciate it!

[SerenModz21]

Great work you've done so far on this PR! Just thought to ask if support for third-party package publishing tools is planned, or would it only be the CLI flag? For example, publishConfig.provenance inside package.json and/or a provenance config option inside the Yarn config (yarn config set provenance true).

[GauBen]

@SerenModz21 Consider it done!

I added support (and documented):

• NPM_CONFIG_PROVENANCE env var YARN_NPM_CONFIG_PROVENANCE
• "publishConfig": { "provenance": true } in package.json
• npmPublishProvenance in .yarnrc.yml

---

@arcanis I set npmPublishProvenance: true in Yarn's .yarnrc.yml to enable provenance statements for the next release. I can revert it if it's not wanted

https://github.com/yarnpkg/berry/pull/6750/files#diff-88fbe28c4102501b94961511a0d70ff895bf39970b4d3fc11917794a239117c5R20-R21

[SerenModz21]

@SerenModz21 Consider it done!

@GauBen Amazing, thank you!

[arcanis]

That sounds good to me - I'd have liked to have some tests though, as I'm refactoring various parts of Yarn and I'm concerned I could accidentally break this in the future.

Do you think you could look at that as a follow-up? For example by copying a publish test and validating the provenance in the test server if the package name is provenance-required (see here).

[GauBen]

I'll give it a shot, but I'd probably need to update workflow files (to provide id-token: write) which would stop them from running on this PR

[arcanis]

Can't we sign with a bogus certificate, just for testing? (not familiar with sigstore at all!)

In the meantime let's merge this iteration

[aduh95]

Is this the actual recommendation? Shouldn't it be using an env variable instead?

Suggested change

	- run: yarn config set npmAuthToken '${{ secrets.NPM_TOKEN }}'
	- run: yarn publish --provenance --tolerate-republish
	- run: yarn publish --provenance --tolerate-republish
	env:
	YARN_NPM_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

[GauBen]

Indeed, it feels cleaner and more self-contained