Add blake3 as a registered/supported hash algorithm.

[rchincha]

Motivations for this PR:

1. Blake3 is a high performance hash and there is growing community interest
2. Blake3 is variable output, but mandate 256-bit output

[sudo-bmitch]

We should specify that implementations "MAY" support the algorithm, and specify the encoded value regexp, similar to the sha512 definition.

[tianon]

A minor typo fix (wrong number of bits in the regex 🙈), a little whitespace pedanticism (that I'm hoping @sudo-bmitch will confirm or reject/deny), and what can probably/hopefully just be a discussion of the URL to link to (not necessarily requesting any change there).

Overall the change looks good and I'm +1; thanks for taking a stab!

[sudo-bmitch]

Minor white-space nit to align paragraphs with other registered algorithms. Otherwise LGTM.

[wking]

In addition to the new section, you'll want a new entry in the registered-algorithms table up around line 140, right?

[rchincha]

Updated

[wking]

Spinning out of this earlier thread:

Linking to the https://github.com/BLAKE3-team/BLAKE3-specs or https://github.com/C2SP/C2SP/blob/main/BLAKE3.md would each make sense to me.

I'm not an approver, so feel free to ignore me, but personally, having a versioned spec that links to a floating document makes me a bit concerned about image-spec v1.2.0 (or whatever image-spec release eventually ships this registration) being a moving target. Can we pin a specific version of the BLAKE3 spec, e.g. via their BLAKE3/v1.0.0 tag with https://github.com/C2SP/C2SP/blob/BLAKE3/v1.0.0/BLAKE3.md ? It currently matches their main content for that file, and if they tweak the file in the future, e.g. with a BLAKE3/v1.1.0, or dev work in preparation for such a release, it wouldn't get retroactively sucked into image-spec v1.2.0, and would take explicit decisions by image-spec maintainers and implementors to pull in whatever the new BLAKE3 changes were.

[rchincha]

Updated

[wking]

I'm not familiar with BLAKE3, but going through the spec, it seems like we might also need to specify the hash hashing mode? Or is it sufficiently obvious from the context that there's no provision for supplying a key or other input, making the keyed_hash and derive_key modes unfeasible?

[tianon]

IMO it's pretty obvious that this is and has to be unkeyed (as you said, there's no way to specify a key).

[rchincha]

Similar to sha1 or sha2 right?

The purpose of the hash here is content-addressability (with non-collision guarantees)

You can still build a keyed hash out of these - https://datatracker.ietf.org/doc/html/rfc2104

[sajayantony]

Is there an RFC we can point to? Also I'm not sure if but can an expert here articulate the difference between https://www.ietf.org/archive/id/draft-aumasson-blake3-00.html and a GitHub link above.

[rchincha]

As per the commentary, RFC was submitted for process sake, but deemed ineffective/unnecessary. So this repo may be source of truth afaik.

[tianon]

@xnox noted in opencontainers/distribution-spec#574 (comment):

Thus one can use Blake3 hash for hash addressed content - and effectively just a steaming CRC. But cryptographic verification would have to use a different hash for signing.

This is a really interesting point and tangentially I think is grounds to consider reverting this PR (unless this was meant specifically in the context of FIPS, not in general, which is the sense I got in my own research on the topic and does seem consistent with the rest of the comment).

[xnox]

@xnox noted in opencontainers/distribution-spec#574 (comment):

Thus one can use Blake3 hash for hash addressed content - and effectively just a steaming CRC. But cryptographic verification would have to use a different hash for signing.

This is a really interesting point and tangentially I think is grounds to consider reverting this PR (unless this was meant specifically in the context of FIPS, not in general, which is the sense I got in my own research on the topic and does seem consistent with the rest of the comment).

Unfortunately Blake2 lost the FIPS SHA-3 contest, Blake3 is much better, but is unlikely to ever have wide adoption in cryptographic libraries and verification. Xxhash on the other hand is a clear winner otherwise without need for cryptographic protection.

The synergies for cryptographic purposes are around SHA-2 today and Shake is the future (due to sensible security properties, and flexibility w.r.t. customized, authenticated, ordered, and parallel hashing properties). And xxhash for non cryptographic hashes.

My comments were general and not scoped specifically to FIPS context. If you look at the CA/Browser forum they too are not enthusiastic about adding choices.

Sometimes less is more.

[rchincha]

For our use case, we want - 1) content-addressability with guarantees, and 2) very fast

Xxhash on the other hand is a clear winner otherwise without need for cryptographic protection.

Perhaps faster than murmurhash but gives us only 2)

Shake is the future

https://en.wikipedia.org/wiki/SHA-3
Keccak family which gives us 1) but 2) needs h/w acceleration.
We already do SHA2 family so why bother.

Blake3 gives us both without anything special, which was the rationale.

From just compute pov, folks consume container images on almighty cloud and wimpy embedded devices (IoT).

[xnox]

For our use case, we want - 1) content-addressability with guarantees, and 2) very fast

Xxhash on the other hand is a clear winner otherwise without need for cryptographic protection.

Perhaps faster than murmurhash but gives us only 2)

Shake is the future

https://en.wikipedia.org/wiki/SHA-3 Keccak family which gives us 1) but 2) needs h/w acceleration. We already do SHA2 family so why bother.

Blake3 gives us both without anything special, which was the rationale.

Whilst excluding 40% of corporate users in USA and Canada and other places, as Blake3 is not FIPS approved. So switching away from SHA2 is always a loss of some market-share and users today.

There is no silver bullet yet. Obviously if one is sure to never use something inside Fedramp or within Common Criteria or within FIPS certification or within Australian/UK cyber security requirements..... then yes blake3 is awesome, as yes whilst slower than xxhash it has collision protection guarantees.

From just compute pov, folks consume container images on almighty cloud and wimpy embedded devices (IoT).

[rchincha]

FIPS requirements pop up when dealing with govt entities. If so, by all means stay with SHA2.
For the rest of us, we need content-addressability and fast.

https://github.com/BLAKE3-team/BLAKE3

git (content-addressability) still defaults to sha1.

--object-format=<format>
Specify the given object <format> (hash algorithm) for the repository. The valid values are sha1 and (if enabled) sha256. sha1 is the default.

In FIPS world, you get a pass until 2030 or maybe longer if sha1 not for security use case.
https://www.nist.gov/news-events/news/2022/12/nist-retires-sha-1-cryptographic-algorithm
https://docs.gitlab.com/development/fips_gitlab/#development-guidelines

[rchincha]

Will be remiss if the following is not mentioned.

https://nvlpubs.nist.gov/nistpubs/specialpublications/nist.sp.800-185.pdf
https://downloads.bouncycastle.org/java/docs/bcprov-jdk14-javadoc/org/bouncycastle/crypto/digests/ParallelHash.html

Details, implementation and benchmarks are sparse though.