Shane Viktor ML-DSA/KEM merge #26715

vdukhovni · 2025-02-12T04:33:04Z

This is a rebase of the ML-DSA and then ML-KEM feature branches onto master.
Performed jointly by Viktor and Shane.
The only material conflicts that were judgement calls are in the TLS-related docs, please read text of the changes.

Checklist

documentation is added or updated
tests are added or updated

The key generation algorithm requires a significant portion of the many algorithms present in FIPS 204. This work is derived from the BoringSSL code located at https://boringssl.googlesource.com/boringssl/+/refs/heads/master/crypto/mldsa/mldsa.cc Instead of c++ templates it uses an ML_DSA_PARAMS object to store constants such as k & l. To perform hash operations a temporary EVP_MD_CTX object is used, which is supplied with a prefetched EVP_MD shake128 or shake256 object that reside in the ML_DSA_KEY object. The ML_DSA_KEY object stores the encoded public and/or private key whenever a key is loaded or generated. A public key is always present if the private key component exists. Reviewed-by: Viktor Dukhovni <[email protected]> Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Matt Caswell <[email protected]> (Merged from openssl#26127)

… Matrix. A DSA_KEY when created will alloc enough space to hold its k & l vectors and then just set the vectors to point to the allocated blob. Local Vectors and Matricies can then be initialised in a similar way by passing them an array of Polnomials that are on the local stack. Reviewed-by: Viktor Dukhovni <[email protected]> Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Matt Caswell <[email protected]> (Merged from openssl#26127)

Reviewed-by: Viktor Dukhovni <[email protected]> Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Matt Caswell <[email protected]> (Merged from openssl#26127)

- Make data encoding work on big-endian systems. - Fix some ML-DSA-44 specific bugs related to w1-vector bits per-coefficient, overall size and high-bits rounding. - Use "do { ... } while (pointer < end)" style consistently. - Drop redundant reference counting of provided keys. - Add parameter blocks for ML-DSA-44 and ML-DSA-87 and turn on associated provider glue. These now pass both keygen and siggen tests (to be added separately). Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Matt Caswell <[email protected]> (Merged from openssl#26127)

Reviewed-by: Viktor Dukhovni <[email protected]> Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Matt Caswell <[email protected]> (Merged from openssl#26127)

Reviewed-by: Viktor Dukhovni <[email protected]> Reviewed-by: Paul Dale <[email protected]> (Merged from openssl#26451)

The encoded hint data consists of omega + k bytes. The bytes at the end of omega section of the buffer may be 0, so the buffer must be cleared initially. Reviewed-by: Viktor Dukhovni <[email protected]> Reviewed-by: Paul Dale <[email protected]> (Merged from openssl#26451)

Reviewed-by: Viktor Dukhovni <[email protected]> Reviewed-by: Paul Dale <[email protected]> (Merged from openssl#26451)

The evp_test line buffer was increased to 32K to deal with the large lines required for PQ messages and signatures. The test data files were generated by parsing AVCP test files using a python script. Reviewed-by: Viktor Dukhovni <[email protected]> Reviewed-by: Paul Dale <[email protected]> (Merged from openssl#26451)

Reviewed-by: Viktor Dukhovni <[email protected]> Reviewed-by: Paul Dale <[email protected]> (Merged from openssl#26451)

Reviewed-by: Viktor Dukhovni <[email protected]> Reviewed-by: Tim Hudson <[email protected]> (Merged from openssl#26483)

Reviewed-by: Viktor Dukhovni <[email protected]> Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Matt Caswell <[email protected]> (Merged from openssl#26400)

Also remove some ACVP test data from ml_dsa.inc since this is now also done using evp_test. Reviewed-by: Paul Dale <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#26505)

Reviewed-by: Paul Dale <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#26505)

Using param builder consumes more resources and it is only beneficial when dealing with bignums. Directly using the param helpers is a better alternative. Reviewed-by: Matt Caswell <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#26529)

Reviewed-by: Viktor Dukhovni <[email protected]> Reviewed-by: Paul Dale <[email protected]> Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#26575)

branch. Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> Reviewed-by: Viktor Dukhovni <[email protected]> (Merged from openssl#26575)

commandline. In order to support this gettables are required in both the key and signature.: Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> Reviewed-by: Viktor Dukhovni <[email protected]> (Merged from openssl#26575)

Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> Reviewed-by: Viktor Dukhovni <[email protected]> (Merged from openssl#26575)

Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Shane Lontis <[email protected]> Reviewed-by: Viktor Dukhovni <[email protected]> (Merged from openssl#26548)

The ossl_ml_dsa_key_get0_libctx() and the various size macros are better in the intneral header Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Shane Lontis <[email protected]> Reviewed-by: Viktor Dukhovni <[email protected]> (Merged from openssl#26548)

Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Shane Lontis <[email protected]> Reviewed-by: Viktor Dukhovni <[email protected]> (Merged from openssl#26548)

t8m · 2025-02-13T09:48:49Z

I'll do the merge tomorrow morning after 24hrs pass.

vdukhovni

Approving additional commits from @t8m.

paulidale

Approving the parts I didn't do.

Reviewed-by: Tomas Mraz <[email protected]> Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Neil Horman <[email protected]> (Merged from #26715)

Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Neil Horman <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from #26715)

This has regressed with #24799 The test configs have to be generated differently based on the fips provider version. Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Neil Horman <[email protected]> (Merged from #26715)

Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Neil Horman <[email protected]> (Merged from #26715)

t8m · 2025-02-14T09:57:08Z

This is now merged!

Reviewed-by: Tomas Mraz <[email protected]> Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Neil Horman <[email protected]> (Merged from openssl#26715)

Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Neil Horman <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#26715)

This has regressed with openssl#24799 The test configs have to be generated differently based on the fips provider version. Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Neil Horman <[email protected]> (Merged from openssl#26715)

Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Neil Horman <[email protected]> (Merged from openssl#26715)

Reviewed-by: Tomas Mraz <[email protected]> Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Neil Horman <[email protected]> (Merged from openssl#26715)

Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Neil Horman <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#26715)

This has regressed with openssl#24799 The test configs have to be generated differently based on the fips provider version. Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Neil Horman <[email protected]> (Merged from openssl#26715)

Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Neil Horman <[email protected]> (Merged from openssl#26715)

slontis and others added 30 commits February 12, 2025 13:03

Add ML-DSA sign/verify

747082c

Reviewed-by: Viktor Dukhovni <[email protected]> Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Matt Caswell <[email protected]> (Merged from openssl#26127)

ML-DSA fixups

90148e9

Reviewed-by: Viktor Dukhovni <[email protected]> Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Matt Caswell <[email protected]> (Merged from openssl#26127)

Add ML_DSA encoders

63790fb

Reviewed-by: Viktor Dukhovni <[email protected]> Reviewed-by: Paul Dale <[email protected]> (Merged from openssl#26451)

ML-DSA: Add support for dup.

d65e0e7

Reviewed-by: Viktor Dukhovni <[email protected]> Reviewed-by: Paul Dale <[email protected]> (Merged from openssl#26451)

ML-DSA: Fix endian issue in ossl_ml_dsa_poly_sample_in_ball().

4a2b2ad

Reviewed-by: Viktor Dukhovni <[email protected]> Reviewed-by: Paul Dale <[email protected]> (Merged from openssl#26451)

ML-DSA: Add python utility used to parse ACVP test data.

89de39e

Reviewed-by: Viktor Dukhovni <[email protected]> Reviewed-by: Paul Dale <[email protected]> (Merged from openssl#26451)

ML-DSA encoder fixups

ab9343c

Reviewed-by: Viktor Dukhovni <[email protected]> Reviewed-by: Tim Hudson <[email protected]> (Merged from openssl#26483)

Add ML-DSA documentation

3d2be19

Reviewed-by: Viktor Dukhovni <[email protected]> Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Matt Caswell <[email protected]> (Merged from openssl#26400)

Add ML-DSA design document.

373a740

Reviewed-by: Viktor Dukhovni <[email protected]> Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Matt Caswell <[email protected]> (Merged from openssl#26400)

ML-DSA Add coverage tests

1bff2c3

Also remove some ACVP test data from ml_dsa.inc since this is now also done using evp_test. Reviewed-by: Paul Dale <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#26505)

ML-DSA: coverage testing fixups

77d6169

Reviewed-by: Paul Dale <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#26505)

Add ML_DSA encoders

f47c333

Reviewed-by: Viktor Dukhovni <[email protected]> Reviewed-by: Paul Dale <[email protected]> Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#26575)

ML-DSA Updates to encoders required because of changes on the master

98de329

branch. Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> Reviewed-by: Viktor Dukhovni <[email protected]> (Merged from openssl#26575)

ML-DSA Add digestsign tests - The digest must be NULL

d83e7be

Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> Reviewed-by: Viktor Dukhovni <[email protected]> (Merged from openssl#26575)

Add ML-DSA to the FIPS provider

bf2fc74

Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Shane Lontis <[email protected]> Reviewed-by: Viktor Dukhovni <[email protected]> (Merged from openssl#26548)

build: fix copy/paste mistake

c69dfca

Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Shane Lontis <[email protected]> Reviewed-by: Viktor Dukhovni <[email protected]> (Merged from openssl#26548)

fips: build ML-DSA for FIPS provider

a2d3dbf

Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Shane Lontis <[email protected]> Reviewed-by: Viktor Dukhovni <[email protected]> (Merged from openssl#26548)

fips: build ML-DSA for FIPS provider

14dbb8b

Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Shane Lontis <[email protected]> Reviewed-by: Viktor Dukhovni <[email protected]> (Merged from openssl#26548)

ml-dsa: add to FIPS provider

0b08699

Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Shane Lontis <[email protected]> Reviewed-by: Viktor Dukhovni <[email protected]> (Merged from openssl#26548)

ml-dsa: add PCT for FIPS provider

665f3f9

Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Shane Lontis <[email protected]> Reviewed-by: Viktor Dukhovni <[email protected]> (Merged from openssl#26548)

test: run ML-DSA tests on FIPS provider

9616986

Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Shane Lontis <[email protected]> Reviewed-by: Viktor Dukhovni <[email protected]> (Merged from openssl#26548)

ml-dsa: add FIPS self test macro

79eeafc

Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Shane Lontis <[email protected]> Reviewed-by: Viktor Dukhovni <[email protected]> (Merged from openssl#26548)

t8m added triaged: feature The issue/pr requests/adds a feature tests: present The PR has suitable tests present labels Feb 13, 2025

t-j-h added this to the 3.5.0 Feature Complete milestone Feb 13, 2025

t-j-h assigned vdukhovni Feb 13, 2025

This was linked to issues Feb 13, 2025

Merge ML-KEM feature branch openssl/project#943

Closed

Merge ML-DSA feature branch openssl/project#965

Closed

t8m assigned t8m and unassigned vdukhovni Feb 13, 2025

vdukhovni commented Feb 13, 2025

View reviewed changes

baentsch mentioned this pull request Feb 13, 2025

ML-DSA compatibility between OQS and ML-DSA openssl/project#1076

Closed

nhorman approved these changes Feb 13, 2025

View reviewed changes

paulidale approved these changes Feb 13, 2025

View reviewed changes

openssl-machine pushed a commit that referenced this pull request Feb 14, 2025

Post-merge make update

605b82d

Reviewed-by: Tomas Mraz <[email protected]> Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Neil Horman <[email protected]> (Merged from #26715)

openssl-machine pushed a commit that referenced this pull request Feb 14, 2025

Post-merge make update

cedc5bc

Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Neil Horman <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from #26715)

openssl-machine pushed a commit that referenced this pull request Feb 14, 2025

Left over doc TODOs

6ab8772

Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Neil Horman <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from #26715)

openssl-machine pushed a commit that referenced this pull request Feb 14, 2025

ssl_test.c: Allow using DHE with TLS-1.2 and older versions

0575755

Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Neil Horman <[email protected]> (Merged from #26715)

t8m closed this Feb 14, 2025

definability pushed a commit to definability/openssl that referenced this pull request Feb 25, 2025

Post-merge make update

c42d2f1

Reviewed-by: Tomas Mraz <[email protected]> Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Neil Horman <[email protected]> (Merged from openssl#26715)

definability pushed a commit to definability/openssl that referenced this pull request Feb 25, 2025

Post-merge make update

1f4d9e5

Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Neil Horman <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#26715)

definability pushed a commit to definability/openssl that referenced this pull request Feb 25, 2025

Left over doc TODOs

f8f929e

Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Neil Horman <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#26715)

definability pushed a commit to definability/openssl that referenced this pull request Feb 25, 2025

ssl_test.c: Allow using DHE with TLS-1.2 and older versions

5f9ee6b

Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Neil Horman <[email protected]> (Merged from openssl#26715)

quarckster pushed a commit to quarckster/openssl-fork that referenced this pull request Feb 27, 2025

Post-merge make update

efb3609

Reviewed-by: Tomas Mraz <[email protected]> Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Neil Horman <[email protected]> (Merged from openssl#26715)

quarckster pushed a commit to quarckster/openssl-fork that referenced this pull request Feb 27, 2025

Post-merge make update

f6ebcb6

Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Neil Horman <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#26715)

quarckster pushed a commit to quarckster/openssl-fork that referenced this pull request Feb 27, 2025

Left over doc TODOs

78bc6d7

Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Neil Horman <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#26715)

quarckster pushed a commit to quarckster/openssl-fork that referenced this pull request Feb 27, 2025

ssl_test.c: Allow using DHE with TLS-1.2 and older versions

38f7161

Reviewed-by: Tim Hudson <[email protected]> Reviewed-by: Neil Horman <[email protected]> (Merged from openssl#26715)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Shane Viktor ML-DSA/KEM merge #26715

Shane Viktor ML-DSA/KEM merge #26715

vdukhovni commented Feb 12, 2025

t8m commented Feb 13, 2025

vdukhovni left a comment

paulidale left a comment

t8m commented Feb 14, 2025

Shane Viktor ML-DSA/KEM merge #26715

Shane Viktor ML-DSA/KEM merge #26715

Conversation

vdukhovni commented Feb 12, 2025

Checklist

t8m commented Feb 13, 2025

vdukhovni left a comment

Choose a reason for hiding this comment

paulidale left a comment

Choose a reason for hiding this comment

t8m commented Feb 14, 2025