Adopt new OSSL_PARAM code generator

[hlandau]

Tasks

Preview Give feedback

1. OSSL_PARAM futures openssl#20940
   approval: ready to merge branch: master triaged: documentation +3
   
2. Determine which algorithms need faster OSSL_PARAM decoding #193
   +0
3. Glue OSSL_PARAM decoding together #192
   +0
4. Analyse the TLS Handshake test to identify algorithms that can benefit from OSSL_PARAM code generator #942
   +0

[hlandau]

After @paulidale's great work on the OSSL_PARAM trie-based code generator we need to make PRs to actually make use of it in all the places we consume OSSL_PARAMs. So far we only use it for one piece of code which was an exemplary usage as part of the original PR.

[levitte]

Side note: I'm thinking of making a mod that places back the symbol information into include/openssl/core_names.h.in. Having that database hidden in the perl module is bad.

[hlandau]

Relevant context including the core changes which this issue is about making use of, and one example change which shows what to do: openssl/openssl#20940

[paulidale]

My suggestion is to automate all of the param handling code so that each set of OSSL_PARAMs has its own code-based TRIE decoder (not a hash table and not a data driven TRIE). This mandates a code generator to produce compilable C for each set of OSSL_PARAMs. I.e. an awful lot of them.

Moreover, all settable params should also be gettable. The converse isn't true but we should do as best as we can. This can be automated.

This will mean a lot of generated code and our build infrastructure isn't ideal in supporting this. We might need to revisit this too.

I've subsequently had further inspiration as to how to do this. I.e. the exemplary example isn't.

[t8m]

My suggestion is to automate all of the param handling code so that each set of OSSL_PARAMs has its own code-based TRIE decoder (not a hash table and not a data drive TRIE). This mandates a code generator to produce compilable C for each set of OSSL_PARAMs. I.e. an awful lot of them.

Moreover, all settable params should also be gettable. The converse isn't true but we should do as best as we can. This can be automated.

This will mean a lot of generated code and our build infrastructure isn't ideal in supporting this. We might need to revisit this too.

I've subsequently had further inspiration as to how to do this. I.e. the exemplary example isn't.

I think this is a noble goal but probably not something that should be a top priority for the team. Instead for now adopting the way used in ossl_cipher_gcm_set/get_ctx_params is worth it and can bring the most benefits.

[nhorman]

what about something a bit simpler in the short term that would accelerate the search time?

E.g. maybe replacing the STACK_OF data structure underpinning the OSSL_PARAM api with a list that we can do inserts to at specific locations? That would let us do a binary search on parameter lookup, reducing O(n) lookup times to O(log n). We would loose some time on param creation, but it might be a net gain overall, and wouldn't be hard to implement

[paulidale]

OSSL_PARAMS aren't stack based. They are an array most of the time. Simply sorting them will pose problems. Some algorithms accept the same parameter multiple times and concatenate their values (yes it's nasty but reasons). Any sort would therefore have to be a stable sort (i.e. order preserving over ties).

The trie approach gives constant time lookup (strictly, length of the thing being searched for). We've currently got a trie that includes all param names which can give fast lookups. It just needs to be integrated more widely.

My proposal was to build a trie for each algorithm's param set and use that -- this would be faster again but of the same big-O complexity.