Compatibility Checks for RPC endpoints

[drahnr]

#1040

Context

We do not control the users client version, which might be either outdated or patched, and hence not comply ot whatever node version the server might be running.

Incompatibilities

There are 3 4 ways of incompatible changes

1. endpoint path
2. call signature change
3. semantic change of fields
4. serialization format changes

---

1. endpoint paths are defined by the rust module structure and transformed by prost for protobuf types - we can technically handle these
2. call signatures are available - we can detect changes via transforming them to i.e. normalized json
3. is out of scope, to my knowledge there are only humans that can detect such issues or a full formal verification which - as of today - is still a intractable problem for any meaningfully sized project
4. serialization changes are handled by protobuf, as long as the wireformat of prost does not change, this is not part of the problem scope

API in-compatibility detection

My current idea is, to provide an endpoint index at a fixed endpoint. The client can then optionally check it's own implementation against the node's provided API version.
The alternative is to use the transport and provide, i.e. for http additional version information on connection. This becomes brittle if we use connection termination or proxies.

Common API version

We can give the API a dedicated version, independently of any crate versions, and use semver to deduct compatibility.

Per endpoint

We could technically do so for each individual endpoint as well. For endpoints having digests might be a better idea, since there seems little room for compatibility, but the common case is works or does not work between version A and B.

Tools

• cargo-semver-check does check the entire public crate API against a defined past version of itself and yields compatibility errors. It might be possible to generate these compatibility objects at compile time. We could look for i.e. types that implement i.e. StoreApi and derive compatibility by just those. It's powered by trustfall which allows queries, and hence we could just scope the queries to our needs. This should be feasible assuming the promises of https://predr.ag/blog/speeding-up-rust-semver-checking-by-over-2000x/ hold up.

• Write a proc-macro to deduct the type information from the APIs and write the format out to a API version, the downside is that it percolates through all nested types deeply, and is a rather large effort Parity did this, the effort was ginormeous and took ~1a

• Use an external tool like buf that provides compatibility checks on the .proto level.

  cd proto/proto
  buf breaking --exclude-imports --against 'https://github.com/0xMiden/miden-node.git#branch=next,subdir=proto/proto'
  account.proto:13:5:Field "1" with name "idx" on message "AccountId" changed option "json_name" from "id" to "idx".
  account.proto:13:11:Field "1" on message "AccountId" changed name from "id" to "idx".

  Derived from that, we can have dedicated API version. Any type using derive [derive(prost::Message)]will hence not be covered see implementation details of the derive proc-macro

[drahnr]

Re-reading this, we might be able to combine cargo-semver-check's underpinning lib trustfal with a protobuf-format parser and write an adapter, and write queries against the trustfall query engine.

[Mirko-von-Leipzig]

If we version the gRPC schema independently (which we will need to do some day), then we might be able to apply semver to it with checks that we're actually adhering to it correct?

As in, on every PR we audit the schema using buf, and then in addition we use trustfall on the blob-encoded types? This would inform our overall gRPC semver somehow.

[drahnr]

The piece that is missing (or I don't know of yet) is the mapping from protobuf schema elements to the generated rust types, and then from those to our internal representation. I think both steps are required to make it bulletproof.

[Mirko-von-Leipzig]

#1072 contains some of my musings regarding gRPC traits.

I wonder if we couldn't (ab)use a gRPC server side trait for the methods serde component. This would then give us a list of all top-level types that form part of the gRPC set.

[Mirko-von-Leipzig]

But we may be digging too deep here for something that may have to be done manually.

[drahnr]

Actually, I need to marinade my previous statement some more, not convinced it is actually required, but depends on the properties of protobuf/protoc/prost.

[drahnr]

Now the question remains is, if there is sufficient information in protobuf to map to miden-base types in fashion that prohibits incompatibilites.

[Mirko-von-Leipzig]

I don't think there is enough info in protobuf as is. miden-base, or at least the types we used from there would have to be separately versioned/checked. Ideally this would be the concern of miden-base, but at the moment that isn't trivially possible.

[sergerad]

Would the intention to be to replace the version header we have in place today?
#804

AFAICT the functionality in this discussion would be a superset of capability provided by the version header.

[drahnr]

To my current understanding, the main issue is having a systematic approach to handle compatibility/incompatibility. Checking a deducted version for compatibility vs another is easy enough, but getting to that version is the hard part. @Mirko-von-Leipzig did I interpret the issue wrong?

[Mirko-von-Leipzig]

@sergerad I initially didn't intend for is to be a replacement but rather in-addition. But it does look like we may be able to do something more robust.

@drahnr no but I did expect the solution to be a bit more manual/simplistic. e.g. client optionally adds the miden-base commit SHA if its a non-release, and node always rejects SHA mismatches.

This actually looks like we might be able to cobble together something more automated which would be really cool.

[drahnr]

If we do this properly, it'll take a hot second, it's no trivial feat. Alternatively, I can start with the more simple version and see how far it's going to carry us.