[WIP] feat: migrate to iroh 0.5

[dignifiedquire]

Context: We will be releasing the new version of iroh (0.5.0) with support for holepunching next week.
This is a WIP PR to track that we can upgrade deltachat easily and test things out.

TODOs

• Release iroh 0.5.0
• Disable usage of DERP.
• Release new version of pgp: feat: update dalek crypto deps rpgp/rpgp#247
• Fix compile issues
• Test

[link2xt]

Is it possible to disable holepunching at first? So we have a stable release with updated iroh without quinn patches, and then enable holepunching in a later version.

[dignifiedquire]

Is it possible to disable holepunching at first?

Not entirely, we can disable usage of the relays, but not all of hole punching.

[rklaehn]

this assumes that the request is complete without gaps. Which I guess is fine, but maybe there should be an assert for the assumption somewhere.

[dignifiedquire]

how would you check for that?

[rklaehn]

You could do an assert on the request. But then again you are creating the request a few lines earlier, so maybe just have a comment where the request is created? Otherwise check that child_index is consecutive.

[dignifiedquire]

@link2xt is this a known issue?

error: higher-ranked lifetime error
   --> deltachat-jsonrpc/src/api/mod.rs:145:1
    |
145 | / #[rpc(
146 | |     all_positional,
147 | |     ts_outdir = "typescript/generated",
148 | |     openrpc_outdir = "openrpc"
149 | | )]
    | |__^
    |
    = note: could not prove `Pin<Box<[async block@deltachat-jsonrpc/src/api/mod.rs:145:1: 149:3]>>: CoerceUnsized<Pin<Box<(dyn Future<Output = Result<Value, yerpc::Error>> + Send + 'd)>>>`
    = note: this error originates in the attribute macro `rpc` (in Nightly builds, run with -Z macro-backtrace for more info)

[link2xt]

@link2xt is this a known issue?

error: higher-ranked lifetime error
   --> deltachat-jsonrpc/src/api/mod.rs:145:1
    |
145 | / #[rpc(
146 | |     all_positional,
147 | |     ts_outdir = "typescript/generated",
148 | |     openrpc_outdir = "openrpc"
149 | | )]
    | |__^
    |
    = note: could not prove `Pin<Box<[async block@deltachat-jsonrpc/src/api/mod.rs:145:1: 149:3]>>: CoerceUnsized<Pin<Box<(dyn Future<Output = Result<Value, yerpc::Error>> + Send + 'd)>>>`
    = note: this error originates in the attribute macro `rpc` (in Nightly builds, run with -Z macro-backtrace for more info)

No, master and stable branches build successfully currently.

[dignifiedquire]

I am confused by the latest failures

• cargo deny check passes locally for me
• the async python test failure seems to be unrelated

[link2xt]

• cargo deny check passes locally for me

run scripts/deny.sh, it updates the index.

[link2xt]

I updated Desktop to use this branch and tried to scan the QR code generated with iroh 0.5, but Android failed with an error "invalid DCBACKUP payload". Is this incompatible with previous version of iroh?

EDIT: I have now compiled Android with iroh 0.5 and backup transfer from Desktop to Android worked.

So the problem is that QR code format changed and only iroh 0.5 can parse the tickets generated by iroh 0.5.

[r10s]

So the problem is that QR code format changed and only iroh 0.5 can parse the tickets generated by iroh 0.5.

ftr, as discussed in n0 chat, we cannot merge this pr in this form for that reason (we would break add-second-device for at least some weeks until all delta chat versions are updated)

[link2xt]

As a temporary solution I forked 0.4.1 and updated dependencies: #4881

[link2xt]

Current plan is to keep using iroh 0.4 (we are currently at iroh 0.4.2 pre-release) to keep backwards compatibility. We may eventually switch backup transfer to using latest iroh-net directly to setup a connection and use it to stream the .tar file. This should also allow to stream the backup in any direction, so would be a user-visible improvement.

[dignifiedquire]

In light of #5041 this becomes more relevant to avoid pulling in two versions of iroh. The biggest blocker to this was backwards compatibility, before proposing any solutions @link2xt @hpk42 could I get an overview of what the stability you need/want for this is?

[link2xt]

@dignifiedquire We are ok with breaking compatibility once, but then it makes sense to switch from using full iroh stack to just setting up a connection and allow backup transfer in both directions (see my previous comment) without agreeing in advance on the content. But then I expect that establishing connections given IP addresses and peer ID should be backwards-compatible in future versions of iroh.