Medium to large transfers fail in relay only mode

[rklaehn]

Doing a medium to large data transfer using the transfer example fails on main, with a timeout.

This fails even when running a local relay.

Steps to reproduce: you need 3 terminals, 1 sender, 1 receiver, 1 relay. Sender and receiver need to be set to relay only mode, which currently works by setting DEV_RELAY_ONLY=1 at compile time.

Relay: RUST_LOG=error cargo run --release -p iroh-relay --features server -- --dev
Sender: DEV_RELAY_ONLY=1 cargo run --release --example transfer --all-features -- provide --size=1g --relay-url http://127.0.0.1:3340
Receiver: DEV_RELAY_ONLY=1 cargo run --release --example transfer --all-features -- fetch --relay-url http://127.0.0.1:3340 <ticket_from_sender>

The transfer will slow down and then fail after a few megabytes.

For a detailed discussion, see https://discord.com/channels/949724860232392765/1324353518017974354

The issue seems to be fixed on #3077, so hopefully it will still be fixed when this code makes it to main.

[flub]

I strongly suspect this is a dup of #2951 which #3062 is aiming to address.

[Arqu]

This seems to be fixed for me. (running with #3099)

[flub]

Let's close this for now then. I handwavingly explain the issues as effective deadlocks or crawling to an extremely slow crawl of flows through the magic socket actors on the way to the relay server. Hence connection timeouts.