POC: JD clients should be able to opt-in to working on pool templates when their tip is ahead

[ckolivas]

Preamble:
We should be able to opt in on jd clients to updated tips and work from the upstream pool when it arrives earlier than local template generation. By making it completely opt-in and default to off with a configurable timeout, we can minimise risk from misbehaving pools.

Rationale:
Pools generally have dedicated very fast hardware, networking, and custom software to ensure very rapid identification and propagation of updatedtips and work generation. Whilst every effort should be made to reproduce this performance as much as possible at the client end, the reality is wildly variable hardware and location limitations means that the pools will have updated tips and work faster the vast majority of the time. In my own testing, this leads to lag of up to 1 second on high performance home PC hardware connected by a fast fibre connection. Low powered hardware is inversely proportionally slower in spite of IPC template generation.

In summary: Pool work virtually always arrives first, we should use it to minimise wasted work/risk of stale blocks.

I set out to implement a working modification to this and found that unfortunately it was vastly more complicated than I expected to implement, and I have very little expertise in rust. So apologies in advance but the working proof of concept code is 100% AI slop created with opus 4.8, but it works - treat only as a POC and not for digestion as is.

I have modified the ckpool sv2 implementation to send the same updated tip and work to JD clients as it does to regular clients. The endpoints can be used for testing:
stratum2+tcp://sv2solo.ckpool.org:3336/9anrRNhBh7869XtNnFcCuGBRZP51E635qGbu457J5kHdszhfRc3 - mining
stratum2+tcp://sv2solo.ckpool.org:3337/9anrRNhBh7869XtNnFcCuGBRZP51E635qGbu457J5kHdszhfRc3 - JD server

The POC concept code is available on the jd-accept-remote-work branch here: https://github.com/ckolivas/sv2-apps/tree/jd-accept-remote-work

Tested from a home fibre connection and i14900k to the sv2 server above show pool work being used 100% of the time on updated tips, despite being 215ms ping from the pool. Same v31.1 bitcoin core with virtually permanent uptime. Changeover to local work is usually 200-600ms later - this is with the latency fix recently committed to the main branch. Mining at a closer pool would potentially be of even more benefit.

New config options configured as:
accept_upstream_tip_work = true
upstream_tip_work_timeout_secs = 30

[TheBlueMatt]

Probably needs a spec (extension) PR, but I think this is reasonable in cases where we require the pool server to provide the actual header the new work is based on. That would ban using this for (Sv1) spy mining, which IMO would be a good thing. Of course any pool offering this to their clients would enable other pools to spy-mine on them, but at least we'd end up improving the "security" of spy mining by requiring a PoW check.

We might also reduce the default timeout as one goal of client-side work generation is to prevent the pool from doing block witholding/selfish mining by submitting blocks both directly and via the pool bitcoind and we don't want to mine on a selfish mining block for any longer than we have to.

[Fi3]

Probably needs a spec (extension) PR

why an extension? We just need the proxy to open 2 connection with pool one with JD and one without and switch the hashpower between the 2

[TheBlueMatt]

Huh. I guess we could do that, yes, but it might be nice to make this more explicit - if the pool is doing dynamic difficulty it might very confused by the fact that we have a connection with zero hashrate dotting around that suddenly has a full farm worth of hashrate for a few seconds then zero again for a while.

[ckolivas]

if the pool is doing dynamic difficulty it might very confused by the fact that we have a connection with zero hashrate dotting around that suddenly has a full farm worth of hashrate for a few seconds then zero again for a while.

A pool should be able to easily cope with that provided a suitable max target has already been sent. Perhaps the JD connection should choose a suitable max target such as round down to nearest power of 2, and send that on the secondary connection when the value changes based on hashrate on the JD connection.

[ckolivas]

│  n  │  min   │  median  │   mean   │   p90    │   max    │  stdev   │
┼─────┼────────┼──────────┼──────────┼──────────┼──────────┼──────────┤
│ 58  │ 6.5 ms │ 223.7 ms │ 237.6 ms │ 392.5 ms │ 609.7 ms │ 128.4 ms │

Here's a dataset of approximately 10 hours of runtime with the jd-accept-remote-work branch showing the time to cutover to local work on the aforementioned hardware.
EDIT: I should add this doesn't even factor in time to creating a template locally, so we have already received a template from the upstream pool and only just updated the tip locally.