`bitcoin_core_sv2::BitcoinCoreSv2JDP` should leverage `getTransactionsByWTxID`

[plebhash]

we have a path for leveraging IPC on #24 that is arguably suboptimal and results in avoidable ProvideMissingTransactions (relies on createNewBlock and locally keeps an optimistic "mempool-mirror")

@Sjors has this draft PR bitcoin/bitcoin#34020 which could help us optimize that

this PR could potentially land on v32, for which new feature deadline is around August 2026

so the scope here is to adapt bitcoin_core_sv2::BitcoinCoreSv2JDP to leverage this new method

without ProvideMissingTransactions trigger

• DeclareMiningJob arrives to JDS
• JDS sends a JdRequest::DeclareMiningJob to BitcoinCoreSv2JDP with:
  • populated wtxid_list: Vec<Wtxid> (from DeclareMiningJob)
• BitcoinCoreSv2JDP sends a getTransactionsByWTxID() to Bitcoin Core
• Bitcoin Core has all txs on its mempool, so it returns an array with all txdata for the DeclareMiningJob
• BitcoinCoreSv2JDP builds a Block (because now it has the full txdata of the DeclareMiningJob)
• BitcoinCoreSv2JDP calls checkBlock()
• BitcoinCoreSv2JDP sends JdResponse::Success (which includes txdata)
• JDC sends PushSolution to JDS
• JDS builds Block and sends to BitcoinCoreSv2JDP
• BitcoinCoreSv2JDP sends submitBlock() to Bitcoin Core

with ProvideMissingTransactions trigger

• DeclareMiningJob arrives to JDS
• JDS sends a JdRequest::DeclareMiningJob to BitcoinCoreSv2JDP with:
  • populated wtxid_list: Vec<Wtxid> (from DeclareMiningJob)
• BitcoinCoreSv2JDP sends a getTransactionsByWTxID() to Bitcoin Core
• Bitcoin Core is missing some txs on its mempool, so it returns an array where empty elements indicate the missing txs
• BitcoinCoreSv2JDP sends a JdResponse::MissingTransactions to JDS, while keeping the non-empty elements in cache
• JDS sends ProvideMissingTransactions to JDC
• JDC sends ProvideMissingTransactions.Success to JDS
• JDS sends a JdRequest::DeclareMiningJob to BitcoinCoreSv2JDP with:
  • populated missing_txs: Vec<Transaction> (contents from ProvideMissingTransactions.Success)
• BitcoinCoreSv2JDP builds a Block (because now it has the full txdata of the DeclareMiningJob)
• BitcoinCoreSv2JDP calls checkBlock()
• BitcoinCoreSv2JDP sends JdResponse::Success (which includes txdata)
• JDC sends PushSolution to JDS
• JDS builds Block and sends to BitcoinCoreSv2JDP
• BitcoinCoreSv2JDP sends submitBlock() to Bitcoin Core

[plebhash]

bitcoin/bitcoin#34020 merged

[plebhash]

@Sjors I need a sanity check here

as you know, the main goal of this issue is to move away from MempoolMirror (which is created via createNewBlock and BlockTemplate.waitNext)

but one detail that's been out of our radar up until now is the fact that aside from fetching txdata from wtxid, we're also using MempoolMirror to cache the current chain state

more specifically, this is how MempoolMirror looks like:

sv2-apps/bitcoin-core-sv2/src/unix_capnp/v31x/job_declaration_protocol/mempool.rs

Lines 12 to 18 in 0917f2a

	pub struct MempoolMirror {
	txdata: HashMap<Wtxid, Transaction>,
	current_prev_hash: Option<BlockHash>,
	current_nbits: Option<CompactTarget>,
	current_min_ntime: Option<u32>,
	current_bip34_height: Option<u32>,
	}

these parameters become ValidationContext:

sv2-apps/bitcoin-core-sv2/src/common/job_declaration_protocol/io.rs

Lines 9 to 20 in 0917f2a

	/// Snapshot of the template parameters used by the validator at decision time.
	///
	/// This lets callers distinguish stale-tip races from other validation failures.
	///
	/// Please check <https://github.com/stratum-mining/sv2-apps/issues/364>
	/// for more details on the regression that motivated this field.
	#[derive(Debug, Clone, Copy)]
	pub struct ValidationContext {
	pub prev_hash: BlockHash,
	pub nbits: CompactTarget,
	pub min_ntime: u32,
	}

this is very important on JDS side because it allows us to assert against many edge cases and assert parameter parity between the current DeclareMiningJob under validation and the corresponding SetCustomMiningJob that will arrive later

---

up until now, I was under the assumption that I was going to completely remove createNewBlock and BlockTemplate.waitNext from the v32x JDP implementation

but unfortunately it seems there's no easy way for me to fetch these ValidationContext parameters via IPC, aside from a BlockTemplate object, right?

in case you confirm this, I guess we'll have to stick with createNewBlock and BlockTemplate.waitNext, at least until TxCollection from #609 becomes a viable option (which will also leverage a BlockTemplate, but not via createNewBlock nor waitNext

[Sjors]

getTip() and waitTipChanged() can give you, and track, the current height and hash (i.e. the prev_hash of the next block). See https://github.com/bitcoin/bitcoin/blob/master/src/ipc/capnp/mining.capnp

You can get nbits and min_ntime (including for the next block) using the getmininginfo RPC.

I should open a PR to add min_ntime to that RPC.

I could also add a getInfo() IPC method, but if using RPC-via-IPC does the job here, it's probably not a good use of scarce Bitcoin Core review time. Update: well, I opened bitcoin/bitcoin#35712 and 2140-dev/bitcoin-capnp-types#32.

As a workaround, if that doesn't land in time, maybe use createNewBlock(useMempool: false) after waitTipChanged() returns a new tip. That gets you the info you need in a relatively light-weight way.