Skip to content

backport: bitcoin/bitcoin#24304: [kernel 0/n] Introduce bitcoin-chainstate#7234

Draft
knst wants to merge 30 commits intodashpay:developfrom
knst:bp-kernel-0
Draft

backport: bitcoin/bitcoin#24304: [kernel 0/n] Introduce bitcoin-chainstate#7234
knst wants to merge 30 commits intodashpay:developfrom
knst:bp-kernel-0

Conversation

@knst
Copy link
Copy Markdown
Collaborator

@knst knst commented Mar 17, 2026 •
edited
Loading

Issue being fixed or feature implemented

This PR sum up all changes that are required to backport bitcoin#24304 and proof-of-concept that it is possible after just couple extra tiny steps.

There are list of PR that are dependencies of this one:

What was done?

Beside changes that already has been merged or having PRs such as #7178 [in review], #7070, #7008, #6992, #6959, #7106 (by Udjin), #7065 (by kwvg) and many other PRs:

  • drop dependency of InstantSend on CActiveMasternodeSync
  • drop dependency of InstantSend on CChainState
  • new stub implementation NullNodeSyncNotifier of CActiveMaternodeSync for governance
  • further refactorings of CGovernanceManager to reduce dependencies on other components
  • add guards for CTxMempool is nullptr [which could be for chainstate case]
  • removed multiple includes of msg_result.h over codebase
  • moved multiple functions between modules to reduce extra dependencies
  • done the backport kernel zero [kernel 0/n] Introduce bitcoin-chainstate bitcoin/bitcoin#24304 which unblocks the whole multiple series of all kernel backports

There are several dirty hacks that meant to be fixed and they will be done by extra PRs; local branches for these fixes are semi-ready:

  • CDKGSessionManager should be split to network & non-network pieces. For right now a workaround for linkage is used by moving some code to net_processing.cpp
  • CSporkManager - the same workaround as CDKGSessionManager
  • llmq::utils::EnsureQuorumConnections and llmq::utils::AddQuorumProbeConnections - the same workaround
  • CQuorumManager::RequestQuorumData should be moved away to avoid network dependencies

There are several dependencies of dash-chainstate that looks excessive, but they are not a blocker and may be removed in any time in the future:
- base58.cpp
- core_write.cpp
- evo/core_write.cpp
- key_io.cpp
- node/interface_ui.cpp
- protocol.cpp
- scheduler.cpp
- node/bloom.cpp
- index/txindex.cpp
- batchedlogger.cpp
- Reduce governance significantly so far as there's no actual governance enabled in chainstate mode (governance is out of blockchain anyway)
- Reduce llmq dependencies, probably to drop: dkgsession, dkgsessionmgr, dkgsessionohandler, debug
- Remove CSigningManager from LLMQContext. LLMQContext should be quorum-only-context; CSigningManager should be inside active/

How Has This Been Tested?

Build succeed with bitcoin#24304, functional & unit tests runs

Run dash-chainstate:

$ src/dash-chainstate /home/knst/.dashcore/
Hello! I'm going to print out some information about your datadir.
        Path: "/home/knst/.dashcore"
        Reindexing: false
        Snapshot Active: false
        Active Height: 2294070
        Active IBD: true
        CBlockIndex(pprev=0x5654a3bdb218, nHeight=2294070, merkle=5c87e557b9e697e9308a4a6bd684e362910fe418ca60e59ae1376e91c487df77, hashBlock=0000000000000010938b2d1c2f1ff08163dae17f4196a41d73b9e6e3c4114cc2)

Breaking Changes

N/A

Checklist:

Go over all the following points, and put an x in all the boxes that apply.

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have made corresponding changes to the documentation
  • I have assigned this pull request to a milestone (for repository code-owners and collaborators only)

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Mar 17, 2026

This pull request has conflicts, please rebase.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Mar 17, 2026 •
edited
Loading

⚠️ Potential Merge Conflicts Detected

This PR has potential conflicts with the following open PRs:

Please coordinate with the authors of these PRs to avoid merge conflicts.

This was referenced Mar 24, 2026
Merged
Merged
Merged
PastaPastaPasta added a commit that referenced this pull request Mar 31, 2026
@PastaPastaPasta
Merge #7248: refactor: simplify CoinJoin class hierarchy
22526ae 
eb2142d fix: added missing thread annotation for CJWalletManagerImpl (Konstantin Akimov)
40cd08d refactor: rename CCoinJoinBaseManager to CoinJoinQueueManager and make it non-virtual (Konstantin Akimov)
6fae09a refactor: remove inheritance from CCoinJoinBaseManager in src/test/coinjoin_basemanager_tests.cpp (Konstantin Akimov)
a0f39f2 refactor: removed CCoinJoinBaseManager from inheritance of CCoinJoinServer and make it a member (Konstantin Akimov)
72229ce refactor: inline class CCoinJoinClientQueueManager to CJWalletManagerImpl (Konstantin Akimov)
3b1c2da refactor: inline the middleman class CoinJoinWalletManager to CJWalletManagerImpl (Konstantin Akimov)
e2d8247 refactor: pass directly reference to CCoinJoinClientManager& instead unique_ptr inside ForEachCJClientMan (Konstantin Akimov)
689b4ec refactor: remove passing pointer to unique_ptr to more clear sequence of object initializtion in CJ (Konstantin Akimov)

Pull request description:

  ## Issue being fixed or feature implemented

  Current coinjoin subsystem have multiple classes that inherits from each other but some of them doesn't provide any real abstraction level yet complicate architecture.

  CJWalletManagerImpl, CCoinJoinClientManager, and CCoinJoinClientQueueManager are 2 separate pieces of implementation of CJWalletManager while CJWalletManagerImpl is already private and could keep all implementation inside.

  ## What was done?

  This PR simplify CJ architecture by removing boiler code and unneeded abstraction levels to make code easier to support, read and improve.

  The changes for this PR has been spotted and half-done during #7234 but it's not a direct dependency of final solution, so far as it was possible to fully cut CJ from dash-chainstate implementation.

   - replaces all references to unique_ptr to just a reference or just unique_ptr to more clear sequence of object initialization in CJ and ownership
   - inline the middleman class CoinJoinWalletManager to CJWalletManagerImpl which is already private and hidden
   - remove class CCoinJoinClientQueueManager; logic inlined into CJWalletManagerImpl
   - remove interface CoinJoinQueueNotify - que notification no longer needed as a separate abstraction
   - replaced inheritance of CCoinJoinServer : public CCoinJoinBaseManager to composition (m_queueman member)
   - remove inheritance of regression tests from CCoinJoinBaseManager
   - rename CCoinJoinBaseManager to CoinJoinQueueManager and make it non-virtual

   ## CoinJoin Class Diagram — Before vs After [by claude]

      BEFORE

      CCoinJoinBaseManager          CoinJoinQueueNotify
       (virtual ctor/dtor)           (interface)
       - vecCoinJoinQueue             - OnQueueUpdate() [pure]
       - cs_vecqueue
              ▲                              ▲
              │ inherits              │ implements
              ├─────────────────┐     │
      CCoinJoinClientQueueManager  CCoinJoinClientManager
       (owned by CJWalletManagerImpl)  (map entry per wallet)
              │
              │ inherits
              │
       CCoinJoinServer ──also inherits──▶ CCoinJoinBaseSession
                                               ▲
                                               │ inherits
                                         CCoinJoinClientSession

      CJWalletManager (abstract)
       └── CJWalletManagerImpl
            ├── unique_ptr<CCoinJoinClientQueueManager>  m_basequeueman
            └── map<wallet* → CCoinJoinClientManager>

      ---
      AFTER

      CoinJoinQueueManager           ← renamed (no C prefix), no virtual ctor/dtor
       - vecCoinJoinQueue               no longer a base class at all
       - cs_vecqueue
       + TryHasQueueFromMasternode()    new TRY_LOCK helpers
       + TryCheckDuplicate()
       + TryAddQueue()

      CCoinJoinServer  ──inherits──▶  CCoinJoinBaseSession
       - m_queueman: CoinJoinQueueManager   ← OWNED by value (composition)
                                               ▲
                                               │ inherits
                                         CCoinJoinClientSession

      CJWalletManager (abstract)
       └── CJWalletManagerImpl
            ├── unique_ptr<CoinJoinQueueManager>  m_queueman  ← nullptr if !relay_txes
            └── map<wallet* → CCoinJoinClientManager>
                     │
                     │  CCoinJoinClientManager
                     │   - m_queueman: CoinJoinQueueManager*   ← non-owning ptr
                     │                 (points into CJWalletManagerImpl::m_queueman)

  ## How Has This Been Tested?
  Run unit & functional tests.

  ## Breaking Changes
  N/A

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

ACKs for top commit:
  UdjinM6:
    utACK eb2142d

Tree-SHA512: 03a648885d86c6e251296b3315ddf2ffb0ab3839526cf10a52bb36f53dc7ff1888bc45f69bbdcddd8a32bd940955cf6e8487e84062d0dde9daa32317721eeb5a
PastaPastaPasta added a commit that referenced this pull request Apr 5, 2026
@PastaPastaPasta
Merge #7250: refactor: replace usages of MessageProcessingResult in H…
9641540 
…andleNewRecoveredSig to variant

eeb84cb refactor: replace usages of MessageProcessingResult in HandleNewRecoveredSig to variant (Konstantin Akimov)

Pull request description:

  ## Issue being fixed or feature implemented

  The public interface `HandleNewRecoveredSig` which is widely used on develop returns `MessageProcessingResult`

  Using MessageProcessingResult is incorrect using here; because `HandleNewRecoveredSig` may return only CInv or CTransactionRef.

  Using MessageProcessingResult here spreads including of heavy `msg_result.h` all over codebase; it makes dependency of llmq/ code on network code.

  This PR is a direct dependency for kernel / dash-chainstate project, see #7234

  ## What was done?
  Replaced MessageProcessingResult to std::variant<CInv, CTransactionRef, std::monostate>.

  Potentially, even CTransactionRef is overhead and should be replaced to special case of CInv, but it's out-of-scope of this refactoring.
  Also, `m_clhandler.ProcessNewChainLock` should be refactored and split to network and non-network part and avoid leaking `MessageProcessingResult` abstraction here too, but I kept it out-of-scope of this refactoring too.

  ## How Has This Been Tested?
  Run unit tests & functional tests.

  ## Breaking Changes
  N/A

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

ACKs for top commit:
  thepastaclaw:
    crACK eeb84cb

Clean refactoring that replaces the generic `MessageProcessingResult` with a precise `std::variant<std::monostate, CInv, CTransactionRef>` for `HandleNewRecoveredSig`. This addresses the TODO that was already in the code.

Verified:
- **ChainLock handler**: `ProcessNewChainLock(-1, ...)` can only produce a single `CInv` or empty result (no `MisbehavingError` when `from == -1`, no `m_transactions`), so extracting `m_inventory.front()` is correct. The TODO comment about splitting `ProcessNewChainLock` is a good note for follow-up.
- **EHF handler**: Now returns `CTransactionRef` directly instead of pushing `GetHash()` into `m_transactions`. The caller in `net_signing.cpp` calls `PeerRelayTransaction(hash)` which routes through `RelayTransaction` → `_RelayTransaction` — same codepath as the old `PostProcessMessage` loop over `m_transactions`.
- **InstantSend + SigShares handlers**: Always returned empty `MessageProcessingResult{}`, now correctly return `std::monostate{}`.
- **Dispatch in `net_signing.cpp`**: Clean `std::get_if` dispatch replacing the opaque `PeerPostProcessMessage` call. Each variant arm calls the appropriate relay function.
- `PeerPostProcessMessage` remains in `net_processing.h` for other callers — no dangling references.
  UdjinM6:
    utACK eeb84cb

Tree-SHA512: 6158fc345ad9a70d2b0d4da0b5c3cfef76642e16d5738fab3aebdc771f68f4b3074bb7638864a3574ae66ad58123341dcd5dd83d39d7fb0737881129809b6427
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 12, 2026

This pull request has conflicts, please rebase.

PastaPastaPasta added a commit that referenced this pull request Apr 28, 2026
@PastaPastaPasta
Merge #7247: refactor: simplify architecture of quorum observer and q…
845f965 
…uorum participant

863ee6b doc: clarify masternode late InitializeCurrentBlockTip comment (UdjinM6)
dc8c6cf fix: replay startup tip into ActiveContext/ObserverContext (UdjinM6)
21fe75f fix: guard GetProTxHash() behind is_masternode check, add CQuorum safety assert (UdjinM6)
27129d9 fix: defer full InitializeCurrentBlockTip broadcast on masternodes (UdjinM6)
55d8025 refactor: safety belt to be sure that CQuorum is initialized properly and no empty qc member (Konstantin Akimov)
8bc5744 fix: review comments - missing forward declaration, protection for no valid members (Konstantin Akimov)
e2df648 refactor: drop argument request_limit_exceeded in ProcessContribQGETDATA (Konstantin Akimov)
b47de25 refactor: remove net related code ProcessContribQGETDATA and ProcessContribQDATA to NetQuorum (Konstantin Akimov)
1be8863 refactor: move out network code to new NetQuorum from CQuorumManager (Konstantin Akimov)
c86e4f6 refactor: it's always either observer_ctx or active_ctx, not both created together (Konstantin Akimov)
6a2f291 refactor: remove duplicated code between QuorumRole's UpdatedBlockTip and InitializeQuorumConnections (Konstantin Akimov)
6c99cb0 fix: parameter 'params' shadows member inherited from type 'CDKGSession' [-Werror,-Wshadow-field] (Konstantin Akimov)
0f1174a refactor: remove multiple friends of CDKGSession (Konstantin Akimov)
f4cbf92 refactor: inline QuorumParticipant to ActiveContext (Konstantin Akimov)
5ded711 refactor: rename src/llmq/observer/context to src/llmq/observer (Konstantin Akimov)
93f168a refactor: remove QuorumRoleBase by moving its functionality directly to QuorumRole (Konstantin Akimov)
9bb50db refactor: merge QuorumObserver and ObserverContext classes to the one entity to reduce amount of abstractions (Konstantin Akimov)
7978822 refactor: quorum members: remove QuorumObserverParent; participant is no more child of 'observer' (Konstantin Akimov)

Pull request description:

  ## Issue being fixed or feature implemented
  It's a follow-up for #7066 and related PRs

  This PR is reducing mental concentration when reading quorum-related code by significant simplification amount of levels of abstractions and inheritance classes from each other.

  This PR is a direct dependency of #7234 (kernel / chainstate project)

  Performance is not expected to be improved by any noticeable margin.

  Compilation time is expected to be improved marginally due to fewer files (-2) to compile and less transitive includes over `src/llmq/quorumsman.h`.

  ## What was done?

  This changes has been spotted during #7234 and extracted to own PR.

   - Remove QuorumObserverParent; decouple QuorumParticipant from QuorumObserver inheritance — they become independent implementations of a common QuorumRole interface.
   - Merge QuorumObserver and ObserverContext into a single class, then remove the transitional QuorumRoleBase by folding it into QuorumRole.
   - Rename src/llmq/observer/context.{h,cpp} to src/llmq/observer.{h,cpp} — flatten the now-single-file subdirectory.
   - Inline QuorumParticipant into ActiveContext — it only applies to active masternode mode; src/active/quorums.{h,cpp} deleted.
   - Remove four friend declarations from CDKGSession (ActiveDKGSession, ActiveDKGSessionHandler, CDKGSessionHandler, CDKGSessionManager); affected members changed from private to protected.
   - Extract network code from CQuorumManager and ActiveContext into new NetQuorum (src/llmq/net_quorum.{h,cpp}) — handles QGETDATA/QDATA processing, quorum peer connections, data recovery, and periodic cleanup.

  ###  Class hierarchy — before vs. after
   *by claude

    Before:
    QuorumObserverParent (interface, implemented by CQuorumManager)
        │
    QuorumObserver (base class with shared state)
        ├── ObserverContext  (observer mode, also CValidationInterface)
        └── QuorumParticipant (active mode, in src/active/quorums.h)

    After:
    QuorumRole (pure virtual interface in quorumsman.h)
        ├── ObserverContext  (observer mode, in src/llmq/observer.h)
        └── ActiveContext    (active mode, QuorumParticipant inlined)

  ## How Has This Been Tested?
  Run unit & functional tests

  ## Breaking Changes
  N/A

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

ACKs for top commit:
  UdjinM6:
    utACK 863ee6b

Tree-SHA512: 13a3d8e533bf811fd5d438e0c2f3007b633e990b725767907d5c4b8eb2da32d06f282f3f61319ce78a17e40ca41dac1b3c791039c5672321e1b43a7174712b2d
knst added 7 commits April 29, 2026 13:08
@knst knst mentioned this pull request Apr 30, 2026
Open
5 tasks
knst and others added 9 commits April 30, 2026 15:33
@knst
fix: add guards if CTxMempool is nullptr in chainstate
85700a2
@knst
refactor: drop dependency of governance/object on core_write (ScriptT…
542e24a 
…oAsmStr)
@knst
refactor: drop pfrom from CGovernanceManager::ProcessVote
1ee8fd9
@knst
refactor: move CDeterministicMN::ToJson to core_write to avoid g_txin…
168d624 
…dex dependency for linking
@knst
fix: circular dependency over rpc/evo_util
912ca95
@knst
refactor: drop dependency of CGovernanceManager on net.cpp
2ab970d
@knst
refactor: move multiple msg_result.h usages headers to implementation
eed4096 
NOTE: the circular dependency over dkgsessionhandler <-> net_processing is not new,
it's pre-existing one.

Dependency of llmq/observer is also pre-existing but has been marked by recursive including over msg_result.h
    "llmq/dkgsessionhandler -> net_processing -> llmq/dkgsessionmgr -> llmq/dkgsessionhandler",
    "llmq/dkgsessionhandler -> net_processing -> llmq/observer -> llmq/dkgsessionhandler",
@knst
refactor: move VerifyChainLock and ChainLockSig between modules to un…
621c171 
…tangle chain-state dependencies

chain-state should be available to validate chainlocks without using
chainlock/handlers.cpp which is tightly connected to network code.

On other hand, VerifyChainLock uses quorumsman, so it can't be just moved to chainlock/clsig or something
like that because it will create new circular dependency over quorumsman.

Though, chainlock::Chainlock and chainlock::ChainLockSig are tightly connected
and there is no usages of ChainLockSig withdout Chainlock except regression tests;
moving ChainLockSig to chainlock.h is fine then.
@knst
Merge bitcoin#24304: [kernel 0/n] Introduce bitcoin-chainstate
bc8642f 
2c03cec ci: Build bitcoin-chainstate (Carl Dong)
095aa6c build: Add example bitcoin-chainstate executable (Carl Dong)

Pull request description:

  Part of: bitcoin#24303

  This PR introduces an example/demo `bitcoin-chainstate` executable using said library which can print out information about a datadir and take in new blocks on stdin.

  Please read the commit messages for more details.

  -----

  #### You may ask: WTF?! Why is `index/*.cpp`, etc. being linked in?

  This PR is meant only to capture the state of dependencies in our consensus engine as of right now. There are many things to decouple from consensus, which will be done in subsequent PRs. Listing the files out right now in `bitcoin_chainstate_SOURCES` is purely to give us a clear picture of the task at hand, it is **not** to say that these dependencies _belongs_ there in any way.

  ### TODO

  1. Clean up `bitcoin-chainstate.cpp`
     It is quite ugly, with a lot of comments I've left for myself, I should clean it up to the best of my abilities (the ugliness of our init/shutdown might be the upper bound on cleanliness here...)

ACKs for top commit:
  ajtowns:
    ACK 2c03cec
  ryanofsky:
    Code review ACK 2c03cec. Just rebase, comments, formatting change since last review
  MarcoFalke:
    re-ACK 2c03cec 🏔

Tree-SHA512: 86e7fb5718caa577df8abc8288c754f4a590650d974df9d2f6476c87ed25c70f923c4db651c6963f33498fc7a3a31f6692b9a75cbc996bf4888c5dac2f34a13b
knst and others added 14 commits April 30, 2026 15:33
@knst
fix: multiple dashifications applied to bitcoin-chainstate.cpp
23eb669
@knst
refactor: reimplementing some methods directly in dash-chainstate to …
bdd7fb8 
…avoid extra heavy dependencies

Implemented:
  - ValueFromAmount (core_write.cpp)
  - GetPrettyExceptionStr (stacktrackes.cpp)
@knst
multiple fixes for Makefile.am
e743e3c
@knst
dashify name bitcoin-chainstate to dash-chainstate
f51f60c
@knst
refactor: move network code from quorumsman.cpp to net_signing.cpp
5058bab
@knst
fixup net-signing
7d0392b
@knst
refactor: move some llmq::utils to net_signing where they are not sup…
7ac30fb 
…posed to be but to fix linkage errors
@knst
refactor: move temporary some code from spork.cpp to net_processing.cpp
6fa8a8f
@knst
refactor: move code back from net_signing but completely commenting s…
656e7a0 
…everal functions
@knst
todo for dkgsessiongmgr.h
49267d8
@knst
finally works!!!! added some more files to Makefile.am. see commit me…
e0ce1d2 
…ssage for detailed notes about further todo-to-remove

TODO to remove:
  base58.cpp \
  core_write.cpp \
  evo/core_write.cpp \
  key_io.cpp \
  node/interface_ui.cpp \
  protocol.cpp \
  scheduler.cpp \
  node/bloom.cpp \
  index/txindex.cpp \
  batchedlogger.cpp \

  Reduce governance significantly so far as there's no actual governance enabled

  Reduce llmq: dkgsession, dkgsessionmgr, dkgsessionohandler, debug

  Remove CSigningManager from LLMQContext. LLMQContext should be quorum-only-context
@knst
Revert "refactor: move code back from net_signing but completely comm…
f8cf15d 
…enting several functions"

This reverts commit 656e7a0.
@knst
Revert "fixup net-signing"
1721273 
This reverts commit 7d0392b.
@knst
Revert "refactor: move network code from quorumsman.cpp to net_signin…
7b62572 
…g.cpp"

This reverts commit 5058bab.
@knst knst mentioned this pull request May 2, 2026
Open
5 tasks
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 5, 2026

This pull request has conflicts, please rebase.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

needs rebase

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@knst