core, util: attach stateroot witness to NeoFS state objects #4099

AnnaShaleva · 2025-11-25T20:52:17Z

As an alternative, we may consider attaching a full serialized stateroot instead of just witness (right now stateroot version is missing from the state object which may be a problem in future if stateroot version will be changed).

codecov · 2025-11-25T20:58:31Z

Codecov Report

❌ Patch coverage is 33.48837% with 143 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.34%. Comparing base (e26aec0) to head (c902752).
⚠️ Report is 2 commits behind head on master.

Files with missing lines	Patch %	Lines
pkg/services/statefetcher/statefetcher.go	0.00%	115 Missing ⚠️
pkg/core/statesync/module.go	69.35%	14 Missing and 5 partials ⚠️
cli/util/upload_state.go	0.00%	3 Missing ⚠️
pkg/core/blockchain.go	66.66%	1 Missing and 1 partial ⚠️
pkg/core/transaction/witness.go	80.00%	1 Missing and 1 partial ⚠️
pkg/network/server.go	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4099      +/-   ##
==========================================
- Coverage   83.44%   83.34%   -0.10%     
==========================================
  Files         352      352              
  Lines       42748    42854     +106     
==========================================
+ Hits        35669    35718      +49     
- Misses       5324     5371      +47     
- Partials     1755     1765      +10

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

AnnaShaleva · 2025-11-26T08:19:42Z

I'll test this code on real network once we finalize the PR.

pkg/core/native/designate.go

roman-khimov · 2025-11-26T09:14:36Z

pkg/services/statefetcher/statefetcher.go

 	}

-	lastFoundIdx, err := strconv.ParseUint(lastItem.Attributes[0], 10, 32)
+	lastItemH, err := neofs.ObjectHead(ctx, s.Pool, s.Account.PrivateKey(), s.ContainerID, lastItem.ID)


Why an additional Head? You can request the attribute from ObjectSearch directly.

How exactly?

From what I understood, we need to include attribute to a search filter. The problem is that this attribute is optional, and there's no such filter like "check attribute if it's present".

No, just add it to the list of requested attributes, you'll get it if it's present.

roman-khimov · 2025-11-26T09:17:03Z

pkg/services/statefetcher/statefetcher.go

 				continue
 			}
-			if err = s.chain.AddContractStorageItems(batch, syncHeight, expectedRoot); err != nil {
+			if err = s.chain.AddContractStorageItems(batch, syncHeight, expectedRoot, witness); err != nil {


I'm not sure why you're passing it always when processing batches of KVs. Intermediate roots will never be correct, only the final one can be, so to me you should be:

checking stateroot witness once before performing any state changes

adding sets of KVs as usual

checking the resulting stateroot to be equal to the one we proved to be correct at the first step

I'm not sure why you're passing it always when processing batches of KVs

I'm just following the current implementation since we're passing expectedRoot every time as far. It was like that since #3844, let's refactor this code.

checking stateroot witness once before performing any state changes

It was an initial implementation, but then I discovered stateroot verification functions bound to the state service and decided to reuse this code (and it can be done only after state service cache initialization on the proper height). Fail-fast strategy is good, but we'll be sure that the whole state is correct only after full MPT recovery at the third step.

So are we OK with the stateroot witness checking code duplication?

Implemented in a separate commit.

roman-khimov · 2025-11-26T09:25:12Z

pkg/core/blockchain.go

+	// TODO: @roman-khimov, we need to notify NeoFS nodes about chain state reset so that they update
+	// NeoFSAlphabet-dependent services accordingly (the current code won't work because it's based
+	// on notifications, and there are no notifications during state sync). I suggest to add a callback
+	// to the Blockchain that will be called once state reset is finished.


It only happens on a new node normally and this can happen with non-local RPC as well, so callbacks can't solve the problem.

AnnaShaleva · 2025-12-02T14:00:17Z

TestStateSyncModule_Init is the last one that failing, so I'm still in progress of fixing it.

A part of #4049. Signed-off-by: Anna Shaleva <[email protected]>

Oracle, StateRoot and Notary services should properly react to state jump (either occured after state reset or after statesync finalization). Otherwise, these services use an out-of-date roles information retrieved from the previous node state. Signed-off-by: Anna Shaleva <[email protected]>

A part of #4049. 1. Check the stateroot witness. 2. Ensure witness' public keys match block's validators at the state sync height. Signed-off-by: Anna Shaleva <[email protected]>

This callback is set when module is not active yet, no locking required. Signed-off-by: Anna Shaleva <[email protected]>

(*Module).syncStage is a subject of change, it may be updated by statechanging Module's callbacks concurrently. Signed-off-by: Anna Shaleva <[email protected]>

Signed-off-by: Anna Shaleva <[email protected]>

roman-khimov · 2025-12-12T15:42:33Z

pkg/core/stateroot/module.go


 // JumpToState performs jump to the state specified by given stateroot index.
-func (s *Module) JumpToState(sr *state.MPTRoot) {
+func (s *Module) JumpToState(sr *state.MPTRoot) error {


Always returns nil (at least as of the commit where it's added).

roman-khimov · 2025-12-12T15:47:03Z

pkg/services/statefetcher/statefetcher.go

+		s.isActive.CompareAndSwap(true, false)
+		return 0, fmt.Errorf("failed to decode state root from attribute: %w", err)
+	}
+	if len(lastItem.Attributes) > 2 {


They will always be returned, but they can be empty is missing from object.

roman-khimov · 2025-12-12T15:48:00Z

pkg/core/statesync/module.go

 	return s.defineSyncStage()
 }

+func (s *Module) InitContractStorageItemsSync(syncHeight uint32, expectedRoot util.Uint256, witness transaction.Witness) {


Empty in 7523f7c

roman-khimov · 2025-12-12T15:51:18Z

pkg/core/dao/dao.go

+	// IntermediateRoot is a computed intermediate root of non-complete MPT
+	// at the StateSyncCheckpoint.
+	IntermediateRoot util.Uint256
+	// TODO: @roman-khimov, technically this and previous commit include a DB schema change since StateSyncCheckpoint is stored in the DB.


Add a note to release, that's it. State sync should be finished with the old node if it's in process.

roman-khimov · 2025-12-12T15:56:06Z

pkg/services/statefetcher/statefetcher.go

+// height is returned.
+func (s *Service) FindStateObject(minH uint32, maxH uint32) (uint32, error) {
+	if maxH > 0 && maxH < minH {
+		// Program bug.


But it's a public interface, can be used by IR, for example.

roman-khimov · 2025-12-12T15:59:10Z

pkg/services/statefetcher/statefetcher.go

+			// TODO: @roman-khimov, we don't include len(KVs) in the state object,
+			//       so the only indicator of state object end is EOF. I suggest
+			//       to change the format of state object files and include
+			//       len(KVs) to the header. Reuploading is required.


Streams are easier to create, how many KVs your DB has? You'll only know this after complete iteration. You have object data hash for consistency check and stateroot hash as well. Shouldn't be a problem.

AnnaShaleva requested a review from roman-khimov as a code owner November 25, 2025 20:52

roman-khimov reviewed Nov 26, 2025

View reviewed changes

AnnaShaleva force-pushed the witnessed-root branch 3 times, most recently from e0fdcb0 to a26d6a8 Compare November 28, 2025 10:48

AnnaShaleva marked this pull request as draft November 28, 2025 10:54

AnnaShaleva force-pushed the witnessed-root branch from a26d6a8 to 3d7e28d Compare November 28, 2025 14:56

AnnaShaleva added 5 commits December 11, 2025 18:14

util: attach witness to NeoFS state object

34e697e

A part of #4049. Signed-off-by: Anna Shaleva <[email protected]>

services: check stateroot witness during state sync

106ecdf

A part of #4049. 1. Check the stateroot witness. 2. Ensure witness' public keys match block's validators at the state sync height. Signed-off-by: Anna Shaleva <[email protected]>

statesync: don't take module's lock on SetOnStageChanged

7523f7c

This callback is set when module is not active yet, no locking required. Signed-off-by: Anna Shaleva <[email protected]>

statesync: always access (*Module).syncStage under mutex taken

d66debe

(*Module).syncStage is a subject of change, it may be updated by statechanging Module's callbacks concurrently. Signed-off-by: Anna Shaleva <[email protected]>

AnnaShaleva force-pushed the witnessed-root branch 2 times, most recently from 27fb9c8 to 8aacb96 Compare December 12, 2025 10:39

core, services: refactor StateSyncModule/StateFetcher integration

c902752

Signed-off-by: Anna Shaleva <[email protected]>

AnnaShaleva force-pushed the witnessed-root branch from 8aacb96 to c902752 Compare December 12, 2025 10:42

AnnaShaleva marked this pull request as ready for review December 12, 2025 10:42

AnnaShaleva requested a review from roman-khimov December 12, 2025 10:42

roman-khimov reviewed Dec 12, 2025

View reviewed changes

core, util: attach stateroot witness to NeoFS state objects #4099

Are you sure you want to change the base?

core, util: attach stateroot witness to NeoFS state objects #4099

Uh oh!

Conversation

AnnaShaleva commented Nov 25, 2025

Uh oh!

codecov bot commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

AnnaShaleva commented Nov 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AnnaShaleva Nov 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AnnaShaleva commented Dec 2, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov bot commented Nov 25, 2025 •

edited

Loading

AnnaShaleva commented Nov 26, 2025 •

edited

Loading

AnnaShaleva Nov 27, 2025 •

edited

Loading