Skip to content

feat: switch to mapping instead of linear search#90

Closed
exo404 wants to merge 4 commits into
mainfrom
45-optimize-recipient-queue-lookup-in-baserecipeintregistry-for-large-queues-addition-and-removal
Closed

feat: switch to mapping instead of linear search#90
exo404 wants to merge 4 commits into
mainfrom
45-optimize-recipient-queue-lookup-in-baserecipeintregistry-for-large-queues-addition-and-removal

Conversation

@exo404
Copy link
Copy Markdown
Contributor

@exo404 exo404 commented Mar 17, 2026

This solves #45

@exo404 exo404 requested a review from RonTuretzky March 17, 2026 18:34
@exo404 exo404 self-assigned this Mar 17, 2026
@exo404 exo404 added the enhancement New feature or request label Mar 17, 2026

/// @notice Mapping to quickly check if an address is queued recipient
/// @dev Maps recipient address to true if queued, false otherwise
mapping(address => bool) public isQueued;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dont we need to clear this queue after they're added?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@exo404 and we should be handling removals as well?

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to optimize recipient queue duplicate detection in AbstractRecipientRegistry by switching the addition-queue duplicate check from an O(n) linear scan to an O(1) mapping lookup, addressing performance concerns raised in #45.

Changes:

  • Added an isQueued mapping to track whether an address is queued.
  • Updated _queueForAddition to use the mapping instead of iterating queuedRecipientsForAddition.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/abstract/AbstractRecipientRegistry.sol Outdated
Comment on lines 41 to 46
if (isRecipientMapping[recipient]) revert RecipientAlreadyExists();
if (isQueued[recipient]) revert RecipientAlreadyQueued();

// Check if already queued to prevent duplicates
for (uint256 i = 0; i < queuedRecipientsForAddition.length; i++) {
if (queuedRecipientsForAddition[i] == recipient) {
revert RecipientAlreadyQueued();
}
}

isQueued[recipient] = true;
queuedRecipientsForAddition.push(recipient);
emit RecipientQueued(recipient, true);
Copy link

Copilot AI Mar 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isQueued is set to true when queueing an addition, but it is never cleared when the addition queue is processed (_processQueue) or manually cleared (clearAdditionQueue). This makes it impossible to re-queue the same recipient later (e.g., add -> remove -> attempt to add again will revert with RecipientAlreadyQueued). Ensure the mapping is reset when entries leave the addition queue (during processing and when clearing).

Copilot uses AI. Check for mistakes.
Comment on lines 39 to 46
function _queueForAddition(address recipient) internal {
if (recipient == address(0)) revert InvalidRecipient();
if (isRecipientMapping[recipient]) revert RecipientAlreadyExists();
if (isQueued[recipient]) revert RecipientAlreadyQueued();

// Check if already queued to prevent duplicates
for (uint256 i = 0; i < queuedRecipientsForAddition.length; i++) {
if (queuedRecipientsForAddition[i] == recipient) {
revert RecipientAlreadyQueued();
}
}

isQueued[recipient] = true;
queuedRecipientsForAddition.push(recipient);
emit RecipientQueued(recipient, true);
Copy link

Copilot AI Mar 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR description references solving #45 (duplicate checks for both addition and removal queues), but this change only adds an O(1) check for the addition queue. The removal queue still uses a linear scan, so large removal queues will keep the same O(n) behavior. Either extend the mapping approach to removals as well, or adjust the PR/issue linkage to reflect the remaining work.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Contributor Author

@exo404 exo404 Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines 41 to 46
if (isRecipientMapping[recipient]) revert RecipientAlreadyExists();
if (isQueued[recipient]) revert RecipientAlreadyQueued();

// Check if already queued to prevent duplicates
for (uint256 i = 0; i < queuedRecipientsForAddition.length; i++) {
if (queuedRecipientsForAddition[i] == recipient) {
revert RecipientAlreadyQueued();
}
}

isQueued[recipient] = true;
queuedRecipientsForAddition.push(recipient);
emit RecipientQueued(recipient, true);
Copy link

Copilot AI Mar 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This introduces new queue-tracking behavior via isQueued, but there are existing Foundry tests for registries and none currently cover re-adding a recipient after it was previously added then removed, or re-queuing after clearAdditionQueue(). Adding tests for these scenarios would catch regressions around clearing the queued mapping when entries leave the queue.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@exo404 exo404 force-pushed the 45-optimize-recipient-queue-lookup-in-baserecipeintregistry-for-large-queues-addition-and-removal branch from 6664e00 to 4d95fb9 Compare March 31, 2026 11:49
@exo404
Copy link
Copy Markdown
Contributor Author

exo404 commented Mar 31, 2026

@RonTuretzky fixed and solved conflicts with main branch to align to the EIP 7201 standard

@RonTuretzky RonTuretzky requested a review from Copilot March 31, 2026 17:11
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +184 to +188
AbstractRecipientRegistryStorage storage $ = _getAbstractRecipientRegistryStorage();
for (uint256 i = 0; i < $.queuedRecipientsForAddition.length; i++) {
$.isQueuedForAdditionMapping[$.queuedRecipientsForAddition[i]] = false;
}
delete $.queuedRecipientsForAddition;
Copy link

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clearing the addition queue is now O(n) and can become uncallable if the queue grows large enough to exceed the block gas limit, leaving the owner unable to cancel pending additions. Consider a constant-gas invalidation pattern (e.g., a queue “epoch/nonce” plus mapping to the epoch) or a batched clearAdditionQueue(uint256 start, uint256 count) approach so the queue can always be cleared progressively.

Copilot uses AI. Check for mistakes.
Comment on lines +195 to +199
AbstractRecipientRegistryStorage storage $ = _getAbstractRecipientRegistryStorage();
for (uint256 i = 0; i < $.queuedRecipientsForRemoval.length; i++) {
$.isQueuedForRemovalMapping[$.queuedRecipientsForRemoval[i]] = false;
}
delete $.queuedRecipientsForRemoval;
Copy link

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same concern as clearAdditionQueue: this O(n) loop can make clearRemovalQueue fail for large queues due to gas limits. A batched clear function or an epoch-based invalidation mechanism would avoid a potential operational dead-end where the removal queue cannot be cleared.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@RonTuretzky wdyt? This is called by owner only

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Best practice would be to limited to some sort of upper limit with like a constant

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread src/abstract/AbstractRecipientRegistry.sol
Copy link
Copy Markdown

@Tranquil-Flow Tranquil-Flow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall

The mapping-based O(1) duplicate detection is the right approach and the ERC-7201 namespaced storage usage is correct. However, two blocking issues need resolution before merge.


🔴 No test coverage

The PR modifies only AbstractRecipientRegistry.sol — zero test files are included. The following scenarios need coverage:

  • Re-queue after removal: Add → process → remove → process → re-add. If the mapping isn't cleared properly, re-adding is permanently blocked.
  • Re-queue after clear: Queue → clearAdditionQueue() → re-queue. Verifies the mapping reset in clear functions works.
  • Double-queue revert: Confirm RecipientAlreadyQueued fires when the same address is queued twice.
  • Cross-queue independence: An address queued for addition should still be queueable for removal independently.

🔴 clearAdditionQueue and clearRemovalQueue are now O(n) and gas-unbounded

The original implementations were delete $.queuedRecipientsForAddition — a single SSTORE. Now they iterate the full array to clear each mapping entry. If the queue grows large enough, these owner-only functions will exceed the block gas limit and become permanently uncallable.

Two options:

  1. Cap queue length with a constant (e.g., MAX_QUEUE_SIZE = 100) enforced in _queueForAddition / _queueForRemoval.
  2. Epoch-based invalidation: Increment a queueEpoch counter; key becomes keccak256(abi.encode(recipient, epoch)) so clearing is O(1).

🟡 New revert path in _queueForRemoval

The address(0) check is a reasonable hardening, but it's a behavioral change — previously address(0) would fail on !$.isRecipientMapping[recipient] with a different revert. Worth noting in the PR description.


✅ What's working well

  • Correct use of ERC-7201 namespaced storage — avoids the layout collision flagged on the earlier approach
  • Separate mappings for add vs. remove queues — handles the edge case of an address being in both
  • Mapping cleanup on all exit paths (_processQueue, clearAdditionQueue, clearRemovalQueue)
  • Good iteration through review feedback

@RonTuretzky
Copy link
Copy Markdown
Contributor

Superseded by #118, which cherry-picks these commits onto the current main with all review feedback addressed (tests, MAX_QUEUE_SIZE cap, both queues). The sorted-array rewrite in #126 further improves the approach.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Optimize Recipient Queue Lookup in BaseRecipeIntRegistry for Large Queues (addition and removal)

4 participants