Skip to content

feat(node): split GC into phases and make blob info cleanup blocking#3162

Draft
shuowang12 wants to merge 1 commit intomainfrom
shuo/GC_phases
Draft

feat(node): split GC into phases and make blob info cleanup blocking#3162
shuowang12 wants to merge 1 commit intomainfrom
shuo/GC_phases

Conversation

@shuowang12
Copy link
Collaborator

Description

  • Split garbage collection into two independent phases:

    • Phase 1 perform_blob_info_cleanup : cleans up per_object_blob_info (and future storage_pool_info). Runs inline at the epoch boundary before latest_event_epoch_sender.send(), blocking new-epoch blob syncs until cleanup completes.
    • Phase 2 start_data_deletion_task : deletes slivers and metadata in a background task after a random delay.
  • Abort the previous epoch's data deletion task before running phase 1 to avoid concurrent writes to aggregate_blob_info.

  • Each phase gets its own DB compaction guard so compactions aren't disabled during the random delay between phases.

This prepares for deterministic snapshots of per_object_blob_info and storage_pool_info at epoch boundaries for node recovery.

Test plan

  • Existing tests pass
  • Verify GC behavior in local testbed across epoch transitions

Release notes

Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required.
For each box you select, include information after the relevant heading that describes the impact of your changes that
a user might notice and any actions they must take to implement updates. (Add release notes after the colon for each item)

  • Storage node:
  • Aggregator:
  • Publisher:
  • CLI:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant