feat(node): disable immediate data deletion after BlobDeleted events by default

[mlegner]

Description

Add a new configuration parameter garbage_collection.enable_immediate_data_deletion, which defaults to false. When processing a BlobDeleted event, data is only deleted immediately if this parameter is set to true.

Test plan

Added a new test case that tests the expected behavior for both enabled and disabled immediate data deletion.

---

Release notes

• Storage node: Add a new configuration parameter garbage_collection.enable_immediate_data_deletion, which defaults to false. When processing a BlobDeleted event, data is only deleted immediately if this parameter is set to true.

[github-actions]

Warning: This PR modifies one of the example config files. Please consider the
following:

• Make sure the changes are backwards compatible with the current configuration.
• Make sure any added parameters follow the conventions of the existing parameters; in
  particular, durations should take seconds or milliseconds using the naming convention
  _secs or _millis, respectively.
• If there are added optional parameter sections, it should be possible to specify them
  partially. A useful pattern there is to implement Default for the struct and derive
  #[serde(default)] on it, see BlobRecoveryConfig as an example.
• You may need to update the documentation to reflect the changes.

[halfprice]

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. More of your lovely PRs please.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[sadhansood]

Just a question - what happens to these blobs which are skipping to delete, we will be able to delete them during periodic end of epoch cleanup i think?

[mlegner]

Just a question - what happens to these blobs which are skipping to delete, we will be able to delete them during periodic end of epoch cleanup i think?

Yes, exactly. Those would be cleaned up during the "normal" GC process.