HDDS-14347. Handle upgrades to support S3 lifecycle

[YutaLin]

What changes were proposed in this pull request?

To ensure safe upgrades and downgrades (rollback) when introducing lifecycle, we need to integrate with the OM Upgrade Framework.

Add OM Layout Feature: Introduce a new OMLayoutFeature (e.g., LIFECYCLE_SUPPORT) to manage the versioning of new lifecycle table.
Block APIs in Pre-finalized State: Reject requests to new lifecycle APIs if the OM is not yet finalized. This prevents state corruption if a rollback occurs

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-14347

How was this patch tested?

CI Pass: https://github.com/YutaLin/ozone/actions/runs/20869524864

[chungen0126]

Thanks @YutaLin. Please add some tests for these changes. Also, HDDS-14359 might block this PR because the smoke tests require environment updates for the 2.1.0 to 2.2.0 upgrade.

[xichen01]

@YutaLin Thanks for your patch.

I may be missing something, but do we need to upgrade the framework for the lifecycle feature?

My understanding is the lifecycle feature only adds a few DB definition and interfaces without altering the existing DB field. If we roll back after upgrading, it might leave some useless data in the DB, but it shouldn't cause data to become unreadable.

[chungen0126]

@YutaLin Thanks for your patch.

I may be missing something, but do we need to upgrade the framework for the lifecycle feature?

My understanding is the lifecycle feature only adds a few DB definition and interfaces without altering the existing DB field. If we roll back after upgrading, it might leave some useless data in the DB, but it shouldn't cause data to become unreadable.

Thanks @xichen01 for the review. You are right that we don't change existing fields, but I'm concerned about skipping the upgrade process for a couple of reasons:

1. Based on my understanding of the design doc. See https://github.com/apache/ozone/blob/master/hadoop-hdds/docs/content/design/upgrade-dev-primer.md?plain=1#L76-L81, adding a new Column Family (CF) must happen during the ON_FINALIZE stage.
2. In my opinion, It is risky If we write lifecycle data and then rollback, some "ghost data" will stay in RocksDB. When we upgrade again later, this data might start the lifecycle check and delete things without notice.

I think it's safer to follow the upgrade framework here. Let me know your thoughts!