Proposal: CREATE OR REPLACE TABLE for the Iceberg REST catalog

[ErikBPF]

Problem

The Iceberg REST catalog returns NotSupported for CREATE OR REPLACE / REPLACE TABLE.

Proposal

Atomic replace via update_table + AssertTableUuid — load the existing table, stage schema/spec/sort/property updates, and commit in one server-side transaction. No drop, so the table UUID, location and snapshot history are preserved by construction. Stale user properties are removed; catalog-managed keys are kept.

table_uuid and current-snapshot-id continuity across a replace are asserted. Snapshot-history preservation holds by construction (update_table never drops the table) but is not exercised by tests, since the catalog layer exposes no write path to create a snapshot first.

Open questions for maintainers

1. Is update_table + AssertTableUuid the preferred mechanism (vs stage-create + commit-transaction)?
2. RemoveSchemasUpdate is omitted (the fixture rejects removing snapshot-referenced schemas) — acceptable, or should superseded schemas be pruned?
3. Property semantics: stale user props removed, catalog-managed keys kept — matches expectations?

[linhr]

Using update_table for REPLACE TABLE doesn't seem a clean solution to me. I feel a new table should get a different UUID without inheriting properties and snapshots from existing ones, but I may be wrong and I'd like to see how Spark libraries support this.

Would you like to investigate the expectation from common Iceberg REST catalogs (Nessie, Lakekeepers, AWS Glue, etc.) around REPLACE TABLE?

[ErikBPF]

Correction / update to my earlier reply — deeper testing surfaced a real bug that's directly relevant to your concern, so I've changed the implementation:

Our replace was not resetting the current snapshot, so a replaced table would still surface the OLD data — which is exactly the "inherits data from the existing table" behavior you flagged. Fixed. REPLACE now:

• Resets the current data — removes the main ref so current-snapshot-id becomes empty; the replaced table starts with no rows (per the Iceberg spec: current-snapshot-id mirrors the main ref, and -1/absent = no current snapshot). So it does not inherit the previous data.
• Keeps identity + history — same table UUID, and schema/snapshot history is retained for time-travel. This is the Iceberg CREATE OR REPLACE contract ("keeps table history"), consistent with Spark RTAS and with Sail's existing managed-path replace.
• Adds concurrency guards (AssertRefSnapshotId + AssertCurrentSchemaId) so a concurrent replace/append conflicts (409), and sanitizes catalog-managed properties.

So the net is close to your intent (no inherited live data) without discarding identity/history. If you'd still prefer a brand-new UUID (full drop+create) as the Sail default, that's a small change — but it diverges from the Iceberg spec and our managed path, so I'd flag it as an explicit decision. Which do you prefer?

(Verified live against MinIO + iceberg-rest: a written snapshot is reset to empty on replace; 29 integration tests pass.)