[Kernel-Spark][DSv2] Wireframe for UCManagedSnapshotManager

[TimothyW553]

🥞 Stacked PR

Use this link to review incremental changes.

• catalogtableutils-ccv2 [Files changed]
  • stack/ccv2-uc-utils [Files changed]
    • stack/snapshotmanager-wireframe [Files changed]
      • stack/snapshotmanager-implementation [Files changed]
        • stack/snapshotmanager-factory-wireup [Files changed]

---

Which Delta project/connector is this regarding?

• Spark
• Standalone
• Flink
• Kernel
• Other (fill in here)

Description

This PR:

• Adds kernelUnityCatalog as a dependency for sparkV2 to wire in UC-managed functionality.
• Introduces UCManagedSnapshotManager skeleton for UC-managed tables that will delegate snapshot/
  commit operations to UCCatalogManagedClient (methods currently TODO).

(Orange is the previous PR, green is what we've introduced in this PR. Routing is deferred)

flowchart TB
    A["Spark table CatalogTable"] -- decide snapshot manager --> B{"Select snapshot manager"}
    B -- path based --> P["PathBasedSnapshotManager loadLatestSnapshot loadSnapshotAt"]
    C["UCCatalogManagedClient"] -- build UC REST client --> D["UC REST client"]
    D -- GET ratified commits and table metadata --> U["Unity Catalog server"]
    X["UCUtils extractTableInfo<br>(tableId, path, uri, token)"] -- needs UC table details --> M["UCManagedSnapshotManager<br>(UCCatalogManagedClient(UCClient))"]
    B -- UC managed --> X
    M --> C

     C:::added
     X:::extraction
     M:::added
    classDef extraction fill:#ffb347,stroke:#d97706,color:#000
    classDef added fill:#b7e1a1,stroke:#2d6a4f,color:#000

Loading

How was this patch tested?

Locally and CI.

Does this PR introduce any user-facing changes?

No.

[huan233usc]

Plz update the description

[huan233usc]

LGTM

[raveeram-db]

Just a nit, looks great overall!