[Spark] Throw exception when validation fails in ALTER COLUMN AFTER

[yaoforx]

Which Delta project/connector is this regarding?

• Spark
• Standalone
• Flink
• Kernel
• Other (fill in here)

Description

This change series adds validation to ensure clustering columns remain in the statistics schema when executing ALTER TABLE ALTER COLUMN col AFTER position commands on clustered Delta tables. Without proper stats schema handling, column reordering can cause clustering columns to lose statistics collection due to position-based indexing rules, degrading query optimization performance.

How was this patch tested?

The test suite verifies the following scenarios:

1. checkEnabled=false (default):

   • First ALTER COLUMN AFTER succeeds (c1 moves after c3)
   • Second ALTER COLUMN AFTER succeeds (c2 moves after c3)
   • No validation errors thrown
   • Column order changes as expected

2. checkEnabled=true:

   • First ALTER COLUMN AFTER succeeds (c1 moves after c3)
   • Second ALTER COLUMN AFTER fails with DELTA_CLUSTERING_COLUMN_MISSING_STATS error
   • Clustering column c2 would lose stats collection
   • Original schema and stats remain unchanged

Does this PR introduce any user-facing changes?

[rahulsmahadev]

Thanks for the contribution, can you also update PR description to capture what we are doing

[yaoforx]

Thanks for the contribution, can you also update PR description to capture what we are doing

Done

[zedtang]

lgtm

[rahulsmahadev]

LGTM