sql/catalog: support batched descriptor writes with deferred validation

[spilchen]

Summary

The catalog's WriteDescriptors API writes all descriptors in a single kv.Batch, which exceeds the Raft command size limit (kv.raft.command.max_size, default 64 MiB) when the descriptor count is large. At ~1.3 KiB per descriptor proto, 100k descriptors produce a ~130 MB batch. This blocks the Disaster Recovery team's 100k/1M descriptor restore work.

See #171338 for the full problem description.

Background

WriteDescriptors (write_descs.go:63) builds a single kv.Batch for every database, schema, table, type, and function descriptor plus their system.namespace entries, then commits via txn.Run(ctx, b). Naive chunking into separate transactions is unsafe because:

• Cross-database reference validation (checkForCrossDatabaseReferences) runs across the entire written set before commit.
• Follow-on writes in the same transaction assume all descriptors are visible.
• A partial commit would leave dangling OFFLINE descriptors.

publishDescriptors has the same shape — it writes the full descriptor set (OFFLINE → PUBLIC transition) in a single batch and will hit the same limit.

Proposed Approach

As discussed in #171338, the catalog should expose an API that:

1. Ingests descriptors in multiple batches within a single transaction, splitting writes into Raft-command-sized chunks rather than one monolithic batch.
2. Skips per-batch validation — individual batches are not independently validated.
3. Runs validation once at the transaction level after all batches have been written, ensuring cross-descriptor reference integrity across the full set.
4. (Optionally) supports flipping all descriptors from OFFLINE → PUBLIC after validation, covering the publishDescriptors path as well.

This keeps single-transaction atomicity (safe rollback on failure) while staying under the Raft command size ceiling.

Code References

• write_descs.go:63 — WriteDescriptors
• restore_job.go:3653-3689 — publishDescriptors
• collection.go:625 — WriteDescToBatch (CPut per descriptor)

Related Issues

• restore: WriteDescriptors batch exceeds Raft command size limit at 100k descriptors #171338 — restore: WriteDescriptors batch exceeds Raft command size limit at 100k descriptors (upstream tracking issue)
• restore: persist descriptor refs in job_info, not RestoreDetails #171336 — related system.job_info payload size fix (separate problem, already addressed)

Jira issue: CRDB-64506

Epic CRDB-62269