Repository navigation
Reject CreateColumnFamilyWithImport with blob files - #15652
xingbowang wants to merge 1 commit into
Conversation
Summary: Why: CreateColumnFamilyWithImport only brings SST files, never blob files. If the imported SSTs hold blob references, the new column family points to blob files that do not exist there. Reads of those keys fail later, and the problem is not visible at import time. Previous behavior: import accepted both of these cases: - The new column family has `enable_blob_files = true`. - Imported files reference blob files, for example an export (Checkpoint::ExportColumnFamily) of a BlobDB column family. Import also wrote kInvalidBlobFileNumber as the new file's oldest blob file number, which hid the link while the blob indexes stayed in the SST. New behavior: CreateColumnFamilyWithImport returns NotSupported in both cases, before it creates the column family. So a rejected import leaves no column family behind. A normal import does not change. Implementation: - DBImpl::CreateColumnFamilyWithImport checks `options.enable_blob_files`, and checks each imported file's `oldest_blob_file_number` in the metadata loop that already exists. The checks read only metadata (no file reads), cost O(number of files), and run before any DB mutex is taken. - Doc line on DB::CreateColumnFamilyWithImport in include/rocksdb/db.h. - Release note under unreleased_history/behavior_changes. Limits and unsupported cases: - The check trusts the metadata. Hand-built metadata that leaves `oldest_blob_file_number` at 0 for an SST that holds blob indexes is not detected, because the check does not read data. SstFileWriter cannot write blob indexes, so its files are not affected. - Checkpoint::ExportColumnFamily is not changed. It still exports a column family whose SSTs reference blob files, but it copies only the SSTs. After this change such an export can never be imported. - `enable_blob_files` can still be turned on with SetOptions after import. New blob files then belong to the new column family, so no reference dangles. Changes visible to existing users: BlobDB users who import into a blob column family, or who import an export of a BlobDB column family, now get NotSupported. Before, they got a column family with dangling blob references. Differential Revision: D131129385
|
@xingbowang has exported this pull request. If you are a Meta employee, you can view the originating Diff in D131129385. |
✅ clang-tidy: No findings on changed linesCompleted in 330.2s. |
✅ Claude Code ReviewAuto-triggered after CI passed — reviewing commit 8998980 SummaryClean, well-scoped defensive change that correctly rejects No high-severity findings. Full review (click to expand)Findings🔴 HIGHNone. 🟡 MEDIUMM1. Doc comment placement covers only the single-metadata overload --
|
| Context | Does code execute? | Assumptions hold? | Action needed? |
|---|---|---|---|
| WritePreparedTxnDB | Yes (via PessimisticTransactionDB) | Yes - delegates to DBImpl | Safe |
| ReadOnly DB | No - returns NotSupported independently | N/A | Safe |
| CompactedDB | No - returns NotSupported independently | N/A | Safe |
| StackableDB | Yes - delegates to underlying DB | Yes | Safe |
| C API | Yes - calls single-metadata overload | Yes - covered | Safe |
| Java JNI | Yes - calls single-metadata overload | Yes - covered | Safe |
DB Mutex Analysis: The new checks execute at lines ~8776-8794, before the first mutex acquisition at line 8809. No DB mutex is held during the new validation. No impact on critical section duration.
Behavioral Compatibility:
- Old behavior: Import silently succeeded, creating a CF with dangling blob references that would cause read failures later.
- New behavior: Import returns
NotSupportedearly, before creating the CF. No orphan CF is left behind. - Impact assessment: Users who were importing BlobDB exports would see a new error. However, the old behavior was already broken (reads would fail), so this is a strict improvement. The release note accurately describes the change.
Positive Observations
-
Correct ordering: Both checks are placed before
CreateColumnFamily(), so a rejected import leaves no orphan column family behind. This avoids the cleanup path (DropColumnFamily/DestroyColumnFamilyHandle) that the function uses for later failures. -
Consistent with existing pattern: The import job already wrote
kInvalidBlobFileNumberat line 253 ofimport_column_family_job.cc, silently discarding blob references. The new checks make this rejection explicit and early. -
Test quality: The test covers both rejection paths (enable_blob_files option and metadata with blob refs), verifies the successful import still works after a rejected one, verifies error message content, and checks that rejected imports leave no CF behind. The test reuses existing fixture members (
export_files_dir2_,metadata_ptr2_,import_cfh2_) that are properly cleaned up by the destructor. -
Release note: Concise and accurate, describes both old and new behavior clearly.
-
Scope discipline: The PR correctly limits itself to the import path and does not attempt to change ExportColumnFamily or other related APIs.
ℹ️ About this response
Generated by Claude Code.
Review methodology: claude_md/ci_review_prompt.md
Limitations:
- Claude may miss context from files not in the diff
- Large PRs may be truncated
- Always apply human judgment to AI suggestions
Commands:
/claude-review [context]— Request a code review/claude-query <question>— Ask about the PR or codebase
Summary:
Why: CreateColumnFamilyWithImport only brings SST files, never blob files.
If the imported SSTs hold blob references, the new column family points to
blob files that do not exist there. Reads of those keys fail later, and the
problem is not visible at import time.
Previous behavior: import accepted both of these cases:
enable_blob_files = true.(Checkpoint::ExportColumnFamily) of a BlobDB column family. Import also
wrote kInvalidBlobFileNumber as the new file's oldest blob file number,
which hid the link while the blob indexes stayed in the SST.
New behavior: CreateColumnFamilyWithImport returns NotSupported in both
cases, before it creates the column family. So a rejected import leaves no
column family behind. A normal import does not change.
Implementation:
options.enable_blob_files,and checks each imported file's
oldest_blob_file_numberin the metadataloop that already exists. The checks read only metadata (no file reads),
cost O(number of files), and run before any DB mutex is taken.
Limits and unsupported cases:
oldest_blob_file_numberat 0 for an SST that holds blob indexes is notdetected, because the check does not read data. SstFileWriter cannot write
blob indexes, so its files are not affected.
family whose SSTs reference blob files, but it copies only the SSTs. After
this change such an export can never be imported.
enable_blob_filescan still be turned on with SetOptions after import.New blob files then belong to the new column family, so no reference
dangles.
Changes visible to existing users: BlobDB users who import into a blob
column family, or who import an export of a BlobDB column family, now get
NotSupported. Before, they got a column family with dangling blob references.
Differential Revision: D131129385