Skip to content

Commit

Permalink
[#25335] docdb: Vector index: DFATAL crash during copying of large rows
Browse files Browse the repository at this point in the history
Summary:
The issue happens when inserted row is larger than a configured packed row limit (32_KB). In this
case the remaining part of the row is being inserted as a non-packed row. When such a row is being
processed, the vector index inserter does not check that a vector’s value can be null for a
packed row and tries to insert it into the vector index, which lead to the insertion failure at
vector index side and subsequent DFATAL.

This change adjusts the logic of packed rows insertion into a vector index: before the insertion
into a vector index, it now checks if vector's value is not a null value.
Jira: DB-14548

Test Plan: Jenkins

Reviewers: slingam, sergei

Reviewed By: slingam, sergei

Subscribers: ybase

Differential Revision: https://phorge.dev.yugabyte.com/D40750
  • Loading branch information
arybochkin committed Dec 18, 2024
1 parent e5a8422 commit eac29fa
Show file tree
Hide file tree
Showing 3 changed files with 15 additions and 0 deletions.
5 changes: 5 additions & 0 deletions src/yb/docdb/rocksdb_writer.cc
Original file line number Diff line number Diff line change
Expand Up @@ -767,6 +767,11 @@ Status ApplyIntentsContext::ProcessVectorIndexesForPackedRow(
continue;
}
auto column_value = decoder.FetchValue(vector_index.column_id());
if (column_value.IsNull()) {
VLOG_WITH_FUNC(3) << "Ignoring null vector value for key '" << key.ToDebugHexString() << "'";
continue;
}

vector_index_batches_[i].push_back(VectorIndexInsertEntry {
.key = KeyBuffer(key.WithoutPrefix(table_key_prefix.size())),
.value = ValueBuffer(*column_value),
Expand Down
2 changes: 2 additions & 0 deletions src/yb/docdb/vector_index.cc
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,8 @@ template<vector_index::IndexableVectorType Vector>
Result<vector_index::VectorLSMInsertEntry<Vector>> ConvertEntry(
const VectorIndexInsertEntry& entry) {

RSTATUS_DCHECK(!entry.value.empty(), InvalidArgument, "Vector value is not specified");

auto encoded = dockv::EncodedDocVectorValue::FromSlice(entry.value.AsSlice());
return vector_index::VectorLSMInsertEntry<Vector> {
.vertex_id = VERIFY_RESULT(encoded.DecodeId()),
Expand Down
8 changes: 8 additions & 0 deletions src/yb/dockv/packed_value.h
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,10 @@ class PackedValueV1 {

bool IsNull() const;

std::string ToString() const {
return Format("{ value: $0 }", value_.ToDebugHexString());
}

static PackedValueV1 Null();

private:
Expand Down Expand Up @@ -79,6 +83,10 @@ class PackedValueV2 {
return !value_.data();
}

std::string ToString() const {
return Format("{ value: $0 }", value_.ToDebugHexString());
}

static PackedValueV2 Null();

private:
Expand Down

0 comments on commit eac29fa

Please sign in to comment.