Skip to content

Commit 3cacfdb

Browse files
committed
fix(topology): clarify disk buffer lifecycle messages
List each lifecycle counter event in the changelog with when it fires and what it means. Explain that unavailable buffer usage means a disk buffer writer was busy. Remove a semicolon from the unreferenced buffer warning and pass the usage snapshot through directly when emitting the replacement event.
1 parent c3c6e32 commit 3cacfdb

3 files changed

Lines changed: 15 additions & 23 deletions

File tree

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
1-
Vector now detects unreferenced disk buffers and reports disk-buffer lifecycle events during configuration reloads through logs and the `topology_disk_buffer_lifecycle_events_total` counter.
1+
Vector now reports disk buffer lifecycle events during startup and configuration reloads. Each event is logged and increments the `topology_disk_buffer_lifecycle_events_total` counter with an `event` tag:
2+
3+
- `removed`: A removed sink still had data in its disk buffer, or Vector could not read how much data it held. The log says whether the sink's current instance keeps draining the buffer in the background or whether the reload waits for it to finish.
4+
- `replacement_unreferenced`: A changed sink no longer uses its previous disk buffer. If the reload succeeds, the new sink will not read any data left in the old buffer.
5+
- `unreferenced`: A disk buffer in the data directory is not used by the current configuration. It may still be draining through a detached sink instance. If not, any data it holds is not being processed.
26

37
authors: Jansen-w

‎src/internal_events/disk_buffer.rs‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -26,10 +26,10 @@ impl InternalEvent for TopologyDiskBufferRemoved<'_> {
2626
"Removing sink with attached disk buffer containing buffered bytes. Configuration reload will wait for its current instance to drain the buffer."
2727
}
2828
(false, true) => {
29-
"Removing sink with attached disk buffer whose buffered-byte count is unavailable. Its current instance will continue draining in the background until the buffer is empty or Vector shuts down."
29+
"Removing sink with attached disk buffer. A disk buffer writer was busy writing or waiting for free space, so the buffer's usage could not be read. Its current instance will continue draining in the background until the buffer is empty or Vector shuts down."
3030
}
3131
(false, false) => {
32-
"Removing sink with attached disk buffer whose buffered-byte count is unavailable. Configuration reload will wait for its current instance to drain the buffer."
32+
"Removing sink with attached disk buffer. A disk buffer writer was busy writing or waiting for free space, so the buffer's usage could not be read. Configuration reload will wait for its current instance to drain the buffer."
3333
}
3434
};
3535
if let Some(usage) = self.usage {
@@ -90,7 +90,7 @@ impl InternalEvent for TopologyDiskBufferReplacementUnreferenced<'_> {
9090
buffer_dirs = ?self.buffer_dirs,
9191
new_buffer_dirs = ?self.new_buffer_dirs,
9292
usage_unavailable = true,
93-
message = "Replacement sink does not use the previous disk buffer, and its buffered-byte count is unavailable. If reload succeeds, any buffered bytes will be left unreferenced.",
93+
message = "Replacement sink does not use the previous disk buffer. A disk buffer writer was busy writing or waiting for free space, so the buffer's usage could not be read. If reload succeeds, any buffered bytes will be left unreferenced.",
9494
),
9595
}
9696
counter!(
@@ -112,7 +112,7 @@ impl InternalEvent for TopologyDiskBufferUnreferenced<'_> {
112112
warn!(
113113
buffer_id = self.buffer_id.to_string_lossy().as_ref(),
114114
buffer_dir = self.buffer_dir.to_string_lossy().as_ref(),
115-
message = "Found disk buffer not referenced by the current configuration. It may still be draining through a detached sink instance; otherwise its buffered bytes are not being processed.",
115+
message = "Found disk buffer not referenced by the current configuration. It may still be draining through a detached sink instance. If not, any buffered bytes it holds are not being processed.",
116116
);
117117
counter!(
118118
CounterName::TopologyDiskBufferLifecycleEventsTotal,

‎src/topology/running.rs‎

Lines changed: 6 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -725,24 +725,12 @@ impl RunningTopology {
725725
.iter()
726726
.any(|path| new_buffer_dirs.contains(path))
727727
{
728-
match usage {
729-
Some(usage) => {
730-
emit!(TopologyDiskBufferReplacementUnreferenced {
731-
component_id: key,
732-
buffer_dirs: &buffer_dirs,
733-
new_buffer_dirs: &new_buffer_dirs,
734-
usage: Some(usage),
735-
});
736-
}
737-
None => {
738-
emit!(TopologyDiskBufferReplacementUnreferenced {
739-
component_id: key,
740-
buffer_dirs: &buffer_dirs,
741-
new_buffer_dirs: &new_buffer_dirs,
742-
usage: None,
743-
});
744-
}
745-
}
728+
emit!(TopologyDiskBufferReplacementUnreferenced {
729+
component_id: key,
730+
buffer_dirs: &buffer_dirs,
731+
new_buffer_dirs: &new_buffer_dirs,
732+
usage,
733+
});
746734
}
747735
}
748736

0 commit comments

Comments
 (0)