Repository navigation
Conversation
Contributor
|
All contributors have signed the CLA ✍️ ✅ |
missedone
force-pushed
the
feat/victoriametrics
branch
from
September 24, 2026 23:02
8c6dfd6 to
b15f7fe
Compare
Author
|
I have read the CLA Document and I hereby sign the CLA |
missedone
marked this pull request as draft
September 25, 2026 06:37
Record the design of a `victoriametrics` sink for vectordotdev#1343: the VictoriaMetrics remote write protocol (zstd), `vmrange` histograms for distributions, bounded state for incremental distributions, tenancy for single-node, vminsert, and vmauth, and retry behavior matching vmagent.
Add a sink that delivers metrics to VictoriaMetrics using the VictoriaMetrics remote write protocol, matching the vmagent remote write client: - Encode `WriteRequest` protobuf directly into a reused buffer and compress each request with a single zstd call, so the frame records its content size. Labels are written sorted by name. - Bound batches by estimated encoded size, 8 MiB by default like `-remoteWrite.maxBlockSize`, so requests stay below `-maxInsertRequestSize`. - Encode distributions as `vmrange` histograms using the bucket layout of VictoriaMetrics/metrics, verified against golden data generated from the Go implementation, and accumulate incremental distributions in bounded, sink-owned state. - Encode sketches as summaries with `_sum`, `_count`, `_min`, and `_max`, as VictoriaMetrics does for Datadog sketches. - Support single-node VictoriaMetrics, cluster tenants in the vminsert URL path or in `vm_account_id`/`vm_project_id` labels (including the metadata tenant fields), and per-event vmauth credentials through the `victoriametrics_token` event secret. - Drop requests only on 400, 409, and 415 and retry everything else, and healthcheck by writing an empty request to the write path. - Send metric metadata by default, matching `-enableMetadata`. Closes vectordotdev#1343
Run the `victoriametrics` sink integration tests against single-node VictoriaMetrics, a vmstorage/vminsert/vmselect cluster, and vmauth routing bearer tokens to tenants.
Add the component reference, generated configuration docs, example configurations, website page, and changelog fragment for the `victoriametrics` sink.
missedone
force-pushed
the
feat/victoriametrics
branch
from
September 27, 2026 22:31
b15f7fe to
20404f8
Compare
missedone
marked this pull request as ready for review
September 29, 2026 20:06
missedone
force-pushed
the
feat/victoriametrics
branch
from
October 17, 2026 20:55
f462b39 to
8b45bbf
Compare
missedone
force-pushed
the
feat/victoriametrics
branch
from
October 19, 2026 19:07
8b45bbf to
10b895f
Compare
missedone
force-pushed
the
feat/victoriametrics
branch
from
October 19, 2026 19:09
10b895f to
34508e2
Compare
Retry body read failures, route tenants before normalizing, reject reserved headers, skip empty sketches, key metadata by name and type, tighten TTL cleanup, drop the tenant from the endpoint tag, and share histogram snapshots.
Match vmagent by dropping labels with empty values, and cover retry behaviour and encoding edge cases end to end.
Add vmagent and a reference VictoriaMetrics to the integration environment, and a test that the sink stores the same series, labels, values and timestamps as vmagent for the same data.
Allow the VictoriaMetrics identifier in docs, justify the intentional numeric casts, use let-else in tenant routing, split the vmagent integration test, and point the generated acknowledgements schema at the bool_or_struct definition like other sinks.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a
victoriametricssink that sends metrics using theVictoriaMetrics remote write protocol:
a Prometheus
WriteRequestprotobuf compressed with zstd, as sent byvmagent. The design isin the RFC included in this PR (
rfcs/2026-09-24-1343-victoriametrics-sink.md).prometheus_remote_writeworks with VictoriaMetrics today, but it falls short in several ways:tenant_idsetsX-Scope-OrgID, which VictoriaMetrics ignores, so cluster tenants need onesink per tenant.
bucketslist, which is imprecise and doesn't line upwith the
vmrangehistograms that VictoriaMetrics client libraries emit.MetricValue::addconcatenatessamples), so memory and CPU grow without bound (likely High CPU usage on
prometheus_remote_writesink #19041).What the sink does:
Protocol: zstd with
X-VictoriaMetrics-Remote-Write-Version: 1. The protobuf is encodeddirectly into a reused buffer, with no intermediate prost structs or grouping map, and each
request is compressed in a single zstd call, so the frame records its content size, as vmagent
does.
Request size:
batch.max_byteslimits the estimated encoded size, 8 MiB by default like-remoteWrite.maxBlockSize, so distributions that expand into hundreds of series can't push arequest over
-maxInsertRequestSize.Distributions: encoded as
vmrangehistograms (18 log-scale buckets per power of ten), usingthe bucket layout of
VictoriaMetrics/metrics. The labels are checked byte for byte againstgolden data generated from the Go implementation. Incremental distributions are accumulated in
bounded, sink-owned state: at most 488 counters per series, expired by
expire_metrics_secs.Sketches: sent as summaries with quantiles plus
_sum,_count,_minand_max, the wayVictoriaMetrics stores Datadog sketches.
Tenancy, set by
tenant.mode:none: single-node VictoriaMetrics, or vmauth.path:/insert/<accountID:projectID>/prometheus/api/v1/write, one batch per tenant.labels:vm_account_id/vm_project_idlabels, with the tenant also set in theMetricMetadatafields vminsert reads.tenant.idis subject to template confinement, and the rendered value must be a valid numerictenant, so event data can't change the URL path.
Per-event credentials for vmauth: an event secret
victoriametrics_tokenis sent as a bearertoken in place of the configured
auth, following thedatadog_api_key/splunk_hec_tokenpattern. Batches are split per token.
Retries match vmagent: requests are dropped only on 400, 409 and 415; every other failure is
retried, including 401/403 from vmauth while credentials are rotated.
Healthcheck writes an empty request to the write path, which checks reachability,
authentication and vmauth routing, not just
/health.Metadata is sent by default (
send_metadata: true), matching-enableMetadata.The commits are grouped for review: RFC, sink and unit tests, integration test environment and
CI, then docs.
References
Closes: #1343
Related: #13890, #19041, #23990
Vector configuration
How did you test this PR?
Unit and end-to-end tests (49) in
src/sinks/victoriametrics/:vmrangelabels and bucket placement against golden data from Go;Integration tests (
cargo vdev int test victoriametrics) cover:Data is read back with
/api/v1/export.Checked against the
vmagentremote write client and thevminsertremote write parser in theVictoriaMetrics source (headers, block limits, status handling, tenant parsing, metadata tenant
fields).
make fmt, clippy,cargo vdev check events,make check-docs, generated docs, changelogfragment check, typos, markdownlint.
Does this PR include user facing changes?
no-changeloglabel to this PR.