Skip to content

feat(grpc): upgrade to Tonic 0.12 and native HTTP 1 types - #26770

Merged
thomasqueirozb merged 11 commits into
masterfrom
tonic-3-http1-cutover
Oct 21, 2026
Merged

thomasqueirozb merged 11 commits into
masterfrom
tonic-3-http1-cutover

Conversation

@thomasqueirozb

@thomasqueirozb thomasqueirozb commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Upgrade the workspace-owned Tonic stack from 0.11 to 0.12, Prost to 0.13, and the affected Axum users to 0.7. Keep external SDK-owned Tonic versions unchanged.
  • Move the shared gRPC server, Vector source/sink, OTLP source, Pub/Sub source, observability API/client, validation runner, and affected HTTP test harnesses to native HTTP 1 types. The Vector sink uses the HTTP/2 client from part 1.
  • Keep compression ownership, bounded body forwarding, trailers, acknowledgements, message-size settings, middleware ordering, and the previous 16 MiB HTTP/2 header limit, now one shared crate::http::GRPC_MAX_HEADER_LIST_SIZE. Protobuf schemas and configuration do not change.
  • Fix issues found by the runtime checks: the Vector sink now sends forward-proxy credentials and applies tls.server_name through CONNECT proxies; Pub/Sub classifies stream resets again and trusts every CA in tls.ca_file.
  • The vector and opentelemetry sources now answer unregistered gRPC services with grpc-status: 12 instead of HTTP 404. Tonic 0.12 adds the UNIMPLEMENTED fallback to Routes::default(), which these sources use; Tonic 0.11 had none, so each such request logged an error and incremented component_errors_total.
  • The Pub/Sub shutdown fix and the vector top key-capture fix are not in this PR. They fix bugs already on master and go in separate PRs.
  • Update dependency licenses and remove obsolete advisory exceptions for dependencies removed by this cutover.

This is part 3 of the Tonic upgrade. Part 4 handles the separate Tonic 0.14 body API and protobuf codec/generator split.

The HTTP crate versions change; gRPC remains HTTP/2 on the wire.

References

Vector configuration

The real-process checks used temporary data directories, ephemeral loopback ports, YAML configurations, and controlled local peers. Representative receiver configuration:

proxy:
  enabled: false
api:
  enabled: true
  address: 127.0.0.1:8686
sources:
  receiver:
    type: vector
    address: 127.0.0.1:6000
sinks:
  console:
    type: console
    inputs: [receiver]
    target: stdout
    encoding:
      codec: json
    acknowledgements:
      enabled: true

Separate scenarios exercised verified TLS, authenticated forward/CONNECT/TLS proxies, OTLP gRPC and HTTP producers, an official pinned Pub/Sub emulator, a real pinned Datadog trace agent, and the antithesis oracle. Pub/Sub TLS used a SAN-valid localhost certificate, a two-CA bundle with its issuer first, and an empty platform trust-store control.

How did you test this PR?

  • Preserved Tonic 0.11 binaries interoperated in both directions with the native binary. The runs delivered 2,400 exact typed events over plaintext and verified TLS with none/gzip/zstd compression, passed 60 size/corruption/early-response cases, and delivered 60 AppSignal events.
  • Native transport checks passed authenticated proxy routes, bypass rules, TLS/authentication rejection, 24 KiB response headers and trailers, acknowledgement retry/rejection, idle PINGs, dead-connection eviction, restart, connection ageing, and shutdown.
  • OTLP checks passed exact logs/metrics/traces over both protocols, gzip, sink error/rejection status mappings, shared admission limits, request timeout and recovery, and shutdown.
  • Pub/Sub checks passed 101 exact messages, including a 4,718,592-byte message, keepalives, CA/hostname/ALPN validation, immediate remote-reset reconnect, delayed connection-abort retry, and acknowledgements with a redelivery control.
  • API checks passed shared-port HTTP/1.1 and h2c health, 24 KiB metadata, v1alpha reflection, streaming/cancellation, exact tap delivery, and counters. Drain health transitions and shutdown passed.
  • Unregistered-service requests to the vector and opentelemetry gRPC listeners return HTTP 200 with grpc-status: 12 and log no error. Master returns HTTP 404 and logs Grpc error. for the same requests.
  • All seven validation topologies passed. The real Datadog trace-agent comparison passed. The native oracle accepted eight exact deterministic payloads in a 3,539,121-byte NDJSON request and rejected malformed input.
  • All 40 protobuf schema inputs match the pre-stack checksums. An all-feature dependency graph has no Tonic 0.11 and no Tonic-family path to http 0.2, http-body 0.4, or Hyper 0.14. SDK-owned Tonic versions are unchanged.

Installed-release interoperability

Ran five Vector processes on macOS arm64. A, C, and E used the installed /opt/homebrew/bin/vector version 0.60.0 (0d0b358). B and D used a fresh branch build, 0.61.0-dev at 854900dab1, built with cargo vdev run and the relay configuration's minimum features. Every link used a Vector sink and source over plaintext loopback.

flowchart LR
    A["A: installed"] --> B["B: branch"]
    B --> C["C: installed"]
    C --> D["D: branch"]
    D --> E["E: installed"]
Loading
Compression on all links Events sent Events received at E Lost / duplicated / changed
None 256 256 0 / 0 / 0
gzip 256 256 0 / 0 / 0
zstd 256 256 0 / 0 / 0

Compared every event and field, including nested objects, arrays, Unicode, nulls, booleans, floats, signed 64-bit limits, nanosecond timestamps, and 64 KiB payloads. Canonical JSON output checksums matched the expected data in all three runs. All processes exited with code 0 and logged no runtime errors.

Vector sink acknowledgements were enabled throughout. The only warning was that A's stdin source cannot acknowledge its producer; this check does not establish upstream durability. This scenario covered JSON logs, not metrics or TLS.

Temporary runtime drivers, credentials, certificates, and generated Rust stubs are not part of this PR.

Does this PR include user facing changes?

  • Yes. Changelog fragments cover the Vector sink proxy fix, the Pub/Sub reset and CA fixes, and the unregistered-service response.
  • No. A maintainer will apply the no-changelog label to this PR.

@github-actions github-actions Bot added domain: sources Anything related to Vector's sources domain: sinks Anything related to Vector's sinks labels Oct 21, 2026
@thomasqueirozb
thomasqueirozb marked this pull request as ready for review October 21, 2026 03:46
@thomasqueirozb
thomasqueirozb requested review from a team as code owners October 21, 2026 03:46
@thomasqueirozb
thomasqueirozb added this pull request to stack #26767 October 21, 2026 03:46
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-21T19:59:36.286011Z 854900d New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@datadoghq-integration datadoghq-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

Static review identified no concrete regression in transport integration, body forwarding, acknowledgements, or Pub/Sub lifecycle and TLS handling. Runtime behavior remains unverified.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 30fcc31

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c930517cc3

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/sources/gcp_pubsub.rs Outdated
Comment thread src/api/grpc_server.rs
Base automatically changed from tonic-2-decompression-prep to master October 21, 2026 18:11

@datadoghq-integration datadoghq-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: FAIL

HTTPS API connections lose automatic TLS configuration after the Tonic upgrade. Pub/Sub also rejects usable CA bundles when native certificate loading reports a recoverable directory error.

Open Bits AI session

🤖 Bits Code Review · Commit c930517

Comment thread lib/vector-api-client/src/client.rs
Comment thread src/sources/gcp_pubsub.rs Outdated
Comment thread lib/vector-common/src/constants.rs
Comment thread src/api/grpc_server.rs

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 854900dab1

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread deny.toml
@thomasqueirozb
thomasqueirozb added this pull request to the merge queue Oct 21, 2026
Merged via the queue into master with commit 96121cc Oct 21, 2026
201 checks passed
@thomasqueirozb
thomasqueirozb deleted the tonic-3-http1-cutover branch October 21, 2026 21:21
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 21, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

domain: sinks Anything related to Vector's sinks domain: sources Anything related to Vector's sources

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants