Expose batch decode info#150
Open
jakubdyszkiewicz wants to merge 1 commit into
Open
Conversation
Contributor
|
It seems the maintainers don't have much time for pull requests. |
Author
|
I'll wait until #151 is resolved. |
Contributor
|
FYI — ported to aovestdipaperino/proto-kafka (a downstream fork) at https://github.com/aovestdipaperino/proto-kafka/tree/port/upstream-150-batch-decode-info. Thanks for the work on the original. |
aovestdipaperino
added a commit
to aovestdipaperino/proto-kafka
that referenced
this pull request
May 19, 2026
…+ decode_batch_info
aovestdipaperino
added a commit
to aovestdipaperino/proto-kafka
that referenced
this pull request
May 19, 2026
First documented release of this fork since the divergence from
tychedelia/kafka-protocol-rs. Bundles three upstream PR ports and a
build fix that had been outstanding.
See CHANGELOG.md for the full notes; in short:
- security: bound Array/CompactArray decoder up-front capacity
(upstream PR kafka-protocol-rs#152)
- added: pub BatchDecodeInfo + RecordBatchDecoder::decode_batch_info
for header-only telemetry (upstream PR kafka-protocol-rs#150)
- changed: v2 record batch CRC32C now via crc-fast for CLMUL
acceleration (upstream PR kafka-protocol-rs#155)
- fixed: pin string git dep to rev 328939d so the build resolves
against bytes 0.5
All three workspace crates (franz, franz_protocol, xtask) move to
0.1.1 in lockstep.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hey 👋
In our project that uses
kafka-protocol-rswe have a use case to provide telemetry on the number of records in the record batch. Telemetry is assembled in a completely different phase than handling the records, so I cannot rely on the decoded batch. I also don't want to decode the batch twice because it's too expensive. Right now, we hacked aroundfn count_records(batches: &Bytes) -> i32function that extracts record count from the batches. It would be great if, instead of decoding this info ourselves, we could just use what was already done in kafka-protocol-rs.Exposing BatchDecodeInfo and
decode_batch_infowould be a nice solution for us.