Skip to content

perf: stream HAR entries incrementally - #17

Merged
Arkptz merged 4 commits into
mainfrom
perf/streaming-har
Apr 22, 2026
Merged

perf: stream HAR entries incrementally#17
Arkptz merged 4 commits into
mainfrom
perf/streaming-har

Conversation

@Arkptz

@Arkptz Arkptz commented Apr 22, 2026

Copy link
Copy Markdown
Owner

Summary

Replace the monolithic har::from_slice parser with a streaming HAR entries reader that iterates entries one at a time via a custom JSON scanner. HAR files from long browser DevTools sessions can exceed 100 MiB; this change keeps memory usage constant regardless of input size, matching the streaming behavior already in place for mitmproxy flow files.

  • Add stream_har_file() that opens a BufReader, scans forward to the "entries": [ array, and yields each entry object individually via bracket-balanced extraction + serde_json::from_slice
  • Replace the temporary read_har_file shim in stream_input with the new streaming iterator
  • Keep read_har_file as a thin convenience wrapper (stream_har_file().collect())
  • Own minimal StreamingHarEntry structs with #[serde(default)] for resilience against incomplete HARs

Type of Change

  • Bug fix (non-breaking)
  • New feature (non-breaking)
  • Breaking change (fix or feature that would cause existing behavior to change)
  • Documentation update
  • Refactoring / internal cleanup
  • CI / build / tooling

Checklist

  • cargo fmt --all clean
  • cargo clippy --all-targets --all-features -- -D warnings clean
  • cargo test passes locally
  • Added or updated tests for the change
  • Updated README / CLI help text if user-facing behavior changed
  • Added entry to CHANGELOG.md under [Unreleased] (if user-facing)
  • Used conventional commit style in commit messages (feat:, fix:, docs:, etc.)

Testing

  • All 8 existing HAR tests continue to pass with identical output
  • stream_har_does_not_materialize_all: constructs a 20-entry HAR, verifies the iterator yields the first entry before consuming all entries
  • stream_matches_read_for_fixtures: runs all three HAR fixtures through both stream_har_file().collect() and read_har_file(), asserts field-by-field equality
  • stream_malformed_entry_returns_error: verifies a bad entry mid-stream surfaces as Err without aborting subsequent valid entries
  • stream_empty_entries_array: verifies "entries": [] yields zero items without error
  • cargo test --test security passes (5/5)
  • cargo publish --dry-run succeeds

Related Issues

None

@Arkptz
Arkptz merged commit 30c88ba into main Apr 22, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant