Repository navigation
👷 ci(fuzz): hide crashers, add a malloc pass - #970
Merged
Merged
Conversation
Merging this PR will degrade performance by 5.58%
|
| Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|
| ❌ | test_feature[transform-number-count-last] |
552.1 µs | 584.7 µs | -5.58% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing gaborbernat:fuzz/harness-hygiene (2001f89) with main (c795023)
Footnotes
-
32 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
gaborbernat
force-pushed
the
fuzz/harness-hygiene
branch
2 times, most recently
from
October 1, 2026 23:49
e391edf to
2b9c7e9
Compare
The deep fuzz job printed up to 400 bytes of each crashing input into its CI log and uploaded crash files as a plain artifact. Both are public on this repository, so the crasher of an unfixed bug went public before its fix. The driver now logs a crasher only by SHA-256, length, harness and seeds, and the workflow uploads crashers only after encrypting them with age to the key in the FUZZ_AGE_RECIPIENT repository variable. Without that variable it uploads nothing. The in-process driver now runs a second pass under PYTHONMALLOC=malloc, because pymalloc pools hide small over-reads from AddressSanitizer. The deep budget splits across the two passes, so the weekly job stays inside the six-hour limit. The fuzz environments pin PYTHONHASHSEED, which tox otherwise randomizes, so a find on a hash-dependent path replays. Past finds now live in tests/fuzz_regressions, and every run replays them first through every harness via the existing seed replay.
Every scheduled deep run used rng seed 0, so each one replayed the same time-bounded mutation sequence and explored nothing new. The job now passes its run number as the seed. The run number appears in the log, so passing it back to --rng-seed reproduces a run.
The deep run pointed the JS harness at tests/serialize/js/_corpus, which does not exist, so it never replayed a corpus file. The fixtures live in tests/js/_corpus as JSON rows, so the driver writes each row's input to its own file, the same extraction tools/js_sanitize.py uses.
The roundtrip and sanitize idempotence checks flagged two cases the library handles as designed. Script text that opens "<!--" and then "<script" before any "-->" breaks the HTML script content restrictions, and its serialization reparses in the double-escaped state; the standard warns such parser-made trees need not survive serialize and reparse. Past 512 open elements the tree builder stops nesting, so reparsing the flattened output nests it differently. Each exemption applies only after a mismatch and only to those inputs: script text matching the restriction's own grammar, or a tree that reached the depth cap. Any other non-idempotent output still fails.
gaborbernat
force-pushed
the
fuzz/harness-hygiene
branch
from
October 2, 2026 00:24
2b9c7e9 to
6238b23
Compare
The scheduled run seeded its mutations with the public run number and printed the mutation index and the full sanitizer report, so anyone could regenerate an unfixed crasher or read where it faults. The seed is now random per run and reaches fuzz.py only through the environment, the log keeps the hash, length and harness, and the origin and sanitizer reports go to the crash directory the workflow encrypts.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The deep fuzz job printed up to 400 bytes of each crashing input into its CI log, and the
📤 Upload crash corpusstep added in #959 published crash files as a plain artifact. Logs and artifacts on a public repository are readable by anyone, so the crasher of an unfixed bug went public before its fix. The in-process driver also ran every input under pymalloc only, whose pools hide an over-read that stays inside a pool from AddressSanitizer, and tox handed the fuzz environments a randomPYTHONHASHSEEDper run, so a find on the hash-salted XML duplicate-attribute check might not replay.The driver now logs a crasher only by SHA-256, length and harness, and keeps the bytes as
.fuzz-crashes/crash-<sha256>, the libFuzzer artifact naming. The seed and mutation index that regenerate it go tocrash-<sha256>.replay, and a deep run writes its sanitizer reports tocrash-sanitizer.<pid>, because a report names the faulting function and line. The workflow uploads crashers only after encrypting each one with age v1.3.2, a checksum-pinned release binary, to the public key in theFUZZ_AGE_RECIPIENTrepository variable. Without the variable it uploads nothing. Maintainers holding the identity key become the only readers, the audience OSS-Fuzz gives its reports until the fix ships. The fuzzing page of the development docs walks through the one-time key setup.Each in-process run now executes twice, under pymalloc and under
PYTHONMALLOC=malloc, over the same mutation sequence. The deep budget splits across the two passes, so the weekly 20-minute slot stays inside the six-hour job limit.fuzzandfuzz-smokepinPYTHONHASHSEED=0, and--rng-seednow reaches the worker. The scheduled job draws a random seed per run and passes it only through the environment, since tox echoes its command line and the seed regenerates every crasher. The deep run also replays the JS minifier fixtures intests/js/_corpus; it pointed at a directory that does not exist and never replayed them.The roundtrip and sanitize idempotence checks skip two documented cases and nothing else: script text that breaks the script content restrictions by opening
<!--and then<scriptbefore any-->, whose serialization the standard does not promise to reparse to the same tree, and trees that reached the 512 open-element cap, past which the builder stops nesting.Past finds (#521, #752, #949, #952) move to
tests/fuzz_regressions/, and every run replays them first through every harness, through the standalone harnesses' file-argument replay and the in-process seed loop. jsoup's FuzzFixesTest replays its own finds the same way.The malloc pass catches the class of #949, a one-past-the-end read that pymalloc hid and only the system allocator exposed. A regression of any listed fix now fails the per-PR smoke gate, and a scheduled find stays private while it waits for a fix.