Conversation
Add coverage-guided fuzz testing using Hypothesis (property-based, pytest-integrated) and Atheris (libFuzzer-based). Four fuzz targets cover the main input parsing surfaces: JSON-RPC command handling, NATS .creds file parsing, Pydantic model validation, and JSON credentials loading. Includes seed corpus, CI jobs with findings published to GitHub Actions job summary, and initial findings documenting two bugs in MessagingConfig (TypeError on non-dict JSON, carriage return handling in creds parser).
…pollution Atheris writes new corpus entries into the corpus directory as it discovers coverage paths. Copy seeds into a temp dir for each run and clean up after, keeping the seed corpus directory unchanged.
- Fix TypeError in _load_credentials_file when JSON file contains a non-dict value (e.g. bare integer) by adding isinstance guard - Fix ruff lint errors: unused variables, extraneous f-string prefixes - Fix nats creds roundtrip test assertion for \r handling - Add .DS_Store to gitignore - Remove local findings from tracking (added to gitignore)
Add hypothesis and atheris fuzz targets for device-connect-server (CredentialsLoader JSON/regex parsing, PIN parsing) and device-connect-agent-tools (MCP tool name parsing, JSON-RPC message parsing). CI now runs a single hypothesis job and single atheris job covering all three packages, producing one combined report each in the GitHub Actions job summary. Fuzz dependencies (hypothesis, atheris) are optional extras in pyproject.toml. The fuzz/ directories are excluded from pip packages via setuptools include filters — pip install users are not affected.
… attribution Move fuzz/ directories under tests/fuzz/ in each package to reduce visual clutter. Extract JSON-RPC parsing logic from connection.py into importable helpers (parse_buffered_payload, parse_event_payload) so fuzz tests exercise real production code instead of duplicated simulations — this immediately caught two non-dict payload bugs (b'0' → int). Fix crash file attribution race in run_atheris.py by snapshotting before/after each target. Remove unused fuzz-atheris optional extra from edge pyproject.toml.
…error The unit test jobs install [dev] extras only, not [fuzz]. After relocating fuzz tests under tests/, pytest now discovers them during unit test collection and fails on the missing hypothesis import. Add --ignore flags to skip tests/fuzz/ in the edge and server unit test jobs.
atsyplikhin
approved these changes
Apr 9, 2026
Collaborator
atsyplikhin
left a comment
atsyplikhin
left a comment
There was a problem hiding this comment.
All checks passing. Reviewed the full diff — fuzz coverage is solid across all three packages, the bug fixes are correct, and the review fixes address the issues I flagged:
- Fuzz tests now live under
tests/fuzz/(less clutter) - Agent-tools JSON-RPC fuzz tests import real production code via extracted
parse_buffered_payload/parse_event_payloadhelpers — and immediately caught two non-dict payload bugs - Crash file attribution race in
run_atheris.pyis fixed - Unused
fuzz-atherisextra removed
LGTM.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add coverage-guided fuzz testing across all three packages using Hypothesis (property-based, pytest-integrated) and Atheris (libFuzzer-based). Fuzz dependencies are optional extras —
pip installusers are not affected.Coverage
.credsparsing, Pydantic model validation, JSON credentials loadingCredentialsLoaderJSON/regex parsing, PIN parsingCI
fuzz-tests-hypothesisjob and singlefuzz-tests-atherisjob covering all 3 packagesBugs found and fixed
MessagingConfig._load_credentials_file():TypeErrorcrash when JSON file contains a non-dict value (e.g.b'0'→json.loadsreturnsint, thenif "nats" in datafails). Fixed withisinstance(data, dict)guard.MessagingConfig._parse_nats_creds_file(): Carriage return (\r) in content gets silently normalized during.strip()extraction. Low severity — documented.What's included
packages/*/fuzz/test_fuzz_*.pypackages/*/fuzz/fuzz_*.pypackages/*/fuzz/corpus/packages/device-connect-edge/fuzz/run_atheris.pypackages/device-connect-edge/fuzz/report_hypothesis.py.github/workflows/ci.ymlpackages/*/fuzz/README.mdTest plan
ruff checkpasses on all fuzz directories