Skip to content

Reduce compilation peak memory by 45% - #28

Merged
danslo merged 4 commits into
mainfrom
reduce-compilation-memory
Sep 29, 2026
Merged

danslo merged 4 commits into
mainfrom
reduce-compilation-memory

Conversation

@danslo

@danslo danslo commented Sep 28, 2026

Copy link
Copy Markdown
Contributor

Store single transitions inline in 12-byte automaton states (previously 32 bytes), preallocate dense transitions, and add memory measurements to cmd/parse-bench.

With ecomscan.yar (26,782 rules), peak RSS drops 304 → 166 MiB (45% less). Across 6,210 real corpus files, median scan time is comparable (15.30 → 15.42 seconds), with 29,260 matches and no errors in every run.

Validation: full test suite, race tests, go vet, and the optional go-yara benchmark tests pass.

@danslo
danslo requested a review from gwillem September 28, 2026 14:22
@gwillem

gwillem commented Sep 28, 2026

Copy link
Copy Markdown
Contributor

very cool, lets go!

@gwillem

gwillem commented Sep 28, 2026

Copy link
Copy Markdown
Contributor

maybe we should remove the anchored stuff since we dont need that

@danslo

danslo commented Sep 29, 2026

Copy link
Copy Markdown
Contributor Author

Good call, done.

@danslo
danslo merged commit 9ede8d8 into main Sep 29, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants