Skip to content
Merged
66 changes: 48 additions & 18 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,52 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [0.2.1] - 2026-03-18

Major performance release: DFA compilation with SIMD-accelerated prefilter.

### Changed

- **DFA backend** replaces NFA for search. All failure transitions are pre-computed
into a flat `[]uint32` transition table at build time, eliminating failure link
following at search time entirely. Premultiplied state IDs allow single-instruction
transitions: `trans[sid + byteClass]`.

- **Match flag in transition table**. High bit of each transition entry indicates
whether the target state is a match state. Non-match bytes require zero masking
in the hot loop — the raw value IS the clean state ID.

- **Inline DFA loop in FindAll**. Previously delegated to `Find()` per match,
causing heap allocation for each `*Match`. Now uses a single DFA traversal
with stack-allocated match values. Allocations reduced from 14 to 4 per call.

### Added

- **SIMD-accelerated start byte prefilter**. Before running the DFA, uses
`bytes.IndexByte` (SIMD-optimized on amd64/arm64) to check if any pattern
start byte exists in the haystack. If none found, returns immediately.

- **Skip-ahead prefilter inside search loop**. When the DFA returns to start
state during search, re-engages the prefilter to skip ahead to the next
position where a match could start. Avoids processing long runs of
non-pattern bytes one at a time.

- `findEarliestStartByte` helper for prefilter position scanning.

### Performance

Benchmarks on Intel i7-1255U (64KB haystack, 4-7 patterns):

| Method | v0.1.0 | **v0.2.1** | Improvement |
|--------|--------|-----------|-------------|
| `Find` | 300 MB/s | **3.4 GB/s** | **11x** |
| `IsMatch` (no match) | 260 MB/s | **5.9 GB/s** | **23x** |
| `IsMatch` (match@32KB) | 545 MB/s | **7.0 GB/s** | **13x** |
| `FindAll` (77B, 10 matches) | 40 MB/s | **100 MB/s** | **2.5x** |

Memory: DFA uses a single flat array (~25KB for 100 states, stride 64).
Zero heap allocations for `IsMatch`.

## [0.1.0] - 2026-01-05

Initial release of the high-performance Aho-Corasick library for Go.
Expand Down Expand Up @@ -36,22 +82,6 @@ Initial release of the high-performance Aho-Corasick library for Go.
- Precomputed root transitions (no failure link following for root)
- Zero-allocation `IsMatch()` hot path

### Performance

Benchmarks on Intel i7-1255U (64KB haystack, 4 patterns):

| Method | Throughput | Allocations |
|--------|------------|-------------|
| `IsMatch` (with match) | 1.6 GB/s | 0 |
| `Find` | 1.1 GB/s | 1 |
| `IsMatch` (no match) | 780 MB/s | 0 |

### Testing

- 27 unit tests covering core functionality
- 2 fuzz tests verifying correctness against `bytes.Contains`/`bytes.Index`
- 93% code coverage
- CI on Linux, Windows, macOS with race detector

[Unreleased]: https://github.com/coregx/ahocorasick/compare/v0.1.0...HEAD
[Unreleased]: https://github.com/coregx/ahocorasick/compare/v0.2.1...HEAD
[0.2.1]: https://github.com/coregx/ahocorasick/compare/v0.1.0...v0.2.1
[0.1.0]: https://github.com/coregx/ahocorasick/releases/tag/v0.1.0
47 changes: 29 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,9 @@ High-performance Aho-Corasick multi-pattern string matching for Go.

## Features

- **1+ GB/s throughput** — comparable to Rust's [aho-corasick](https://github.com/BurntSushi/aho-corasick)
- **Dense array transitions** — optimized NFA with O(1) state transitions
- **Up to 7 GB/s throughput** — DFA compilation with SIMD-accelerated prefilter
- **Flat transition table** — fully compiled DFA with premultiplied state IDs
- **SIMD prefilter** — `bytes.IndexByte` skip-ahead for non-matching regions
- **Byte class compression** — reduces memory by grouping equivalent bytes
- **Multiple match semantics** — LeftmostFirst (Perl) and LeftmostLongest (POSIX)
- **Zero dependencies** — pure Go, no cgo
Expand All @@ -22,7 +23,7 @@ High-performance Aho-Corasick multi-pattern string matching for Go.
go get github.com/coregx/ahocorasick
```

Requires Go 1.21+
Requires Go 1.25+

## Quick Start

Expand Down Expand Up @@ -62,15 +63,21 @@ func main() {

## Performance

Benchmarks on Intel i7-1255U (64KB haystack, 4 patterns):
Benchmarks on Intel i7-1255U (64KB haystack, 4-7 patterns):

| Method | Throughput | Allocations |
|--------|------------|-------------|
| `IsMatch` (with match) | **1.6 GB/s** | 0 |
| `Find` | **1.1 GB/s** | 1 |
| `IsMatch` (no match) | 780 MB/s | 0 |
| `IsMatch` (with match) | **7.0 GB/s** | 0 |
| `IsMatch` (no match) | **5.9 GB/s** | 0 |
| `Find` | **3.4 GB/s** | 1 |
| `FindAll` (77B input) | 100 MB/s | 4 |

Comparable to Rust's aho-corasick crate (~1-2 GB/s).
### How it achieves this

1. **DFA compilation** — all failure transitions pre-computed at build time into a flat `[]uint32` array. Search is a single table lookup per byte: `trans[sid + class]`.
2. **SIMD prefilter** — before running the DFA, `bytes.IndexByte` (SSE2/AVX2 on amd64) scans for pattern start bytes. Skips entire regions where no match is possible.
3. **Skip-ahead on start state** — when the DFA returns to its start state during search, the prefilter re-engages to jump ahead, avoiding byte-by-byte scanning of non-matching text.
4. **Match flag embedding** — the high bit of each transition entry flags match states, enabling single-instruction match detection with no separate lookup.

## API

Expand Down Expand Up @@ -108,6 +115,20 @@ LeftmostFirst // First pattern in list wins (Perl-compatible, default)
LeftmostLongest // Longest pattern wins (POSIX-compatible)
```

## Architecture

```
Builder.Build()
-> NFA construction (trie + failure links)
-> DFA compilation (flat transition table, premultiplied state IDs)
-> Prefilter setup (start byte collection for SIMD scanning)

Search: IsMatch / Find / FindAll
-> SIMD prefilter (bytes.IndexByte skip-ahead)
-> DFA traversal (trans[sid + class], one operation per byte)
-> Match flag check (raw & matchFlag, one AND per byte)
```

## Use Cases

- **Log analysis** — scan for error patterns in log files
Expand All @@ -116,16 +137,6 @@ LeftmostLongest // Longest pattern wins (POSIX-compatible)
- **DNA sequencing** — find multiple motifs simultaneously
- **Regex acceleration** — as prefilter for `foo|bar|baz` alternations

## How It Works

The [Aho-Corasick algorithm](https://en.wikipedia.org/wiki/Aho%E2%80%93Corasick_algorithm) builds a finite automaton from patterns:

1. **Trie construction** — patterns form a prefix tree
2. **Failure links** — enable backtracking without re-reading input
3. **Dense transitions** — O(1) state lookup via byte class indexing

This allows matching all patterns simultaneously in O(n) time regardless of pattern count.

## Related Projects

- [coregex](https://github.com/coregx/coregex) — High-performance regex engine (uses this library)
Expand Down
2 changes: 1 addition & 1 deletion ahocorasick.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,4 @@
package ahocorasick

// Version is the current library version.
const Version = "0.1.0-dev"
const Version = "0.2.1"
Loading
Loading