Skip to content

feat(ingest): add glob selection to local ingest - #272

Open
rodboev wants to merge 12 commits into
kenn-io:mainfrom
rodboev:pr/197-ingest-glob-selection
Open

feat(ingest): add glob selection to local ingest#272
rodboev wants to merge 12 commits into
kenn-io:mainfrom
rodboev:pr/197-ingest-glob-selection

Conversation

@rodboev

@rodboev rodboev commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Ingest can now select files from a mixed source tree with include and exclude patterns. Preflight, ordinary ingest, streamed ingest, and the command-line report use the same selection rules, so a matching file is counted and imported consistently.

Exclusions take precedence, directories remain traversable when only a descendant can match, and empty include rules preserve the current behavior. Patterns use the standard path-matching grammar and stay scoped to the requested source tree; watched-inbox exclusions remain literal. Existing --exclude values now use glob matching; use bracket expressions such as report[[]1].txt for literal metacharacters.

The issue report came from a workflow that had to enumerate exclusions in a mixed tree to keep one import report and provenance record. This change covers glob source selection only; the separate per-file size budget and add replacement behavior remain outside the scope.

Refs #197.

Carry include and exclude rules through preflight, ingest, and CLI while preserving watcher literal matching and daemon boundaries.
Carry include and exclude rules through preflight, ingest, and CLI while preserving watcher literal matching and daemon boundaries.
Carry include and exclude rules through preflight, ingest, and CLI while preserving watcher literal matching and daemon boundaries.
@roborev-ci

roborev-ci Bot commented Sep 4, 2026

Copy link
Copy Markdown

roborev: Combined Review (9ba0929)

Verdict: One medium-severity issue found; otherwise, reviewers found no concerns.

Medium

  • internal/ingest/ingest.go:606-614, 655-659 — If an included directory is replaced by a regular file between classification and WalkDir, ensureSourceDir can recurse toward the filesystem root indefinitely and stack-overflow the daemon. Detect this source change and report an error, or reject paths that are not equal to or descendants of walkRoot.

Reviewers: 2 done | Synthesis: codex, 6s | Total: 7m17s

@roborev-ci

roborev-ci Bot commented Sep 4, 2026

Copy link
Copy Markdown

roborev: Combined Review (46e35cd)

Verdict: No Medium, High, or Critical issues found.

Reviewers agree the code is clean.


Reviewers: 2 done | Synthesis: codex, 4s | Total: 8m28s

@rodboev rodboev changed the title Add glob selection to local ingest feat(ingest): add glob selection to local ingest Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant