Skip to content

[E13–S9 09/15] Add daemon-first document processing commands - #233

Open
salmonumbrella wants to merge 1 commit into
kenn-io:mainfrom
salmonumbrella:contrib/s3-operator-cli
Open

[E13–S9 09/15] Add daemon-first document processing commands#233
salmonumbrella wants to merge 1 commit into
kenn-io:mainfrom
salmonumbrella:contrib/s3-operator-cli

Conversation

@salmonumbrella

Copy link
Copy Markdown
Contributor

What changed

  • Adds docbank processing profiles|plan|build|status on top of the authenticated daemon API. Plans resolve one exact current content version, and builds require both the reviewed fingerprint and explicit consent.
  • Keeps operator and machine output bounded: human status includes partial failures, --json emits stable objects, and processing build --ndjson emits exactly one job record followed by one terminal status record.
  • Adds docbank rendition get for the exact verified YAML-headed Markdown artifact. The client checks the transport digest, immutable identities, frontmatter, body checksum, and navigation before writing any bytes to stdout.
  • Extends docbank search with lexical, semantic, hybrid, and auto modes over an explicit content-version fence. Ambiguous embedding profiles require --binding, while the existing name and extracted-text search stays unchanged when --mode is absent.

Why

The authenticated processing API exists, but operators should not need to assemble HTTP requests or bypass the daemon to use it. These commands keep disclosure review, consent, immutable rendition reads, and source-fenced retrieval explicit while preserving the existing daemon-first ownership boundary.

Usage

docbank processing profiles
docbank processing plan /docs/report.pdf --profile private --json
docbank processing build /docs/report.pdf \
  --profile private \
  --plan-fingerprint "$PLAN_FINGERPRINT" \
  --consent \
  --ndjson

docbank rendition get "$ATTACHMENT_ID" > report.md

docbank search "termination date" \
  --mode hybrid \
  --profile private \
  --binding semantic \
  --source-version "$CONTENT_VERSION_ID" \
  --explain

Repeat --source-version to search a larger bounded consumer fence. A semantic or hybrid search selects the embedding binding automatically only when the profile has exactly one.

Refs #176

Stacks on #232.

@roborev-ci

roborev-ci Bot commented Aug 28, 2026

Copy link
Copy Markdown

roborev: Combined Review (5485a57)

Verdict: Three medium-severity issues require attention before merge.

Medium

  • Projection metadata length is cumulatively overcountedinternal/vectorindex/generation.go:172-183, 287-295
    finishBuildProjection adds fixed header metadata to metadataLength, and Builder.Add persists the mutated projection. Each additional set recounts the fixed metadata, causing valid multi-set generations near the byte limit to be rejected. Keep fixed-header accounting separate or calculate using a temporary copy.

  • Processing start events are incorrectly marked terminalinternal/api/routes_processing.go:67-95; cmd/docbank/processing.go:206-218
    Start responses set Terminal: true even when the job is queued, running, or awaiting retry. NDJSON clients may treat unfinished processing as complete. Mark events terminal only for genuinely terminal statuses, or keep the stream open until completion and direct asynchronous clients to the status endpoint.

  • PyMuPDF processing lacks isolation for untrusted documentsdocument/pymupdf/provider.go:200
    PDFs are sent directly to an operator-selected executable without the network, namespace, or descendant-process isolation used by the adjacent local provider. A parser exploit could execute as the daemon owner and access or exfiltrate vault and other owner-readable data. Route PyMuPDF through an equivalent isolated, attested runner and fail closed when isolation cannot be enforced.


Reviewers: 2 done | Synthesis: codex, 7s | Total: 29m12s

@mariusvniekerk mariusvniekerk changed the title Add daemon-first document processing commands [E13–S9 09/15] Add daemon-first document processing commands Aug 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant