[lukashold]

Latest integration follow-up (b45ce0c)

Integrates main 9e5b70d1c231d7db558e278a2622b519ca063494, retaining its timestamp-stable batch fixture and regression. Resolves the two Windows print-dialog CI failures from head 8218c730: even a synthetic printer source answers asynchronously, changing the dialog layout and the Print/Save action while fixed-frame tests are clicking.

Normal print-dialog tests now wait with a bounded deadline for discovery and let the layout settle. Their source deliberately answers late. Existing tests that intentionally observe loading or a late answer retain their explicit gates. PDF content/page selection, cut-stack layout, driver options and failure assertions are preserved. Removing the synchronization reproduces the missing Save as PDF control; restoring it passes all 13 print tests. All 9 batch tests, strict scoped Clippy, formatting, assets, layers and 32 WASM crates also passed. The concurrent equivalent fix from 5c926984 is retained in history; the final merged tree is identical to the locally validated tree.

Prior-head evidence: 8218c730 passed Linux and FreeBSD tests, all four service lifecycle platforms, Windows ARM64 build and installation, WASM, Nix, packaging lint and dependency checks. Its Windows suite failed the two print tests addressed above. These results are not claims of current-head CI success. Maintainer review and release-package qualification remain outstanding.

---

This adds a standalone PdfCraft Hot Folder task manager and an optional boot service. Enabled tasks can process scans while the editor is closed, with searchable output, scheduled execution and durable recovery after interruption.

The branch retains the scan/orientation/cropping work from #1012 and integrates main through afa8e184. Historical OCR remains separate in #1059. The shared optimization corrections and lossless API are reviewed separately in #1282 and are preserved here alongside the current optimizer presets.

Processing and recovery

• Named tasks; run now, pause and retry; once/interval/daily/weekly/monthly schedules; stable-file detection, subfolders, duplication/import/export, logs and failure filtering.
• Local and OS-mounted network folders, explicit FTPS or deliberately selected plain FTP, Outlook attachments through Microsoft Graph device authorization, and ordered open PdfCraft project manifests.
• OCR with primary-system-language selection and English fallback; rotation, deskew, paper-edge cropping, page ranges, recognition regions, preprocessing, spread splitting and conservative white-page removal with optional archives.
• Up to 16 saving steps: PDF, image-only PDF, DOCX, RTF, HTML, text, PNG, JPEG, TIFF, XLSX, EPUB and JSON analysis. Group by file, folder or settled run; naming templates and collision-safe output.
• Supervised task processes, cancellation and time/memory limits isolate unavailable shares and crashed workers. Transient errors retry with bounded backoff; malformed unchanged inputs remain blocked by fingerprint instead of being silently omitted from groups.
• Bounded discovery handles more than 10,000 retained inputs. Durable write-ahead receipts commit grouped outputs together. Every required output is checked against the published bytes/hashes before an original is moved/deleted, including after restart. Cancellation, missing/replaced outputs or incomplete publication never authorize source cleanup.
• Completed receipt recovery no longer requires installed OCR models. New work still reports a missing-model error and remains retryable. Windows confinement accepts only the authorized original/canonical root; conservative alias matching can reject overlaps but cannot grant access or probe another task's offline share.
• The task manager has English, German, French, Dutch, Spanish, Norwegian Bokmål and Swedish catalogs, with primary-system-language selection and English fallback.

Optional PDF optimization

Each PDF saving step exposes the same Off / Lossless / Reduce / Custom settings in the manager, engine and CLI/MCP task schema. Off remains the default.

Lossless excludes image re-encoding/downsampling and retains text, tags, forms, links and image streams. Reduce uses the shared 150 ppi/JPEG quality 60 defaults. Custom exposes separate color/grayscale resolution thresholds and Retain/ZIP/JPEG quality controls; downsampling and JPEG re-encoding can lose image detail. Lossless applies to the PDF entering this final step and cannot reverse separately selected rasterization or image-only conversion.

Only a strictly smaller candidate replaces the processed PDF; equal/larger candidates retain the original processed bytes. The UI and headless status report the actual published before/after sizes. Inputs are never overwritten. Signed inputs are refused before page selection, rasterization or grouping. Explicit Keep/Move/Delete policies still apply only after all outputs and their durable proofs succeed.

Writer integration: the generation-aware full-save correction from #1143 is backported from da54b14f085c930350d9b23839bd404cf4c3a625 (COS implementation and five regressions). The added Hot Folder regression exercises Off/Lossless/Reduce/Custom publication while preserving live extension data and preventing stale page references from acquiring replacement content. Final installer acceptance must still be repeated on the released package.

Installation and service

The companion is optional in Windows MSI and macOS PKG, and a separate Linux DEB/RPM. It registers a real boot service: Windows SCM under LocalService, Linux systemd under a dedicated account, or a macOS LaunchDaemon under a configured non-root account. Closing the manager leaves the service running. Nothing is processed until a task is enabled; the drag-and-drop DMG installs only the editor.

Credentials use protected service-account storage and are excluded from task exports. Administration APIs reject unrelated executable/configuration paths. A stable Windows service SID and repeated SCM recovery preserve access across account changes. MSI upgrades retain task data and an administrator-selected service account; they do not silently reset its logon. No MCP server or network listener starts implicitly.

Latest CI follow-up (fe18b08c)

The previous head passed the renderer cancellation regression on FreeBSD but failed catalog font coverage on all three native platforms and an incompatible early-upgrade assertion in Windows MSI validation.

• Rephrase the Bulgarian caption without U+045D, preserving its meaning and bundled-font coverage. All 11 font integration tests pass locally.
• Keep late MSI upgrades to preserve the existing SCM account. Explicitly remove only PdfCraft's legacy image-menu keys before registry writes; selected menu components then recreate them. Validate cleanup rows for every supported image extension and their execution order in the compiled MSI.
• The updated compiled-MSI check fails on the old MSI (missing cleanup table) and passes on the corrected synthetic MSI. The Windows lifecycle test now checks upgrade menu opt-out and obsolete subkey removal alongside service-account, task-data and executable preservation.
• Scoped strict Clippy, formatting, assets, layers and WASM passed. No service was installed locally. Current-head Windows lifecycle/ARM64 and full native CI remain pending; local MSI inspection is not release-package acceptance.

Validation of this integration

Previous integration head: 06aca3df7dcf481497320b62d0fcb1d7180056fd, integrating main afa8e1843cfed613f3d403119f6c2649bce824c9 (0.8.0).

• Fix the FreeBSD cancellation test's scheduler-sensitive two-second spin with a blocking startup handshake. Cancellation must still complete while parser initialization is held; both threads are released/joined before result assertions. The prior failure was FreeBSD run 38689653547. Current-head FreeBSD CI remains to be checked.

• Preserve lossless image streams when integrating the new fragment-merging optimizer and Standard/Mobile presets. Lossless mode disables fragment merging and rejects conflicting custom settings. A 120-strip fixture fails with merging enabled and passes with the correction.

• Refuse Hot Folder task/source/project calls under unsupported separate automation read/write roots before I/O; the supported shared --root behavior remains. Service mutations respect either confinement mode. The regression uses valid read/write paths and creates a configuration when the guard is removed.

• Keep fixed replay timestamps and injected wall clocks independent. The regression fails with the stale clock settings retained.

• Accept an OCR checkbox caption such as □Haupt only when its explicit box mark overlaps the matching box and the caption extends right on the same line. Synthetic positive/negative geometry cases and real-model scan regressions cover this. The skewed 150 dpi fixture uses a legible 10 pt telephone label; field-name and geometry assertions remain intact, with no general OCR-accuracy claim.

• Reconcile save tests with repaired page trees, signature tests with the current password minimum, history tests with logical page labels, and printer-option tests with the layout paper sent separately from driver defaults. Keep optional service packaging and upstream installer upgrade choices. Reconcile duplicate parity/catalog entries and regenerate attribution. New Vietnamese/Lithuanian catalog additions await native review.

• Selected native suites: 1,435 passing cases and two ignored across automation, Hot Folder, engine, forms, optimizer, renderer and UI. The combined run used the old 8 pt form fixture and had that one failure; the corrected 10 pt fixture then passed its focused rerun. Other automation integration suites also passed earlier in this refresh.

• Scoped cargo clippy --all-targets -- -D warnings, cargo fmt --all -- --check, and cargo xtask wasm passed (32 WASM crates).

• Assets (272 repository, 6 bundled, 25 fetched), layer boundaries (39 crates), and parity checks passed.

• Safe PowerShell server/stop-wait harnesses passed; merged MSI XML parsed with unique IDs. No local service installation was performed.

• FreeBSD execution and current-head cross-platform CI remain for GitHub Actions. Optional craft-fonts and privileged Windows symlink cases were not qualified locally.

Earlier evidence is not current-head validation: integration 116cd60c passed 3,173 workspace tests and a real four-profile CLI preservation run. Head 2ec3883b passed Windows/Linux CI, service lifecycle, packaging and ARM64 checks; its FreeBSD cancellation-test failure is addressed here. Fork Actions is disabled, so upstream PR workflows provide current CI results. No Actions settings were changed. Maintainer review and final package qualification remain outstanding.

Platform acceptance and remaining limits

The earlier artifact-backed server acceptance run 38167184192 passed all 11 scenarios and clean uninstall on Windows Server 2022 (20348) and 2025 (26100). It installed the actual x64 MSI from build 38164423890, source 0bfebc1de485ea8000184911faeb860335d91230, SHA-256 cf4e9da483ace9c7bacb208e0f2733a9896e27bf5fe4011743548c35ae11a2b4. That is earlier packaged-worker evidence, not a claim that a new MSI for the current optimization tree has already been accepted. It covers LocalService OCR, malformed-input isolation, automatic output/SMB retry, active-worker hard kill, bounded stop, SCM recovery and upgrade preservation of tasks/account/authenticated SMB access.

SMB acceptance used authenticated encrypted loopback shares. Server 2019, specific NAS devices, domain/gMSA deployments and real AD/GPO rollouts remain unqualified. Separate synthetic MSI/DEB/PKG lifecycle checks passed on Windows Server 2022/2025, Ubuntu 24.04 and macOS 15; those do not qualify every final OCR executable or architecture. Signed/notarized release acceptance and RPM lifecycle tests remain separate.

This is a clean-room implementation without ABBYY code/assets or proprietary task/project compatibility. Blank-page detection is white-only; OCR and layout quality remain bounded by PdfCraft's engine, with no ABBYY parity or general accuracy claim. Outlook requires public-client registration and consent; remote protocol checks use fixtures. FTP/Outlook/project attachments support PDF/PNG/JPEG/TIFF; local folders also support BMP.

The integrated baseline repairs retain their provenance: upstream now supplies Preferences scrolling and selection handling; the Bulk Properties harness adjustment from #1015 remains. Release and packaging lint use main's shared craft-common-actions workflows unchanged; the repository's packaging/*/package.* scripts build and package the Hot Folder app, and packaging/lint-hot-folder.sh adds its installer checks through the shared lint's extra-checks-script input. Under pdfcraft-cli mcp, Hot Folder service changes and credential changes need --allow-hot-folder-admin.

See docs/hot-folder*.md for limits and setup. No private documents or mailbox data are included.

AI assistance: implemented with OpenAI Codex for @lukashold.

Summary by CodeRabbit

• New Features
  • Added a standalone Hot Folder app for scheduled PDF and image processing, with local, FTP/FTPS, Outlook, and project sources; configurable preprocessing, OCR, output formats, and optimization.
  • Added optional Hot Folder background-service installation for Windows, macOS, and Linux.
  • Added scan correction with automatic rotation, deskewing, and paper-edge cropping, with or without text recognition.
  • Expanded offline OCR with multilingual recognition and system-locale language selection.
  • Added lossless PDF optimization, EPUB and XLSX exports, and read-only PDF content analysis.
  • Added automation controls for Hot Folder tasks, sources, and service management.
• Bug Fixes
  • Improved handling of PDF object generations, scanned-checkbox labels, and image optimization safeguards.
  • Made optimization succeed cleanly when selected hidden-information categories are absent.

[Harrison-Booth]

Provenance and surface check (read of the diff against main, plus a scoped test run; the PR is too large for GitHub's diff view, so I diffed the branch locally). Comment only. This PR adds network clients, a boot-service installer and credential storage, so the questions for the owner come first.

Questions for the owner

1. hot_folder_service is a registered automation/MCP tool with actions install, uninstall, enable, start, stop and prepare-account. It refuses these under --root or separate automation roots (tested), but an unconfined pdfcraft-cli mcp or run session can install a boot service. Is that intended, or should these actions be CLI-only, or need a separate opt-in?
2. hot_folder_source (save_ftp_password, forget_credentials, outlook_begin/poll) is also an MCP tool and is not gated by anything beyond the usual MCP opt-in. Same question.
3. deny.toml widens three existing RUSTSEC ignores (RUSTSEC-2026-0370/0371/0372, zvariant, zbus_names, zvariant_derive) to cover the new keyring Secret Service backend (zbus 4). That adds a second reason to carry unmaintained crates. Is a different keyring backend acceptable instead?
4. release.yml is rewritten by about 570 lines in a PR from an outside contributor, in the workflow that holds the Apple, Windows (Azure) and signing secrets. It also adds curl ... sh.rustup.rs | sh in one job and unpinned actions/upload-artifact@v4, download-artifact@v4, dtolnay/rust-toolchain@stable and Swatinem/rust-cache@v2 in several (other steps are SHA-pinned). Someone with release ownership should read that file line by line.

Network surface (all explicit, none at import or startup per docs/hot-folder-sources.md; I read the code to confirm the endpoints)

• FTP/FTPS (hot_folder_sources/ftp.rs): TcpStream to a user-configured host. Explicit TLS is the default, certificates are checked against OS roots, plain FTP only with security: "plain". Passive data connections go to the control connection's peer address.
• Outlook (outlook.rs): ureq to fixed hosts graph.microsoft.com and login.microsoftonline.com, redirects disabled, @odata.nextLink checked against the host. Uses a device-code flow with a client ID the user must register themselves.
• Boot service (crates/hot-folder-service): Windows service, systemd/launchd units; only through explicit install actions, and the crate says nothing is installed by linking it.
• Credentials: keyring for interactive use; service mode uses permission-protected files under <config>/secrets with owner/ACL checks.

Dependencies: 82 new packages in Cargo.lock, all from crates.io, no git dependencies. Notable direct additions: ureq (rustls, ring), rustls, rustls-native-certs, keyring, windows-service, ctrlc, sysinfo, processkit, rten-imageproc, rten-tensor, rten onnx_format, plus rcgen/x509-parser for tests. I did not run cargo deny (not installed here) and did not check each crate's licence.

Models and assets: no binary files in the diff. The three OCR assets (PP-OCRv6 recognition ONNX, PP-LCNet orientation ONNX, dictionary) are listed in ATTRIBUTION.toml as Apache-2.0 PaddleOCR files from Hugging Face / GitHub at pinned commits, with SHA-256 and licence hashes, and fetched by cargo xtask models, not committed. I did not download them or verify the hashes against the source.

Clean-room: docs/hot-folder.md cites ABBYY's public Hot Folder user guide as the reference and says no ABBYY code or assets are included; the code states that ABBYY project files are not supported. Behaviour from public docs only, as far as I can tell from the text; I cannot verify what the author read.

Ran: cargo test -p pdfcraft-hot-folder-service with CARGO_BUILD_JOBS=2 on Linux: 6 passed. Not run: the engine, automation, OCR and app tests, the Windows and macOS service code and installers, and the CI changes.

[lukashold]

@Harrison-Booth thanks for the careful read. Answers in order:

1. hot_folder_service over MCP: not intended to be open by default. pdfcraft-cli mcp now refuses every action except status unless the user starts it with --allow-hot-folder-admin (5da74e3). The check lives in Automation (with_hot_folder_admin), so nested dispatch and the compact tool_call path go through it too. pdfcraft-cli run scripts are unchanged, since the user writes and starts those.
2. hot_folder_source over MCP: same gate. save_ftp_password, forget_credentials, outlook_begin and outlook_poll need --allow-hot-folder-admin. credential_status and outlook_folders still work without it, and neither returns a secret. Both new tests fail with the check disabled.
3. keyring / zbus 4: I'd like to keep the Secret Service backend. The three advisories are "unmaintained", not vulnerabilities, and the same crates are already in the tree through accesskit_unix. Secret Service is the standard protected store on Linux desktops. The alternatives are worse for users: linux-native (kernel keyutils) loses passwords at reboot, and plain files are weaker than a keyring for interactive use (service mode already uses permission-checked files). If you'd rather drop it anyway, the protected-file store is a small switch, so say the word.
4. release.yml: agreed, and it's gone. The branch now uses main's 43-line call to the shared craft-common-actions release workflow unchanged (a23b7c8). That workflow runs packaging/*/package.*, which build and package the Hot Folder app themselves, and uploads all of dist/release, so no secrets-bearing workflow changes remain in this PR. packaging-lint.yml is back to main's shared job too, plus one extra-checks-script (packaging/lint-hot-folder.sh) for the Hot Folder installer scripts. The remaining branch workflow additions (hot-folder-services.yml, hot-folder-server.yml, the CI job) pin every action by SHA and have no curl | sh.

🤖 Addressed by Claude Code