Add Apple Intelligence provider by yoavf · Pull Request #45 · yoavf/jst

yoavf · 2026-08-16T19:08:06Z

Summary

Adds an opt-in Apple Intelligence provider for macOS 27.0 beta and later.

Calls the on-device FoundationModels system model through a bundled Swift helper, without Rust-to-Swift bindings or a network hop.
Preserves the hosted JST server as the default provider; --provider apple and JST_PROVIDER=apple enable local mode.
Packages, signs, notarizes, and installs the helper correctly for macOS archives and Homebrew while Linux and Windows remain single-binary releases.
Fails before helper launch on macOS 26 and earlier with a clear requirement and server-provider fallback.
Documents availability, privacy, packaging, and manual installation.

Fixes #20

Validation

cargo test --workspace --all-targets --locked
cargo clippy --workspace --all-targets --locked -- -D warnings
Built the universal macOS package and verified both architectures.
Queried the on-device model on this Mac; jst --provider apple --dry show the current directory returned pwd.

Greptile Summary

Adds an opt-in Apple Intelligence translation provider while preserving the hosted server as the default.

Introduces a Swift FoundationModels helper and a bounded JSON process interface in the Rust CLI.
Shares prompt construction between hosted and local providers.
Extends macOS release, signing, notarization, Homebrew, and installation workflows to package the helper.
Documents provider selection, platform requirements, privacy, status reporting, and manual installation.

Confidence Score: 5/5

The PR appears safe to merge because no blocking failure remains within the eligible follow-up scope.

No blocking failure remains.

Important Files Changed

Filename	Overview
crates/cli/src/main.rs	Adds provider selection, Apple helper discovery and process management, bounded I/O, platform checks, status handling, response validation, and provider-aware revisions.
crates/apple-intelligence/main.swift	Implements the FoundationModels helper, availability reporting, guided structured generation, JSON encoding, and error reporting.
crates/shared/src/prompt.rs	Centralizes system and user prompt construction so hosted and Apple providers use the same translation requirements.
crates/server/src/openai_compatible.rs	Reuses shared prompt builders for hosted translation while retaining provider request, fallback, timeout, and response-sanitization behavior.
scripts/build-apple-intelligence-helper.sh	Builds arm64 and x86_64 macOS 27 helper binaries and combines them into a universal executable.
scripts/build-macos-release.sh	Adds the universal Apple helper to the macOS release directory and verifies both packaged architectures.
scripts/sign-and-notarize-macos.sh	Requires, signs, verifies, archives, and notarizes the Apple helper alongside the Rust CLI.
scripts/render-homebrew-formula.sh	Installs the Apple helper into Homebrew's private libexec directory for CLI discovery.

_{Reviews (2): Last reviewed commit: "Address Apple provider review feedback" | Re-trigger Greptile}

github-actions · 2026-08-16T19:10:40Z

The macOS CLI preview for this PR is ready. With GitHub CLI authenticated, copy this one command to download and run it:

preview_dir="$(mktemp -d)" && gh run download 32024820029 --repo yoavf/jst --name jst-pr-preview-macos-universal --dir "$preview_dir" && chmod +x "$preview_dir/jst" && "$preview_dir/jst" --dry list files in the current directory

Replace the example arguments after jst to test another command. The artifact is retained for 7 days.

View the preview build

qodo-code-review · 2026-08-17T11:04:30Z

PR Summary by Qodo

Add opt-in Apple Intelligence provider via bundled Swift helper on macOS 27+

✨ Enhancement 📝 Documentation ⚙️ Configuration changes 🧪 Tests 🕐 40+ Minutes

AI Description

• Add --provider apple / JST_PROVIDER=apple to use on-device Apple Intelligence on macOS 27+.
• Invoke a bundled Swift helper over stdin/stdout JSON, avoiding Rust↔Swift FFI and networking.
• Update macOS packaging/signing/Homebrew to ship the helper alongside the universal CLI binary.

Diagram

graph TD
  CLI(["jst CLI"]) -->|"--provider server"| SRV(["Hosted JST server"]) --> LLM{{"OpenAI-compatible LLM"}}
  CLI -->|"--provider apple"| HLP(["Swift helper"]) --> FM{{"FoundationModels"}}
  PKG(["macOS packaging"]) -->|"build/sign/notarize"| CLI
  PKG -->|"build/sign/notarize"| HLP
  subgraph Legend
    direction LR
    _cmp(["Component"]) ~~~ _ext{{"External/system"}}
  end

High-Level Assessment

The following are alternative approaches to this PR:

1. Direct Rust↔Swift bindings (FFI) to FoundationModels

➕ Single-process execution (no helper binary lookup/process spawn)
➕ Potentially lower overhead per request
➖ High maintenance/risk given beta-only Apple framework and evolving APIs
➖ More complex build system and cross-language ABI concerns
➖ Harder signing/notarization story vs an isolated helper binary

2. Route Apple provider through the existing server as a backend

➕ Keeps CLI simpler (single network flow)
➕ Centralizes model availability checks
➖ Breaks the privacy/offline goal (network hop)
➖ Introduces server-side macOS constraints and deployment complexity
➖ Still needs a macOS runtime somewhere to access FoundationModels

Recommendation: The separate Swift helper with a JSON protocol is the best tradeoff for a beta-only Apple framework: it keeps Rust free of unstable Swift bindings, preserves the hosted provider as the default, and cleanly scopes macOS-only packaging/signing changes while leaving Linux/Windows as single-binary releases.

Files changed (13) +652 / -38

Enhancement (4) +541 / -18

main.swiftIntroduce Swift helper using FoundationModels with JSON stdin/stdout contract +210/-0

Introduce Swift helper using FoundationModels with JSON stdin/stdout contract

• Adds a small 'jst-apple-intelligence' executable that checks 'SystemLanguageModel' availability, accepts a JSON prompt request, and emits a structured JSON translation response. Supports '--status' for availability reporting and maps generated effects/parts to the server response schema.

crates/apple-intelligence/main.swift

Cargo.tomlAdd serde and env-backed clap parsing for provider selection +2/-1

Add serde and env-backed clap parsing for provider selection
• Adds 'serde' to support JSON serialization for the helper request format and enables clap's 'env' feature to read 'JST_PROVIDER' as a default provider override.
crates/cli/Cargo.toml

main.rsAdd provider routing and Apple helper invocation with preflight checks +300/-16

Add provider routing and Apple helper invocation with preflight checks

• Introduces a 'Provider' enum and '--provider'/'JST_PROVIDER' option, extending '--status' to target the selected provider. Implements Apple provider translation via a spawned helper process with JSON request/response, size validation, timeout, helper discovery (adjacent binary or Homebrew libexec), and macOS 27+ preflight failures with clear fallback guidance; adds focused unit tests for parsing and validation utilities.

crates/cli/src/main.rs

prompt.rsAdd shared 'build_user_prompt' helper and tests for revision structure +29/-1

Add shared 'build_user_prompt' helper and tests for revision structure
• Introduces a shared function to format the user prompt for both initial requests and revisions, avoiding ad-hoc concatenation across crates. Adds unit coverage verifying the structured revision prompt format.
crates/shared/src/prompt.rs

Refactor (1) +4 / -15

openai_compatible.rsReuse shared user-prompt builder for revision requests +4/-15

Reuse shared user-prompt builder for revision requests
• Replaces server-local revision prompt construction with 'jst_shared::build_user_prompt', ensuring consistent user prompt formatting across server and Apple helper paths. Updates tests accordingly.
crates/server/src/openai_compatible.rs

Documentation (2) +62 / -4

ARCHITECTURE.mdDocument Apple provider flow and macOS helper packaging requirements +17/-4

Document Apple provider flow and macOS helper packaging requirements

• Updates the architecture overview to describe the new Apple provider path (CLI → helper → FoundationModels) and its macOS 27+ availability checks. Notes packaging constraints (helper must ship alongside the CLI binary; Homebrew installs under libexec).

ARCHITECTURE.md

README.mdAdd user docs for Apple Intelligence provider usage and installation +45/-0

Add user docs for Apple Intelligence provider usage and installation

• Documents how to enable the Apple provider ('--provider apple' / 'JST_PROVIDER=apple'), how to check availability, and the privacy/offline properties. Describes macOS archive/Homebrew helper installation expectations and manual install guidance.

README.md

Other (6) +45 / -1

Cargo.lockAdd serde as a direct CLI dependency in the lockfile +1/-0

Add serde as a direct CLI dependency in the lockfile

• Updates the resolved dependency graph to include 'serde' for the CLI crate, aligning with new request serialization needs for the Apple helper protocol.

Cargo.lock

Cargo.tomlEnable tokio io/process/time features for helper process execution +1/-1

Enable tokio io/process/time features for helper process execution
• Expands the workspace tokio feature set to include async IO/utilities, process management, and timeouts used when spawning and communicating with the Apple helper.
Cargo.toml

build-apple-intelligence-helper.shAdd script to build a universal Swift helper targeting macOS 27.0 +31/-0

Add script to build a universal Swift helper targeting macOS 27.0
• Adds a build script that compiles the Swift helper for arm64 and x86_64 with a macOS 27.0 deployment target, then uses 'lipo' to create a universal binary suitable for distribution.
scripts/build-apple-intelligence-helper.sh

build-macos-release.shBundle the Apple helper into the macOS universal release output +2/-0

Bundle the Apple helper into the macOS universal release output
• Extends the macOS release build to produce 'jst-apple-intelligence' alongside 'jst' and prints 'lipo' metadata for both binaries for verification.
scripts/build-macos-release.sh

render-homebrew-formula.shInstall Apple helper into Homebrew libexec +1/-0

Install Apple helper into Homebrew libexec
• Updates the generated Homebrew formula to install 'jst-apple-intelligence' into 'libexec' while leaving 'jst' in 'bin', matching the CLI helper discovery logic.
scripts/render-homebrew-formula.sh

sign-and-notarize-macos.shSign, verify, and notarize the Apple helper with the macOS archive +9/-0

Sign, verify, and notarize the Apple helper with the macOS archive
• Adds signing and verification steps for 'jst-apple-intelligence' and validates it after archive extraction, ensuring notarized macOS distributions include both signed executables.
scripts/sign-and-notarize-macos.sh

qodo-code-review · 2026-08-17T11:10:47Z

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📜 Skill insights (0)

1. ~~Unbounded parts reject translations~~ ✓ Resolved 🐞 Bug ☼ Reliability

Description

The detailed Apple schema only describes the eight-part maximum in prose, so the model can emit more
than eight fragments and make validate_apple_response reject an otherwise usable translation. This
affects interactive Apple requests, which always request the detailed response shape.

Code

crates/apple-intelligence/main.swift[R89-90]

+    @Guide(description: "One to eight command fragments in order. Their fragments must concatenate exactly to command.")
+    var parts: [GeneratedPart]

Evidence
Apple documents .maximumCount(_:) as the generation guide that enforces an inclusive array upper
bound. The helper currently supplies only a description, while the CLI rejects parts.len() > 8; by
contrast, the hosted-provider sanitizer clears unusable parts and retains the translation.
crates/apple-intelligence/main.swift[89-90]
crates/cli/src/main.rs[333-344]
crates/cli/src/main.rs[143-146]
crates/server/src/openai_compatible.rs[322-338]
🌐 Apple states that maximumCount(_:) enforces a maximum number of elements in a generated array and shows it passed as an explicit @Guide constraint.

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The Apple detailed-generation schema does not enforce its stated maximum of eight parts, but the Rust client rejects responses above that limit. Enforce the schema bound and preserve valid commands when explanation parts are unusable.

## Issue Context
FoundationModels provides `.maximumCount(8)` for an enforceable array bound. The hosted provider already clears unusable explanation parts rather than rejecting the whole translation.

## Fix Focus Areas
- crates/apple-intelligence/main.swift[89-90]
- crates/cli/src/main.rs[333-344]
- crates/server/src/openai_compatible.rs[322-338]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

2. ~~sw_vers version detection breaks under compat mode~~ ✓ Resolved 🐞 Bug ≡ Correctness

Description

ensure_apple_intelligence_supported() parses only the major version number from `sw_vers
-productVersion` output; if SYSTEM_VERSION_COMPAT is set in the environment (a documented macOS
mechanism), sw_vers reports a legacy '10.16'-style version instead of the real major version,
causing macos_major_version to return 10 and the Apple provider to be incorrectly rejected on a
genuinely supported macOS 27 host.

Code

crates/cli/src/main.rs[R406-427]

+    let output = Command::new("sw_vers")
+        .arg("-productVersion")
+        .output()
+        .map_err(|_| {
+            JstError::AppleIntelligence(
+                "requires macOS 27.0 or later, but the macOS version could not be determined"
+                    .to_string(),
+            )
+        })?;
+    let version = String::from_utf8_lossy(&output.stdout);
+    let Some(major) = macos_major_version(&version) else {
+        return Err(JstError::AppleIntelligence(
+            "requires macOS 27.0 or later, but the macOS version could not be determined"
+                .to_string(),
+        ));
+    };
+    if major < 27 {
+        return Err(JstError::AppleIntelligence(format!(
+            "requires macOS 27.0 or later (this Mac is running {}); use --provider server instead",
+            version.trim()
+        )));
+    }

Evidence

The new preflight relies exclusively on parsing sw_vers output rather than a more robust API, and
does not account for environment variables that alter that command's reported version, so a
correctly supported Mac can be spuriously told it needs 'macOS 27.0 or later'.

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`ensure_apple_intelligence_supported` in `crates/cli/src/main.rs` shells out to `sw_vers -productVersion` and parses only the leading major-version number to decide whether to allow the Apple Intelligence provider. This command's output can be altered by the `SYSTEM_VERSION_COMPAT` environment variable (a long-standing macOS compatibility shim), which makes it report a legacy `10.16`-style version string even on modern macOS releases, causing `macos_major_version` to return `10` and the preflight to incorrectly refuse to run on a supported macOS 27 host.

## Issue Context
The function is new in this PR and gates the entire Apple provider path; a false negative here silently disables the feature with a misleading 'requires macOS 27.0 or later' message even though the host is fully capable.

## Fix Focus Areas
- crates/cli/src/main.rs[399-429]
- crates/cli/src/main.rs[431-433]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

3. ~~Unbounded stdout read in status check~~ ✓ Resolved 🐞 Bug ☼ Reliability

Description

print_apple_intelligence_status() calls .output().await on the helper process and parses
output.stdout with no size limit, unlike translate_with_apple_intelligence which explicitly
rejects responses over MAX_RESPONSE_BYTES; a misbehaving helper binary can make the CLI buffer
unbounded data in memory for the --status path.

Code

crates/cli/src/main.rs[R349-363]

+    let output = tokio::process::Command::new(apple_helper_path()?)
+        .arg("--status")
+        .kill_on_drop(true)
+        .output()
+        .await
+        .map_err(|error| {
+            JstError::AppleIntelligence(format!("could not start the bundled helper: {error}"))
+        })?;
+    if !output.status.success() {
+        let detail = String::from_utf8_lossy(&output.stderr);
+        return Err(JstError::AppleIntelligence(helper_error_message(&detail)));
+    }
+    let status: serde_json::Value = serde_json::from_slice(&output.stdout).map_err(|_| {
+        JstError::AppleIntelligence("returned an invalid status response".to_string())
+    })?;

Evidence

translate_with_apple_intelligence enforces if output.stdout.len() > MAX_RESPONSE_BYTES before
parsing, but print_apple_intelligence_status has no equivalent check before calling
serde_json::from_slice on output.stdout, showing an inconsistency introduced by this PR's new status
code path.

crates/cli/src/main.rs[347-376]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`print_apple_intelligence_status` reads the Apple Intelligence helper's `--status` output via `tokio::process::Command::output()` and parses `output.stdout` with `serde_json::from_slice` without any size cap, unlike the translation path which enforces `MAX_RESPONSE_BYTES`.

## Issue Context
A buggy or compromised helper binary (or a manually-substituted one when `JST_APPLE_INTELLIGENCE_HELPER` is set) could write an arbitrarily large amount of data to stdout, which would be fully buffered into memory before any validation occurs.

## Fix Focus Areas
- crates/cli/src/main.rs[347-363]
- crates/cli/src/main.rs[19-20] (reuse MAX_STATUS_RESPONSE_BYTES or MAX_RESPONSE_BYTES)

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

4. ~~Helper path resolved without integrity verification~~ ✗ Dismissed 🐞 Bug ⛨ Security

Description

apple_helper_path() selects the first candidate path that satisfies is_file(), with no
verification of code signature, ownership, or permissions, before the CLI spawns it as a subprocess
with the same privileges as jst itself. If the directory holding the CLI (or the Homebrew libexec
directory) is writable by another local user or process, a substituted jst-apple-intelligence
binary would be executed silently.

Code

crates/cli/src/main.rs[R385-388]

+    let candidates = apple_helper_candidates(&executable);
+    if let Some(helper) = candidates.iter().find(|path| path.is_file()) {
+        return Ok(helper.clone());
+    }

Evidence

apple_helper_candidates() derives candidate paths purely from the running executable's location and
a fixed 'libexec' sibling directory, and apple_helper_path() picks the first that is_file() with no
additional trust check, then translate_with_apple_intelligence/print_apple_intelligence_status spawn
it directly.

crates/cli/src/main.rs[378-397]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`apple_helper_path` picks the first candidate that is a regular file with no additional trust checks (code signature, ownership, permissions) before it is executed as a subprocess by `translate_with_apple_intelligence` and `print_apple_intelligence_status`.

## Issue Context
This is a defense-in-depth concern: in a deployment where the CLI's own installation directory or the Homebrew `libexec` directory is writable by another local principal, an attacker could replace the helper binary and have it silently executed by `jst`.

## Fix Focus Areas
- crates/cli/src/main.rs[378-397]
- crates/cli/src/main.rs[435-441]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Context

✅ Web pages:

🌐 maximumCount(_:) | Apple Developer Documentation

🌐 count(_:) | Apple Developer Documentation

🌐 Deep dive into the Foundation Models framework - WWDC25 - Videos - Apple Developer

+32 more

Review mode: 🧠 Deep: This adds substantial, security- and packaging-sensitive provider logic across Rust, Swift, prompts, and release scripts, with many independent paths where redundant review could catch subtle integration defects.

Tip of the day

💡 Did you know, you can route each action level your way: inline, summary, both, or drop

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

cloudflare-workers-and-pages · 2026-08-17T11:24:57Z

Deploying jst with Cloudflare Pages

Latest commit:	`3e1884b`
Status:	✅ Deploy successful!
Preview URL:	https://38d7f4c1.jst-5mg.pages.dev
Branch Preview URL:	https://agent-apple-intelligence-on.jst-5mg.pages.dev

View logs

Add Apple Intelligence provider

6406e9c

yoavf marked this pull request as ready for review August 17, 2026 11:03

qodo-code-review Bot reviewed Aug 17, 2026

View reviewed changes

Comment thread crates/apple-intelligence/main.swift Outdated

Comment thread crates/cli/src/main.rs

Comment thread crates/cli/src/main.rs Outdated

Comment thread crates/cli/src/main.rs

Address Apple provider review feedback

3e1884b

yoavf merged commit 732ee4c into main Aug 17, 2026
6 checks passed

yoavf deleted the agent/apple-intelligence-on-device branch August 17, 2026 11:42

Navigation Menu

Add Apple Intelligence provider - #45

Add Apple Intelligence provider#45
yoavf merged 2 commits into
mainfrom
agent/apple-intelligence-on-device

yoavf commented Aug 16, 2026 •

edited by greptile-apps Bot

Loading

Uh oh!

github-actions Bot commented Aug 16, 2026 •

edited

Loading

Uh oh!

qodo-code-review Bot commented Aug 17, 2026

Uh oh!

qodo-code-review Bot commented Aug 17, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cloudflare-workers-and-pages Bot commented Aug 17, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

yoavf commented Aug 16, 2026 • edited by greptile-apps Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Validation

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

github-actions Bot commented Aug 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

qodo-code-review Bot commented Aug 17, 2026

PR Summary by Qodo

Uh oh!

qodo-code-review Bot commented Aug 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review by Qodo

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cloudflare-workers-and-pages Bot commented Aug 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Deploying jst with Cloudflare Pages

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

yoavf commented Aug 16, 2026 •

edited by greptile-apps Bot

Loading

github-actions Bot commented Aug 16, 2026 •

edited

Loading

qodo-code-review Bot commented Aug 17, 2026 •

edited

Loading

cloudflare-workers-and-pages Bot commented Aug 17, 2026 •

edited

Loading