Add portable secure background-job executors

ARCHITECTURE.md

@@ -254,6 +254,11 @@ Note: there is no `routes/ai.py` and no `POST /api/ai`. An older mini-app AI pro
 Host-mediated device/browser access uses the versioned capability broker; see
 [`CAPABILITIES.md`](CAPABILITIES.md) for the manifest, app API, wire protocol,
 provider contract, lifecycle rules, and trust-tier escape hatches.
+Server-side app jobs have a separate two-tier model: ordinary reviewed scripts
+retain the Möbius process authority, while `background_agent` jobs run through
+one reviewed data contract and the strongest secure executor available on the
+host. See [`BACKGROUND_JOBS.md`](BACKGROUND_JOBS.md) for the contract,
+Bubblewrap/Landlock selection, history, and verification strategy.
 
 | Tier | Boundary and capability | UX / standalone consequence |
 |---|---|---|
@@ -1056,4 +1061,7 @@ cover it deterministically.
 ## See also
 
 - **Build / test / run commands and the dev loop:** `CONTRIBUTING.md`. (The #1 deploy gotcha — a stale `/data/platform/frontend/dist` masking a fresh image — is covered under *Frontend serving priority* above.)
+- **Secure server-side app jobs:** `BACKGROUND_JOBS.md` defines the
+  background-agent data contract, portable executor design, historical
+  rationale, and topology-level verification.
 - **Subsystem deep-dives are inlined above** as their own sections: *Stop-chat contract*, *AskUserQuestion interception*, *Chat persistence — single-writer actor*, *Navigation back-stack + drawer model*, *Service worker + offline*, and *Mini-app manifest (mobius.json)*. (The chat-persistence v2 design + staged-rollout notes remain internal/gitignored — the as-built contract is the section above.)

BACKGROUND_JOBS.md

@@ -0,0 +1,221 @@
+# Secure background jobs
+
+Möbius has two server-side app-job tiers:
+
+- **Ordinary jobs** run reviewed, owner-installed app scripts with the
+  historical authority of the Möbius process.
+- **Background-agent jobs** declare `permissions.background_agent: true`.
+  They can run an AI agent without an owner watching, so they receive a
+  narrower, owner-reviewed data contract enforced by a process sandbox.
+
+This document defines that contract, why its implementation is portable, and
+the verification required to change it. Browser iframe isolation is a separate
+boundary; see [`CAPABILITIES.md`](CAPABILITIES.md) for browser-side apps.
+
+## The stable design
+
+The stable part of the system is a small semantic contract:
+
+```text
+JobAccess
+  source_read
+  storage_write
+  extra_read
+  extra_write
+```
+
+For the current manifest vocabulary this means:
+
+- app source is readable and not writable;
+- the app's numeric storage is readable and writable;
+- shared Memory data is absent, read-only, or writable exactly as reviewed;
+- supported provider credential directories are writable because their CLIs
+  may refresh credentials;
+- the app gets a minimal environment, a short-lived app token, and a unique
+  writable home/temp directory;
+- the editable platform checkout, database, service token, other app data,
+  undeclared shared data, sibling-process control, and host UNIX sockets are
+  denied (read-only image runtime under `/app` remains visible);
+- outbound IP networking remains available.
+
+The runner derives this contract once. Executors consume it; they do not
+interpret manifests or contain Memory-specific rules.
+
+Two executors implement the contract:
+
+1. **Bubblewrap** is preferred when a real namespace probe succeeds. It
+   supplies private mount, PID, IPC, and UTS namespaces and hides masked paths.
+2. **Landlock** is the fallback when the kernel exposes ABI 6 or newer and a
+   complete enforcement probe succeeds. `setpriv` installs filesystem rules;
+   a small helper adds Landlock process/abstract-socket scopes and seccomp
+   denials for pathname UNIX sockets and direct sibling-process inspection.
+
+If neither probe succeeds, the job does not run. There is no unsandboxed
+fallback for a reviewed background agent.
+
+### Startup and scheduled execution
+
+Executor portability is only useful if every launch reaches it consistently:
+
+- bootstrap initialization waits on the existing `/api/ready` contract before
+  requesting its scoped token and job context;
+- interactive and scheduled launches receive the backend's configured local
+  address rather than assuming port 8000;
+- Run now and cron prefer the served checkout's runner;
+- startup schedule reconciliation prefers the served checkout's scaffold and
+  rewrites older persisted entries through the current runner.
+
+The baked runner and scaffold remain degraded-boot floors. They are not the
+normal path after a platform update: preferring them would preserve old launch
+behavior until the next image rebuild even though the served backend had
+already advanced.
+
+## Design philosophy
+
+### Contract over mechanism
+
+An app asks for access, not for Bubblewrap or Landlock. Deployment mechanics
+must not leak into the manifest. This keeps app review stable when kernels,
+container runtimes, and hosting platforms change.
+
+### Probe behavior, not host names
+
+Möbius does not branch on Railway, Docker, Kubernetes, architecture, or an
+environment variable claiming a feature exists. Bubblewrap is selected only
+after the namespace operation needed by a real job succeeds. Landlock is
+selected only after its ABI, filesystem restriction, process scoping, and
+socket filter all work together.
+
+The probes run at job launch. They are cheap compared with an agent job and
+avoid a capability cache that can become stale after a container or host
+change.
+
+### Prefer the strongest working executor; fail closed
+
+Bubblewrap remains first because its private namespaces provide a stronger and
+easier-to-explain boundary. Landlock is not presented as identical: protected
+path metadata and process IDs may remain visible even though contents,
+mutation, signalling, inspection, and local socket access are denied.
+
+The shared contract is therefore expressed as allowed and denied operations,
+not as an identical filesystem view. A future job that genuinely requires a
+private PID or mount namespace must become a new explicit requirement; it must
+not silently receive the Landlock executor.
+
+### One policy, small adapters
+
+There is one path policy and two launch builders. There is deliberately no
+executor plugin registry, host capability database, background probe daemon,
+deployment matrix in production code, or general guarantee algebra. Add such
+machinery only after a real second policy requires it.
+
+Use maintained system interfaces where possible. In particular, util-linux
+`setpriv` owns Landlock filesystem rule construction. Möbius keeps only the
+small helper needed for protections that tool does not expose.
+
+### Make the decision inspectable
+
+While a job runs, its existing lease records `executor: process|bubblewrap|
+landlock`. A Landlock fallback records why Bubblewrap was rejected. If no
+executor qualifies, the durable app-job log records both probe diagnostics.
+This reuses the lease and failure log rather than adding a database or health
+service.
+
+## Why the system reached this point
+
+Background-agent isolation was introduced when Memory moved from
+platform-owned code into a modular system app. Bubblewrap was a sound initial
+executor: it could make the container filesystem read-only, mask owner data,
+mount only reviewed paths, and isolate processes with familiar namespace
+semantics.
+
+Nested Bubblewrap is not only an image property, though. The outer container
+runtime must permit namespace and mount setup. The bundled Docker Compose
+deployment was later given the required capabilities and security profile, so
+that deployment worked. Managed runtimes that do not expose equivalent outer
+container controls can reject Bubblewrap before app code starts. Memory made
+the gap visible because it was the first Store app to combine
+`background_agent` with install-time initialization.
+
+That initialization exposed three independent integration assumptions in
+sequence: the backend was not ready to mint a scoped token, scheduled jobs
+assumed the old local port and baked runner, and the managed host rejected
+Bubblewrap's namespace setup. Each correction belongs to its owning layer:
+readiness in bootstrap launch, address/runner selection in the shared job
+handoff, and host portability in secure executor selection. None belongs in
+Memory itself.
+
+The resulting lesson is narrower than “build a sandbox framework”:
+Bubblewrap was coupled to one deployment topology, while the reviewed access
+contract was not. Landlock provides a second enforcement path on modern
+restricted hosts without requiring namespace creation. Keeping both small
+preserves stronger isolation where available and portability where it is not.
+
+Related architecture already documented elsewhere:
+
+- [`ARCHITECTURE.md`](ARCHITECTURE.md) defines “solve at the core,” “design for
+  the next change,” and “keep the shared foundation lean.”
+- [`CAPABILITIES.md`](CAPABILITIES.md) establishes the broader pattern that
+  declarations are owner-readable contracts and mechanisms are narrow
+  providers.
+- [`SECURITY.md`](SECURITY.md) distinguishes hardened technical boundaries
+  from accepted trade-offs.
+- The original runner comments explain the narrower background-agent data
+  contract and why jobs write as the `mobius` data owner.
+- The Compose security settings explain why nested Bubblewrap needs explicit
+  outer-runtime support.
+
+## Alternatives considered
+
+| Alternative | Why it is not the current design |
+|---|---|
+| Bubblewrap only | Excludes demonstrated managed hosts that deny nested namespaces. |
+| Landlock only | Discards stronger private namespaces and excludes older kernels where Bubblewrap already works. |
+| Run unsandboxed if probing fails | Silently violates the permission the owner reviewed. |
+| Branch on deployment name | Brittle: the relevant property is kernel/runtime behavior, not branding. |
+| Add retries or a durable job queue | Does not fix an executor that can never start; solves a different problem. |
+| Cache host capabilities or run a probe daemon | Adds invalidation and lifecycle machinery to avoid millisecond launch probes. |
+| General executor/plugin framework | No demonstrated third executor or second policy justifies the abstraction. |
+
+## Verification contract
+
+Every executor must pass the same adversarial data test:
+
+- read app source and declared shared data;
+- write app storage and its unique temp directory;
+- fail to read the service token and database;
+- fail to write outside declared writable paths;
+- run durable writes as the `mobius` data owner.
+
+Each executor also verifies its mechanism-specific boundary:
+
+- Bubblewrap: real namespace creation, masked owner data, and process-group
+  revocation.
+- Landlock: ABI 6+, filesystem enforcement, denied sibling signals and process
+  inspection, denied `AF_UNIX` sockets, parent-death termination, and temp
+  cleanup.
+
+Selection tests cover Bubblewrap preference, Landlock fallback, and the
+fail-closed case with both diagnostics. CI may skip a real executor only when
+the host cannot provide it; each supported deployment topology must therefore
+run one end-to-end secure-job smoke test rather than treating a skip as proof.
+
+A release-level startup smoke should install a trivial `background_agent` app
+or Memory on a fresh volume, wait for its ready marker, and fail with the
+executor diagnostics if initialization cannot start. This catches image,
+kernel, outer-runtime, callback-address, and startup-order integration failures
+that unit tests cannot.
+
+## Change checklist
+
+When this boundary changes:
+
+1. Keep manifest interpretation in the runner and enforcement in executors.
+2. State any executor asymmetry explicitly; do not weaken the common contract.
+3. Run the shared adversarial suite against every available executor.
+4. Run one real secure job in each supported deployment topology.
+5. Verify job-group termination, parent-death behavior, and temp cleanup.
+6. Check both AMD64 and ARM64 images because syscall numbers, system packages,
+   and seccomp resolution are architecture-sensitive.
+7. Keep failures actionable and never silently run a background agent as an
+   ordinary process.

Dockerfile

@@ -36,13 +36,13 @@ RUN ln -s ../lib/node_modules/npm/bin/npm-cli.js /usr/local/bin/npm \
 # Chromium copy via the symlinks below (~/.agent-browser is where
 # agent-browser looks by default).
 #
-# Background-agent jobs run as the unprivileged mobius user, but bwrap must
-# create mount/PID namespaces inside the outer Docker container. Debian's
-# audited setuid mode retains only bwrap's small setup capability set and drops
-# it before execing the job. docker-compose.yml supplies the three required
-# capabilities absent from Docker's default bounding set.
+# Background-agent jobs prefer Bubblewrap: its audited setuid mode retains only
+# the setup capabilities needed for mount/PID namespaces and drops them before
+# execing the unprivileged job. docker-compose.yml supplies the outer-container
+# grants absent from Docker's defaults. util-linux + libseccomp provide the
+# Landlock fallback on modern kernels whose runtimes deny nested namespaces.
 RUN apt-get update && apt-get install -y --no-install-recommends \
-    cron curl ca-certificates git sudo procps util-linux bubblewrap age \
+    cron curl ca-certificates git sudo procps util-linux bubblewrap libseccomp2 age \
     libnss3 libnspr4 libatk1.0-0 libatk-bridge2.0-0 libcups2 \
     libdrm2 libxkbcommon0 libatspi2.0-0 libxcomposite1 libxdamage1 \
     libxfixes3 libxrandr2 libgbm1 libpango-1.0-0 libcairo2 libasound2t64 \
@@ -53,6 +53,8 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
     && npm install -g agent-browser@0.31.1 \
     && agent-browser install \
     && mv /root/.agent-browser /opt/agent-browser \
+    && setpriv --help 2>&1 | grep -q -- '--landlock-access' \
+    && ldconfig -p | grep -q 'libseccomp\\.so\\.2' \
     && chmod 4755 /usr/bin/bwrap \
     && test "$(stat -c '%a' /usr/bin/bwrap)" = 4755 \
     && git_version="$(git --version | awk '{print $3}')" \

SECURITY.md

@@ -33,6 +33,11 @@ being external attackers reaching the public HTTPS endpoint.
   the app's exact installed permissions. Opacity protects ambient **owner**
   authority — it is not a promise that ordinary app code never sees its own
   scoped credential.
+- **Background-agent jobs:** apps declaring `permissions.background_agent`
+  run under the reviewed data contract in
+  [`BACKGROUND_JOBS.md`](BACKGROUND_JOBS.md). Möbius prefers Bubblewrap after a
+  real namespace probe, otherwise uses a fully probed Landlock ABI 6+ boundary,
+  and fails closed if neither executor can enforce the contract.
 - **Rate limiting:** 120 req/min global, 3-5/min on auth endpoints.
   Uses TCP peer address (not X-Forwarded-For).
 

backend/app/app_jobs.py

@@ -20,23 +20,44 @@
 
 
 def runner_script() -> Path:
-  baked = Path("/app/scripts/app-job-runner.py")
-  if baked.is_file():
-    return baked
-  return Path(__file__).resolve().parent.parent / "scripts" / "app-job-runner.py"
+  live = Path(__file__).resolve().parent.parent / "scripts" / "app-job-runner.py"
+  if live.is_file():
+    return live
+  return Path("/app/scripts/app-job-runner.py")
 
 
-def runner_command(app_id: int, job_path: Path) -> list[str]:
-  return [sys.executable, str(runner_script()), str(app_id), str(job_path)]
+def runner_command(
+  app_id: int, job_path: Path, *, wait_for_ready: bool = False,
+) -> list[str]:
+  """Build the common supervisor command for one app job.
 
+  Bootstrap installs happen inside FastAPI's lifespan, before the server can
+  answer the capability calls the supervisor makes.  Only that launch path
+  needs to wait for the already-defined readiness contract; ordinary cron and
+  manual jobs run against an already-serving backend.
+  """
+  command = [sys.executable, str(runner_script())]
+  if wait_for_ready:
+    command.append("--wait-for-ready")
+  command.extend((str(app_id), str(job_path)))
+  return command
 
-def launch_app_job(app_id: int, job_path: Path, source_dir: Path):
+
+def launch_app_job(
+  app_id: int, job_path: Path, source_dir: Path, *, wait_for_ready: bool = False,
+):
   """Launch the common wrapper detached from the API worker's pipes."""
+  env = dict(os.environ)
+  # The runner is also invoked by cron, so it cannot rely on the shell's
+  # localhost default.  Every direct launch receives the same configured base
+  # URL the backend itself uses.
+  env["API_BASE_URL"] = get_settings().api_base_url
   return subprocess.Popen(
-    runner_command(app_id, job_path),
+    runner_command(app_id, job_path, wait_for_ready=wait_for_ready),
     stdout=subprocess.DEVNULL,
     stderr=subprocess.DEVNULL,
     cwd=str(source_dir),
+    env=env,
     close_fds=True,
     start_new_session=True,
   )