Skip to content

Commit b1e0e3e

Browse files
committed
Merge remote-tracking branch 'upstream/main'
2 parents 08da3e9 + 081afc8 commit b1e0e3e

169 files changed

Lines changed: 24139 additions & 2162 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.agents/skills/build-from-issue/SKILL.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,3 +34,5 @@ Every PR from this issue-backed workflow must have its own existing issue and us
3434
Do not apply acceptance or roadmap decisions on behalf of a maintainer. Do not introduce `agent:*` workflow labels.
3535

3636
For first-party security-sensitive crypto, use `openshell-crypto` and extend its facade/backend contracts when needed. Follow the [crypto architecture](../../../crates/openshell-crypto/CRYPTO.md) for compatibility, failure handling, and explicit coverage exceptions.
37+
38+
When changing crypto dependencies, run `mise run rust:deny:policy`, including its dependency-boundary regression fixtures. Review any `deny.toml` wrapper change against the documented exception scope; an existing allowance does not authorize new security-sensitive operations outside the facade.

‎.agents/skills/fix-security-issue/SKILL.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,3 +18,5 @@ Use this skill after an authorized `review-security-issue` review identifies an
1818
Begin any fix comments with `> **🔧 security-fix-agent**`. Do not change human disposition or introduce `agent:*` workflow labels.
1919

2020
For first-party security-sensitive crypto, use `openshell-crypto` and extend its facade/backend contracts when needed. Follow the [crypto architecture](../../../crates/openshell-crypto/CRYPTO.md) for compatibility, failure handling, and explicit coverage exceptions.
21+
22+
When changing crypto dependencies, run `mise run rust:deny:policy`, including its dependency-boundary regression fixtures. Review any `deny.toml` wrapper change against the documented exception scope; an existing allowance does not authorize new security-sensitive operations outside the facade.

‎.agents/skills/helm-dev-environment/SKILL.md‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -421,6 +421,30 @@ above) instead of relying only on `OPENSHELL_GATEWAY_ENDPOINT`; some tests call
421421
### Kubernetes E2E Notes
422422

423423
Use `mise run e2e:kubernetes` for the standard Helm-backed Kubernetes suite.
424+
Set `OPENSHELL_E2E_KUBE_TEST=kubernetes_image_update` to scope the Rust suite to
425+
combined image, command, environment, and metadata updates with retained sandbox,
426+
SSH, and PVC identities. The next start verifies the new argv and environment. This target is
427+
included in both existing Agent Sandbox API-version lanes; unlike the Python
428+
Docker suite, it exercises the driver that supports image updates.
429+
430+
Image updates default to disabled outside test fixtures. The e2e wrapper opts
431+
in only for a fresh gateway fixture that it owns; set
432+
`OPENSHELL_E2E_KUBE_IMAGE_UPDATES=0` to prevent that opt-in. Do not propagate
433+
activation to an existing shared gateway or enable it during a mixed-version
434+
rollout. For manual testing, follow the retirement and activation checklist in
435+
`docs/kubernetes/high-availability.mdx` before setting
436+
`gatewayConfig: {"openshell.gateway": {enable_sandbox_updates: true}}`.
437+
The gateway process equivalents are `--enable-sandbox-updates true` and
438+
`OPENSHELL_ENABLE_SANDBOX_UPDATES=true`. Retire every old process and Job,
439+
settle outstanding driver work, prevent old binaries returning, and do not
440+
treat PostgreSQL connection markers or membership TTL expiry as retirement
441+
proof. SQLite testing supports one gateway process only. Image-update clients
442+
must call the separate `SandboxUpdates` service on the existing gateway
443+
listener, not add a method to the baseline `OpenShell` service or inspect new
444+
public `SandboxStatus` image fields. Resume a pending operation with
445+
`SandboxUpdates.UpdateSandbox`, the current resource version, and no selected
446+
fields, following the documented request-ID semantics.
447+
424448
The kube e2e wrapper creates only one port-forward, to `svc/openshell`; it no
425449
longer forwards the unauthenticated health listener or runs a `/readyz` e2e
426450
target. `/readyz` remains covered by server unit/integration tests.
@@ -611,3 +635,13 @@ for dependencies still declared in `Chart.yaml`.
611635
| `deploy/kube/manifests/envoy-gateway-openshell.yaml` | GatewayClass and BackendTrafficPolicy for Envoy Gateway (`mise run helm:gateway:apply`) |
612636
| `tasks/scripts/helm-k3s-local.sh` | k3d cluster create/delete/start/stop/status |
613637
| `tasks/scripts/keycloak-k8s-setup.sh` | Keycloak deploy, realm import, and development TLS trust anchor |
638+
639+
## Sandbox update regression lane
640+
641+
Run `mise run e2e:kubernetes:updates` for the testkit update suite against both
642+
supported Agent Sandbox APIs with the combined Helm chart and its bundled
643+
workspace resources. It verifies stopped-only updates, retained CR/PVC
644+
identity and contents, command/environment changes, and request replay.
645+
`OPENSHELL_E2E_UPDATE_ORIGINAL_IMAGE` and `OPENSHELL_E2E_UPDATE_TARGET_IMAGE` can
646+
select mirrored Alpine 3.21 and 3.22 manifests. Defaults use the public registry
647+
cache; keep immutable digests when overriding them.

‎.agents/skills/launch-openshell-gator/SKILL.md‎

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,22 @@ If this fails, run the local Codex login flow outside the gator launch. If Codex
8282

8383
The launcher passes the non-secret host account ID as literal `CODEX_ACCOUNT_ID` for Codex's local workspace routing. The harness must not use the opaque `CODEX_AUTH_ACCOUNT_ID` credential placeholder for that comparison. Access tokens remain placeholders, and refresh material remains gateway-only.
8484

85+
The image contains task-independent tooling and a prompt template, not the run's PR assignment or test authorization. The launcher supplies those instructions at runtime through `OPENSHELL_AGENT_TASK` in the canonical process command. The supervisor captures the assembled prompt in memory and supplies it on stdin for every cycle. Task text is non-secret and may appear in runtime command metadata; never put credentials in the task. Task-only changes reuse the same tooling build inputs, although the launcher still builds/tags an image per invocation.
86+
87+
For Kubernetes gateways, select image delivery explicitly: `--k3d-cluster NAME`
88+
builds with Docker and imports into a running local cluster with the matching
89+
current kube context; `--publish-image REGISTRY/REPOSITORY:TAG` builds and pushes,
90+
then launches by resolved digest; `--image REGISTRY/REPOSITORY@sha256:DIGEST`
91+
reuses trusted compatible tooling without local image tools. Do not infer
92+
locality from a loopback endpoint. The k3d selection asserts which cluster
93+
serves the gateway; imported images require `if_not_present` or `never` pull
94+
policy. Registry builds use the builder's platform and existing registry auth;
95+
nodes need their own pull access. `--build-only` builds/delivers without
96+
gateway mutations, provider credential reads, or a sandbox launch. Reused
97+
images must match the local manifest's agent ID and payload version and support
98+
the runtime task contract; legacy baked-task images fail startup. See
99+
`scripts/agents/gator/README.md` for examples and option compatibility.
100+
85101
### Step 5: Verify Gateway Is Registered And Alive
86102

87103
Use the target gateway from the operator request or current session context. Do not assume a gateway name. If the operator did not specify one, list registered gateways and ask before launching when the correct target is ambiguous.
@@ -163,7 +179,7 @@ sandbox_name="gtr-pr-${pr_number}-watch"
163179
"Review and monitor PR #${pr_number} through the gator-gate workflow. Scope this invocation only to PR #${pr_number}."
164180
```
165181

166-
The launcher queries the gateway's selected compute driver, builds the gator image in the matching Docker or Podman image store, stages the immutable payload, imports provider profiles, configures provider credentials and refresh, and starts the agent supervisor as the sandbox's canonical main process. The detached main process survives loss of the host CLI connection and reconnects to a restarted gateway. Unless `--keep` is set, the sandbox is marked ephemeral so the gateway deletes it after the canonical main process exits and its terminal result is finalized. `CONTAINER_ENGINE`, when set, must match the gateway driver.
182+
The launcher queries the gateway's selected compute driver and prepares the image through the selected build/import/publish or prebuilt path before importing provider profiles, configuring provider credentials and refresh, and starting the agent supervisor as the sandbox's canonical main process. The detached main process survives loss of the host CLI connection and reconnects to a restarted gateway. Unless `--keep` is set, the sandbox is marked ephemeral so the gateway deletes it after the canonical main process exits and its terminal result is finalized. `CONTAINER_ENGINE`, when set, must match Docker/Podman gateways or Docker for k3d imports; registry builds for Kubernetes may use either local engine.
167183

168184
The launcher streams image-build and provisioning output until the detached workload is ready, then exits. Use `openshell logs <sandbox-name>` or the TUI for runtime output.
169185

‎.agents/skills/review-github-pr/SKILL.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -153,7 +153,7 @@ Read through the full diff (and the PR description if available). Produce a summ
153153
suggested change.
154154
- Do not fabricate concerns or claim a behavioral regression without evidence
155155
for both the prior and proposed behavior.
156-
- **Cryptographic boundaries**: For changed first-party security-sensitive crypto, verify use of `openshell-crypto`, backend dispatch, failure propagation without fallback, and protocol/storage compatibility. Missing operations belong in the facade/backend contracts. Check dependency-owned and non-security exceptions against the repository's [crypto architecture](../../../crates/openshell-crypto/CRYPTO.md); protocol/parsing types alone are not bypasses. Report undocumented bypasses or scope expansion under **Potential Concerns**.
156+
- **Cryptographic boundaries**: For changed first-party security-sensitive crypto, verify use of `openshell-crypto`, backend dispatch, failure propagation without fallback, and protocol/storage compatibility. Missing operations belong in the facade/backend contracts. Check dependency-owned and other explicit exceptions against the repository's [crypto architecture](../../../crates/openshell-crypto/CRYPTO.md); protocol/parsing types alone are not bypasses. Check `mise run rust:deny:policy` results and review changes to `deny.toml` wrappers; package-level allowances do not authorize new bypasses within those packages. Report undocumented bypasses or scope expansion under **Potential Concerns**.
157157
- **Agent infrastructure**: When the PR changes behavior, commands, or development workflows, use the `sync-agent-infra` maintenance map to check that related skills were updated. When it adds, removes, or renames skills or crates; changes workflow relationships or skill coverage; modifies issue or PR templates; or changes agent cross-references, apply the full consistency checklist. Report missing companion updates or drift under **Potential Concerns**.
158158

159159
## Step 5: Output

‎.agents/skills/watch-github-actions/SKILL.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -164,6 +164,13 @@ not substitute the current `main` tip or the event's older PR base SHA. Merge
164164
groups and manual runs use their explicit baseline. Findings are reported by
165165
`Reject new high or critical findings`; distinguish those from scanner failures.
166166

167+
For `Cargo Deny`, the metadata-only commands clear Rust compiler wrappers after
168+
entering mise; they do not require sccache. For OpenSSL backend and dynamic-linkage
169+
checks, use `nix develop .#crypto` from outside the default shell. The release
170+
toolchains in the default shell expose an unprefixed system AWS-LC that conflicts
171+
with OpenSSL. The crypto shell uses native OpenSSL and Cargo's symbol-prefixed
172+
AWS-LC build for dual-backend tests.
173+
167174
For `Protobuf Compatibility`, check the logged train and comparison baseline.
168175
Branch Checks compares the prospective merge tree with its target; release qualification
169176
compares the tagged candidate with the previous stable release. Both use

‎.github/workflows/branch-checks.yml‎

Lines changed: 53 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -197,8 +197,17 @@ jobs:
197197
prepare-shell: "true"
198198
shell-installable: .#devShells.x86_64-linux.default
199199

200+
- name: Install pinned cargo-deny
201+
run: mise install --locked github:EmbarkStudios/cargo-deny
202+
203+
# Enter mise inside the Nix shell so Nix's cargo-deny cannot shadow the pin.
204+
# mise sets RUSTC_WRAPPER=sccache, but this metadata-only job does not
205+
# install sccache. Clear wrappers after mise applies its environment.
206+
- name: Verify crypto dependency guardrail
207+
run: mise exec github:EmbarkStudios/cargo-deny -- env RUSTC_WRAPPER= RUSTC_WORKSPACE_WRAPPER= uv run --no-project python tasks/scripts/test-crypto-dependency-policy.py
208+
200209
- name: Check dependencies
201-
run: cargo deny check licenses bans sources
210+
run: mise exec github:EmbarkStudios/cargo-deny -- env RUSTC_WRAPPER= RUSTC_WORKSPACE_WRAPPER= cargo deny check licenses bans sources
202211

203212
rust-format:
204213
name: Rust format
@@ -226,6 +235,8 @@ jobs:
226235
cargo fmt --manifest-path tests/tmachine/Cargo.toml --all -- --check
227236
cargo fmt --manifest-path tests/runner/Cargo.toml --all -- --check
228237
cargo fmt --manifest-path tests/suites/conformance/Cargo.toml -p sandbox -- --check
238+
cargo fmt --manifest-path tests/suites/features/Cargo.toml -p keycloak-provider-refresh -- --check
239+
cargo fmt --manifest-path tests/support/Cargo.toml --all -- --check
229240
230241
cargo-lockfiles:
231242
name: Cargo lockfiles
@@ -288,6 +299,8 @@ jobs:
288299
cargo clippy --locked --manifest-path tests/tmachine/Cargo.toml --all-targets -- -D warnings
289300
cargo clippy --locked --manifest-path tests/runner/Cargo.toml --all-targets -- -D warnings
290301
cargo clippy --locked --manifest-path tests/suites/conformance/Cargo.toml -p sandbox --all-targets -- -D warnings
302+
cargo clippy --locked --manifest-path tests/suites/features/Cargo.toml -p keycloak-provider-refresh --all-targets -- -D warnings
303+
cargo clippy --locked --manifest-path tests/support/Cargo.toml --all-targets -- -D warnings
291304
292305
rust-test:
293306
name: Rust test (${{ matrix.system }})
@@ -321,14 +334,52 @@ jobs:
321334
- name: Verify crypto backend independence
322335
shell: nix develop -c bash -euo pipefail {0}
323336
run: |
324-
cargo tree --locked -p openshell-crypto --no-default-features --edges normal,build,dev --prefix none > /tmp/openshell-crypto-dependencies.txt
337+
cargo tree --locked -p openshell-crypto --no-default-features --edges normal,build,dev --target all --prefix none > /tmp/openshell-crypto-dependencies.txt
325338
if grep -Eq '^(ring|aws-lc-rs|aws-lc-sys|aws-lc-fips-sys) v' /tmp/openshell-crypto-dependencies.txt; then
326339
echo "ERROR: backend-free crypto build includes Ring or AWS-LC" >&2
327340
cat /tmp/openshell-crypto-dependencies.txt >&2
328341
exit 1
329342
fi
330343
cargo test --locked -p openshell-crypto --no-default-features --test independent_backend
331344
345+
- name: Verify OpenSSL backend
346+
shell: nix develop .#crypto -c bash -euo pipefail {0}
347+
env:
348+
OPENSSL_NO_VENDOR: "1"
349+
OPENSSL_STATIC: "0"
350+
run: |
351+
cargo tree --locked -p openshell-crypto --no-default-features --features openssl --edges normal,build,dev --target all --prefix none > /tmp/openshell-openssl-dependencies.txt
352+
if grep -Eq '^(ring|aws-lc-rs|aws-lc-sys|aws-lc-fips-sys|openssl-src) v' /tmp/openshell-openssl-dependencies.txt; then
353+
echo "ERROR: OpenSSL-only crypto build includes another backend or vendored OpenSSL" >&2
354+
cat /tmp/openshell-openssl-dependencies.txt >&2
355+
exit 1
356+
fi
357+
cargo clippy --locked -p openshell-crypto --no-default-features --features openssl --all-targets -- -D warnings
358+
cargo test --locked -p openshell-crypto --no-default-features --features openssl
359+
for feature in tls-tonic tls-kube tls-sqlx; do
360+
cargo check --locked -p openshell-crypto --no-default-features --features "openssl,$feature"
361+
done
362+
cargo clippy --locked -p openshell-crypto --features openssl --all-targets -- -D warnings
363+
cargo test --locked -p openshell-crypto --features openssl
364+
365+
- name: Verify dynamic OpenSSL linkage on Linux
366+
if: runner.os == 'Linux'
367+
shell: nix develop .#crypto -c bash -euo pipefail {0}
368+
env:
369+
OPENSSL_NO_VENDOR: "1"
370+
OPENSSL_STATIC: "0"
371+
run: |
372+
cargo test --locked -p openshell-crypto --no-default-features --features openssl --test openssl_tls --no-run --message-format=json > "$RUNNER_TEMP/openssl-artifacts.jsonl"
373+
executable=$(uv run --no-project python -c 'import json, sys; artifacts = [json.loads(line) for line in sys.stdin]; paths = [a["executable"] for a in artifacts if a.get("reason") == "compiler-artifact" and a.get("target", {}).get("name") == "openssl_tls" and a.get("executable")]; assert len(paths) == 1, paths; print(paths[0])' < "$RUNNER_TEMP/openssl-artifacts.jsonl")
374+
ldd "$executable" > "$RUNNER_TEMP/openssl-linkage.txt"
375+
cat "$RUNNER_TEMP/openssl-linkage.txt"
376+
for library in libssl libcrypto; do
377+
if ! grep -Eq "^[[:space:]]*$library\\.so[^[:space:]]* => /" "$RUNNER_TEMP/openssl-linkage.txt"; then
378+
echo "ERROR: $library did not resolve to a dynamic library" >&2
379+
exit 1
380+
fi
381+
done
382+
332383
- name: Test
333384
shell: nix develop -c bash -euo pipefail {0}
334385
env:

‎.github/workflows/e2e-suites.yml‎

Lines changed: 16 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,11 @@ jobs:
5050
family: conformance
5151
package: sandbox
5252
runner: ubuntu-24.04
53+
- environment: ubuntu-docker-rootful-keycloak
54+
installer: deb
55+
family: features
56+
package: keycloak-provider-refresh
57+
runner: ubuntu-24.04
5358
runs-on: ${{ matrix.runner }}
5459
timeout-minutes: 90
5560
steps:
@@ -77,13 +82,17 @@ jobs:
7782
uses: actions/cache@caa296126883cff596d87d8935842f9db880ef25 # v5.1.0
7883
with:
7984
path: ~/.cache/tmachine
80-
# tmachine keys its disk layers on a hash of the Ansible tree and
81-
# the cache versions in setup.rs and install.rs. actions/cache
82-
# doesn't save after an exact key hit, so a key that misses one of
83-
# those would rebuild the changed layers on every run. No
84-
# restore-keys fallback: restoring an older entry would carry its
85-
# stale disks into the new one, which then only grows.
86-
key: tmachine-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('tests/tmachine/ansible/**', 'tests/tmachine/images.nix', 'tests/tmachine/src/qemu/setup.rs', 'tests/tmachine/src/qemu/install.rs') }}
85+
# tmachine keys its disk layers on a hash of the Ansible tree, the
86+
# environments' playbook lists and setup inputs (config.nix, and
87+
# files such as the Keycloak realm file) and the cache versions in
88+
# setup.rs and install.rs. actions/cache doesn't save after an
89+
# exact key hit, so a key that misses one of those would rebuild the
90+
# changed layers on every run. Environments share no layers, and
91+
# only the first job to save a key gets its disks in, so each
92+
# environment and installer needs its own key. No restore-keys
93+
# fallback: restoring an older entry would carry its stale disks
94+
# into the new one, which then only grows.
95+
key: tmachine-${{ runner.os }}-${{ runner.arch }}-${{ matrix.environment }}-${{ matrix.installer }}-${{ hashFiles('tests/tmachine/ansible/**', 'tests/tmachine/config.nix', 'tests/tmachine/images.nix', 'tests/tmachine/src/qemu/setup.rs', 'tests/tmachine/src/qemu/install.rs', 'scripts/keycloak-realm.json') }}
8796

8897
- name: Download integration inputs
8998
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1

0 commit comments

Comments
 (0)