Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 40 additions & 20 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@ which owns availability and compatibility checks. The bundled deployment also
advertises Kata Containers and Firecracker on KVM-capable nodes. The native
Linux runc payload is build-time optional and must be explicitly included and
enabled by an operator.
Creation-time network policies support unrestricted networking, blocking new
flows except the YuanRong control and published sandbox-port routes, or denying
exact and leading-wildcard DNS names.
Creation-time network policies and atomic runtime replacement support
unrestricted networking, blocking new flows except the YuanRong control and
published sandbox-port routes, or denying exact and leading-wildcard DNS names.
Experimental whole-device NVIDIA GPU requests require runsc. Configurable
writable-storage requests are supported by runsc and Firecracker.

Expand Down Expand Up @@ -124,9 +124,10 @@ node components and produces the AKernel all-in-one image using the selected
runtime image and its matching service configuration.

The control-plane and RRT release version is independent of the optional
actor-based `openyuanrong_sdk` installed in the Python runtime profile. Keep
the latter on its explicitly pinned legacy version unless that backend is
being upgraded and tested as a separate compatibility change.
actor-based `openyuanrong_sdk` installed in the Python runtime profile. This
actor backend is deprecated and retained only for compatibility with existing
applications. Keep it on its explicitly pinned legacy version; do not advance
it with the default `openyuanrong-sandbox` backend or use it for new features.

Initialize submodules with `git submodule update --init --recursive` before
building. The all-in-one image builds the sandboxd binaries, including
Expand Down Expand Up @@ -219,14 +220,19 @@ usable `/dev/kvm` device.

The bundled sandboxd configuration enables per-sandbox network ACLs. Pooled TAP
networking requires the host `tun` module and a usable `/dev/net/tun`. The
default iptables backend additionally requires `br_netfilter`, conntrack,
connmark/CONNMARK, and bridge netfilter. The optional bpfnat backend instead
requires eBPF `SCHED_CLS`, TC `clsact`, writable bpffs, and permission to load
BPF programs and manage TC filters. Both require free TCP/UDP port 53 on the
sandbox bridge. Drain existing sandboxes before enabling ACLs or upgrading a
default iptables backend additionally requires `iptables`, `ip6tables`,
`ipset`, IPv4/IPv6 filter tables, `br_netfilter`, `xt_physdev`, conntrack,
conntrack-netlink, connmark/CONNMARK, timeout-capable `hash:ip` sets, and
IPv4/IPv6 bridge netfilter. The optional bpfnat backend
instead requires Linux 5.17 or newer for `bpf_loop`, eBPF `SCHED_CLS`, TC
`clsact`, writable bpffs, and permission to load BPF programs and manage TC
filters. Both require free TCP/UDP port 53 on the sandbox bridge. Drain
existing sandboxes before enabling ACLs or upgrading a
node from a pre-ACL configuration; sandboxd refuses to initialize ACLs when
old sandbox records remain. A sandbox without a network policy stays
unrestricted. See `deploy/README.md` for deployment requirements and
unrestricted. Schema v2 supports independent ingress and egress defaults,
allow and deny rules over IPv4 CIDRs, domains, protocols, and ports, plus an
independent DNS policy. See `deploy/README.md` for deployment requirements and
`sdk/python/README.md` for API limits.

Dragonfly distribution is optional and disabled by default. Enable it during
Expand Down Expand Up @@ -352,6 +358,18 @@ with Sandbox(network_policy=NetworkPolicy.block()) as sb:
print(sb.commands.run("echo control-plane-access").stdout)
```

Configure a generic egress allowlist:

```python
from akernel_sdk import NetworkPolicy, NetworkRule

policy = NetworkPolicy.allowlist(
[NetworkRule(domain="*.example.com", protocol="tcp", port_range=443)]
)
with Sandbox(network_policy=policy) as sb:
print(sb.commands.run("curl https://api.example.com").stdout)
```

Required environment:

```bash
Expand Down Expand Up @@ -428,14 +446,16 @@ make sdk-check
```

The Python SDK installs `openyuanrong-sandbox` as its default execution
backend. The actor-based `openyuanrong-sdk` backend is available through the
`openyuanrong-sdk` extra. Installing that extra leaves both distributions
present, so `openyuanrong-sandbox` remains the automatic default unless
`AKERNEL_BACKEND=openyuanrong-sdk` is set before import. Backend selection
happens once during import and backend modules are loaded lazily on first use.
Keep public `Sandbox`, `Commands`, `Filesystem`, and value types independent
of both native packages; all native conversions belong under
`akernel_sdk._backends`.
backend. The actor-based `openyuanrong-sdk` backend is deprecated and retained
only for compatibility with existing applications through the
`openyuanrong-sdk` extra. Do not update its pinned legacy version alongside
the default backend or extend it with new capabilities. Installing that extra
leaves both distributions present, so `openyuanrong-sandbox` remains the
automatic default unless `AKERNEL_BACKEND=openyuanrong-sdk` is set before
import. Backend selection happens once during import and backend modules are
loaded lazily on first use. Keep public `Sandbox`, `Commands`, `Filesystem`,
and value types independent of both native packages; all native conversions
belong under `akernel_sdk._backends`.

Dockerfile direct launch is a supported AKernel SDK capability through
`DockerContext` and
Expand Down
11 changes: 7 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,12 +119,15 @@ python -m pip install akernel-sdk
# Source
python -m pip install ./sdk/python

# Also install the actor backend
# Also install the deprecated actor compatibility backend
python -m pip install "akernel-sdk[openyuanrong-sdk]"
```

When the actor extra is installed, both backend packages are present and
`openyuanrong-sandbox` remains the automatic default. Set
The actor-based `openyuanrong-sdk` backend is deprecated and retained only for
compatibility with existing applications. New applications should use the
default `openyuanrong-sandbox` backend. When the actor extra is installed,
both backend packages are present and `openyuanrong-sandbox` remains the
automatic default. Set
`AKERNEL_BACKEND=openyuanrong-sdk` before importing `akernel_sdk` to select
the actor backend:

Expand Down Expand Up @@ -206,7 +209,7 @@ See the complete [basic usage example](./sdk/python/examples/basic_usage.py), th
- [x] Kata Containers runtime on KVM-capable nodes
- [x] Firecracker microVM runtime on KVM-capable nodes
- [x] Optional native Linux runc runtime
- [x] Sandbox network ACL
- [x] Stateful sandbox network ACLs for CIDRs, domains, protocols, and ports
- [ ] Fork-based sandbox launch based on gVisor
- [x] Same-node checkpoint recovery for runsc and Firecracker
- [ ] Support for GKE and AWS
Expand Down
7 changes: 4 additions & 3 deletions builder/node.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@ ARG AKERNEL_ENABLE_RUNC=false
ARG AKERNEL_ENABLE_FIRECRACKER=true
ARG SANDBOXD_BUILD_IMAGE=golang:1.25.5-bookworm
ARG DISTILL_FS_BUILD_IMAGE=rust:1.85.0-bookworm
ARG OPEN_YR_VERSION=0.10.1rc3
ARG OPEN_YR_VERSION=0.10.1rc4
ARG OPEN_YR_CORE_WHEEL_URL=
ARG OPEN_YR_CORE_WHEEL_SHA256=
ARG OPEN_YR_RELEASE_BASE_URL=https://openyuanrong.obs.cn-southwest-2.myhuaweicloud.com/release
ARG OPEN_YR_CORE_AMD64_SHA256=d303a25587919ce64bae8a7e193cab4b0f33ef12213968fa96a1ba563df629d7
ARG OPEN_YR_CORE_ARM64_SHA256=c611e4e2e4c08e696b60bf8ad394261b2a2bca5b18100b8e84351c2cbc289f1c
ARG OPEN_YR_CORE_AMD64_SHA256=65c1f27e7e700a253a2e907dea0273e85f1c76610e48c93544caa6bcc07ac3af
ARG OPEN_YR_CORE_ARM64_SHA256=29d25c3388c2913346035ee8df9b8159e702de7a8e6f783e3e218ab773896333
ARG GVISOR_DOWNLOAD_IMAGE=ubuntu:24.04
ARG GVISOR_RELEASE
ARG GVISOR_AMD64_URL
Expand Down Expand Up @@ -243,6 +243,7 @@ RUN apt-get update && \
fuse3 \
gnupg \
iproute2 \
ipset \
iptables \
jq \
kmod \
Expand Down
4 changes: 2 additions & 2 deletions builder/runtime.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,14 @@ ARG PYTHON_311_VERSION=3.11.13
ARG PYTHON_312_VERSION=3.12.11
ARG PYTHON_313_VERSION=3.13.5
ARG PYTHON_314_VERSION=3.14.6
ARG OPEN_YR_VERSION=0.10.1rc3
ARG OPEN_YR_VERSION=0.10.1rc4
ARG OPEN_YR_LEGACY_SDK_VERSION=0.9.9

FROM ${AKERNEL_RUNTIME_BASE_IMAGE} AS rrt-download

ARG OPEN_YR_VERSION
ARG RRT_RUNTIME_URL=https://openyuanrong.obs.cn-southwest-2.myhuaweicloud.com/release/${OPEN_YR_VERSION}/linux/amd64/rrt-runtime-amd64
ARG RRT_RUNTIME_SHA256=c22d3c95b38845763e9f27553a065a959458743bbf42358c609bc2f1f9fce789
ARG RRT_RUNTIME_SHA256=7c2064531e91fba8b9bbe96ac9e706cd496a198551657ee5461a078a2f7ea9ea

RUN apt-get update && \
apt-get install -y --no-install-recommends ca-certificates curl && \
Expand Down
4 changes: 3 additions & 1 deletion builder/scripts/sandboxd_network_prepare.sh
Original file line number Diff line number Diff line change
Expand Up @@ -57,11 +57,13 @@ fi
# netfilter hooks. Host provisioning must load br_netfilter; this hook only
# configures the node container's network namespace.
if [[ "${enable_network_acl,,}" == "true" && "${nat_backend}" == "iptables" ]]; then
if [[ ! -e /proc/sys/net/bridge/bridge-nf-call-iptables ]]; then
if [[ ! -e /proc/sys/net/bridge/bridge-nf-call-iptables ||
! -e /proc/sys/net/bridge/bridge-nf-call-ip6tables ]]; then
echo "br_netfilter is unavailable; load it on the host before starting the AKernel node" >&2
exit 1
fi
"${SYSCTL_BIN}" -w net.bridge.bridge-nf-call-iptables=1
"${SYSCTL_BIN}" -w net.bridge.bridge-nf-call-ip6tables=1
fi

# bpfnat validates this setting when its local-DNAT path is enabled. Apply it
Expand Down
28 changes: 17 additions & 11 deletions deploy/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,17 +57,21 @@ The bundled standalone, Helm, and Terraform sandboxd configurations enable
per-sandbox network ACLs. Runsc, Kata, and Firecracker require the host `tun`
module and a usable `/dev/net/tun` for their pooled TAP endpoints. A sandbox
created without a policy remains on the unrestricted fast path. The default
`iptables` backend additionally requires `ip_tables`, `br_netfilter`,
conntrack, connmark/CONNMARK support, and
`net.bridge.bridge-nf-call-iptables=1`. The optional `bpfnat` backend instead
requires Linux eBPF `SCHED_CLS`, TC `clsact`, supported hash and array maps, a
writable bpffs at `/sys/fs/bpf` (or permission to mount one), and permission
to load BPF programs and manage TC filters. Both backends require TCP and UDP
port 53 on the sandbox bridge to be free and at least one usable upstream
nameserver. AKernel's privileged node container prepares the selected
backend's namespace-local settings. Host provisioning must load the required
kernel modules before the node pod starts; the Terraform node bootstrap does
this automatically.
`iptables` backend additionally requires the `iptables`, `ip6tables`, and
`ipset` userspace commands; IPv4/IPv6 filter-table, `br_netfilter`,
`xt_physdev`, conntrack and conntrack-netlink, connmark/CONNMARK, and
timeout-capable `hash:ip` ipset support; and both bridge netfilter sysctls for
iptables and ip6tables set to `1`. Sandboxd probes the IPv6 physdev rule and
the required ipset type when it initializes this backend. The optional `bpfnat`
backend instead requires Linux 5.17 or newer for `bpf_loop`, eBPF
`SCHED_CLS`, TC `clsact`,
supported hash and array maps, a writable bpffs at `/sys/fs/bpf` (or
permission to mount one), and permission to load BPF programs and manage TC
filters. Both backends require TCP and UDP port 53 on the sandbox bridge to be
free and at least one usable upstream nameserver. AKernel's privileged node
container prepares the selected backend's namespace-local settings. Host
provisioning must load the required kernel modules before the node pod starts;
the Terraform node bootstrap does this automatically.

Drain all sandboxes from a node before enabling ACLs or upgrading an existing
deployment to a release that enables them. Sandboxd deliberately refuses to
Expand All @@ -78,6 +82,8 @@ sandboxd is healthy.
Sandboxd selects the ACL implementation matching the configured `iptables` or
`bpfnat` NAT backend. DNS policies manage each sandbox's `/etc/resolv.conf`; a
caller mount that owns that path is rejected while ACL support is enabled.
Schema v2 domain traffic rules also use the managed DNS proxy to install
TTL-bound address grants, even when no separate DNS policy is supplied.

`make config` is interactive by default. It writes:

Expand Down
7 changes: 4 additions & 3 deletions deploy/standalone/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,9 +92,10 @@ later creation of `sandbox0` cannot change the advertised node address. Set
override.

The standalone configuration enables per-sandbox network ACLs. With the
default iptables backend, `start.sh` loads `br_netfilter` on the host before
the node starts; the node then enables bridge netfilter in its own network
namespace. The host also requires conntrack plus connmark/CONNMARK support.
default iptables backend, `start.sh` loads IPv6 filter-table, `br_netfilter`,
`xt_physdev`, conntrack/connmark, and timeout-capable ipset modules on the host
before the node starts; the node then enables IPv4 and IPv6 bridge netfilter in
its own network namespace.
The optional bpfnat backend instead
requires TC eBPF support and a writable bpffs. TCP and UDP port 53 on the
sandbox bridge must remain free for sandboxd's managed DNS proxy. Before
Expand Down
33 changes: 29 additions & 4 deletions deploy/standalone/start.sh
Original file line number Diff line number Diff line change
Expand Up @@ -308,19 +308,44 @@ prepare_host_network_modules() {
fi

if [[ "$(id -u)" -eq 0 ]]; then
"${modprobe_bin}" ip_tables
"${modprobe_bin}" iptable_filter
"${modprobe_bin}" ip6_tables
"${modprobe_bin}" ip6table_filter
"${modprobe_bin}" br_netfilter
elif sudo -n "${modprobe_bin}" br_netfilter; then
"${modprobe_bin}" xt_physdev
"${modprobe_bin}" nf_conntrack
"${modprobe_bin}" nf_conntrack_netlink
"${modprobe_bin}" xt_conntrack
"${modprobe_bin}" xt_connmark
"${modprobe_bin}" ip_set
"${modprobe_bin}" ip_set_hash_ip
"${modprobe_bin}" xt_set
elif sudo -n "${modprobe_bin}" ip_tables &&
sudo -n "${modprobe_bin}" iptable_filter &&
sudo -n "${modprobe_bin}" ip6_tables &&
sudo -n "${modprobe_bin}" ip6table_filter &&
sudo -n "${modprobe_bin}" br_netfilter &&
sudo -n "${modprobe_bin}" xt_physdev &&
sudo -n "${modprobe_bin}" nf_conntrack &&
sudo -n "${modprobe_bin}" nf_conntrack_netlink &&
sudo -n "${modprobe_bin}" xt_conntrack &&
sudo -n "${modprobe_bin}" xt_connmark &&
sudo -n "${modprobe_bin}" ip_set &&
sudo -n "${modprobe_bin}" ip_set_hash_ip &&
sudo -n "${modprobe_bin}" xt_set; then
:
else
log_error "Unable to load br_netfilter; run this script as root or allow passwordless sudo for modprobe"
log_error "Unable to load required iptables ACL modules; run this script as root or allow passwordless sudo for modprobe"
exit 1
fi

if [[ ! -e /proc/sys/net/bridge/bridge-nf-call-iptables ]]; then
if [[ ! -e /proc/sys/net/bridge/bridge-nf-call-iptables ||
! -e /proc/sys/net/bridge/bridge-nf-call-ip6tables ]]; then
log_error "br_netfilter loaded but bridge netfilter sysctls are unavailable"
exit 1
fi
log_info "Loaded host br_netfilter module for the iptables ACL backend"
log_info "Loaded host filter, bridge, conntrack, and ipset modules for the iptables ACL backend"
}

# Start the AKernel all-in-one container. Traefik runs separately so traffic
Expand Down
2 changes: 1 addition & 1 deletion deploy/terraform/aliyun/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -321,7 +321,7 @@ variable "extra_node_pools" {

variable "sandboxd_nat_backend" {
type = string
description = "Sandboxd NAT backend. When set to 'iptables', ip_tables kernel module will be loaded at boot."
description = "Sandboxd NAT backend. The iptables mode loads IPv4/IPv6 bridge-netfilter modules at boot."
default = "iptables"
}

Expand Down
2 changes: 1 addition & 1 deletion deploy/terraform/huaweicloud/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -365,7 +365,7 @@ variable "extra_node_pools" {

variable "sandboxd_nat_backend" {
type = string
description = "Sandboxd NAT backend. When set to 'iptables', ip_tables kernel module will be loaded at boot."
description = "Sandboxd NAT backend. The iptables mode loads IPv4/IPv6 bridge-netfilter modules at boot."
default = "iptables"
}

Expand Down
5 changes: 3 additions & 2 deletions deploy/terraform/shared/node-bootstrap.sh.tftpl
Original file line number Diff line number Diff line change
Expand Up @@ -27,15 +27,16 @@ systemctl enable akernel-tun-module.service
systemctl start akernel-tun-module.service

%{ if sandboxd_nat_backend == "iptables" ~}
# ip_tables provides NAT and br_netfilter provides bridged ACL enforcement.
# ip_tables provides NAT. IPv6 bridge filtering prevents traffic from bypassing
# the IPv4-only ACL policy surface.
cat >/etc/systemd/system/akernel-network-modules.service <<'EOF'
[Unit]
Description=Load AKernel iptables networking modules on boot
After=network-pre.target

[Service]
Type=oneshot
ExecStart=/bin/sh -c 'modprobe ip_tables && modprobe br_netfilter'
ExecStart=/bin/sh -c 'modprobe ip_tables && modprobe iptable_filter && modprobe ip6_tables && modprobe ip6table_filter && modprobe br_netfilter && modprobe xt_physdev && modprobe nf_conntrack && modprobe nf_conntrack_netlink && modprobe xt_conntrack && modprobe xt_connmark && modprobe ip_set && modprobe ip_set_hash_ip && modprobe xt_set'
RemainAfterExit=yes

[Install]
Expand Down
Loading