Replace actor eth0 move with veth networking#1
Merged
Conversation
EItanya
force-pushed
the
transparent-egress-veth-networking
branch
from
May 29, 2026 13:59
b66b5d1 to
2e8c8fd
Compare
EItanya
force-pushed
the
main
branch
2 times, most recently
from
June 4, 2026 16:25
e12128d to
727d8aa
Compare
EItanya
force-pushed
the
transparent-egress-veth-networking
branch
from
June 5, 2026 15:54
2e5e6c7 to
3f5cbae
Compare
What was happening:
1. runsc checkpoint pause succeeded.
2. After that, substrate tried to run runsc state and runsc delete for the app container.
3. But after checkpointing the root sandbox container, the runsc control server was no longer usable,
so those post-checkpoint commands failed with connection refused.
4. That made CheckpointWorkload return an error even though the actual checkpoint had already
succeeded.
5. The ActorTemplate stayed stuck in WaitGoldenActor.
The change removes the post-checkpoint state/delete calls from ateom-gvisor. After checkpoint
succeeds, it only cleans up the actor network and returns success. This matches the existing contract
in the code: atelet owns resetting the actor directories after checkpoint/upload, and it already
calls resetActorDirs(...).
Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>
EItanya
force-pushed
the
transparent-egress-veth-networking
branch
from
June 8, 2026 22:27
3f5cbae to
326110a
Compare
EItanya
added a commit
that referenced
this pull request
Jun 9, 2026
* Replace actor eth0 move with veth networking
* Group nftables rules with chain definitions
* Address actor inbound networking TODOs
* Trim actor networking comment
* Update licenses for nftables dependency
* Fix gvisor cleanup issue
What was happening:
1. runsc checkpoint pause succeeded.
2. After that, substrate tried to run runsc state and runsc delete for the app container.
3. But after checkpointing the root sandbox container, the runsc control server was no longer usable,
so those post-checkpoint commands failed with connection refused.
4. That made CheckpointWorkload return an error even though the actual checkpoint had already
succeeded.
5. The ActorTemplate stayed stuck in WaitGoldenActor.
The change removes the post-checkpoint state/delete calls from ateom-gvisor. After checkpoint
succeeds, it only cleans up the actor network and returns success. This matches the existing contract
in the code: atelet owns resetting the actor directories after checkpoint/upload, and it already
calls resetActorDirs(...).
Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>
* Address actor network cleanup review
* Fix ateom cleanup failure handling
---------
Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>
EItanya
added a commit
that referenced
this pull request
Jun 10, 2026
* Replace actor eth0 move with veth networking
* Group nftables rules with chain definitions
* Address actor inbound networking TODOs
* Trim actor networking comment
* Update licenses for nftables dependency
* Fix gvisor cleanup issue
What was happening:
1. runsc checkpoint pause succeeded.
2. After that, substrate tried to run runsc state and runsc delete for the app container.
3. But after checkpointing the root sandbox container, the runsc control server was no longer usable,
so those post-checkpoint commands failed with connection refused.
4. That made CheckpointWorkload return an error even though the actual checkpoint had already
succeeded.
5. The ActorTemplate stayed stuck in WaitGoldenActor.
The change removes the post-checkpoint state/delete calls from ateom-gvisor. After checkpoint
succeeds, it only cleans up the actor network and returns success. This matches the existing contract
in the code: atelet owns resetting the actor directories after checkpoint/upload, and it already
calls resetActorDirs(...).
Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>
* Address actor network cleanup review
* Fix ateom cleanup failure handling
---------
Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>
EItanya
added a commit
that referenced
this pull request
Jun 10, 2026
* Replace actor eth0 move with veth networking
* Group nftables rules with chain definitions
* Address actor inbound networking TODOs
* Trim actor networking comment
* Update licenses for nftables dependency
* Fix gvisor cleanup issue
What was happening:
1. runsc checkpoint pause succeeded.
2. After that, substrate tried to run runsc state and runsc delete for the app container.
3. But after checkpointing the root sandbox container, the runsc control server was no longer usable,
so those post-checkpoint commands failed with connection refused.
4. That made CheckpointWorkload return an error even though the actual checkpoint had already
succeeded.
5. The ActorTemplate stayed stuck in WaitGoldenActor.
The change removes the post-checkpoint state/delete calls from ateom-gvisor. After checkpoint
succeeds, it only cleans up the actor network and returns success. This matches the existing contract
in the code: atelet owns resetting the actor directories after checkpoint/upload, and it already
calls resetActorDirs(...).
Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>
* Address actor network cleanup review
* Fix ateom cleanup failure handling
---------
Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>
EItanya
added a commit
that referenced
this pull request
Jun 10, 2026
* Replace actor eth0 move with veth networking
* Group nftables rules with chain definitions
* Address actor inbound networking TODOs
* Trim actor networking comment
* Update licenses for nftables dependency
* Fix gvisor cleanup issue
What was happening:
1. runsc checkpoint pause succeeded.
2. After that, substrate tried to run runsc state and runsc delete for the app container.
3. But after checkpointing the root sandbox container, the runsc control server was no longer usable,
so those post-checkpoint commands failed with connection refused.
4. That made CheckpointWorkload return an error even though the actual checkpoint had already
succeeded.
5. The ActorTemplate stayed stuck in WaitGoldenActor.
The change removes the post-checkpoint state/delete calls from ateom-gvisor. After checkpoint
succeeds, it only cleans up the actor network and returns success. This matches the existing contract
in the code: atelet owns resetting the actor directories after checkpoint/upload, and it already
calls resetActorDirs(...).
Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>
* Address actor network cleanup review
* Fix ateom cleanup failure handling
---------
Signed-off-by: Eitan Yarmush <eitan.yarmush@solo.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related issue: agent-substrate#122
Summary
This replaces the
ateom-gvisornetworking path that moved the worker pod's Kubernetes-providedeth0into the actor/gVisor network namespace.Instead, the worker pod keeps its real
eth0, andateom-gvisorcreates a point-to-point veth pair between the worker pod namespace and the actor namespace. The actor-side peer is renamed toeth0, receives the actor-side address, and uses the worker-side veth as its default gateway.The PR also adds temporary nftables compatibility rules so existing inbound and outbound behavior continues to work while preserving the worker pod's own network connectivity.
Why
Moving the pod's real
eth0makes the worker pod lose normal Kubernetes network connectivity while an actor is active. That blocks pod-local networking components, including the planned transparent egress capture and AgentGateway integration, because those components remain in the worker pod namespace while actor traffic leaves through an interface that was moved elsewhere.Keeping
eth0in the worker pod namespace gives Substrate a stable worker-owned networking boundary for future transparent egress policy enforcement.Validation
go test ./cmd/ateom-gvisor ./cmd/atelet ./internal/ateompath ./internal/controllersgo test ./cmd/ateom-gvisor ./internal/serverbootNO_DEV_ENV=true BUCKET_NAME=ate-snapshots KO_DOCKER_REPO=localhost:5001 KUBECTL_CONTEXT=kind-kind ./hack/run-e2e.sh ./internal/e2e/suites/demo -run TestDemo3 -count=1Checklist