[Feature Request] Route auto-resume through CubeAPI for policy and quota enforcement

[han2ni3bal-pixel]

Background

We are designing an external Policy Gateway to enforce per-tenant limits, initially:

• Maximum total sandboxes
• Maximum running sandboxes

Create and user-initiated resume requests can be admitted through CubeAPI. However, the current auto-resume path is:

CubeProxy -> cube-lifecycle-manager -> CubeMaster

This bypasses CubeAPI and any policy or quota checks implemented there. As a result, a paused sandbox may auto-resume even when the tenant has already reached its running-sandbox limit. Reconciliation can correct the counter afterward, but cannot prevent the quota violation.

Proposal

Route auto-resume through an internal CubeAPI endpoint:

CubeProxy -> cube-lifecycle-manager -> CubeAPI -> Policy Gateway -> CubeMaster

The lifecycle manager would continue to own resume locking and request coalescing. CubeAPI would perform policy admission before calling CubeMaster, then commit or release the quota reservation based on the result.

This keeps CubeMaster unaware of tenants and quotas.

Questions

1. Does the community agree that auto-resume should pass through CubeAPI?
2. Would making the lifecycle manager depend on CubeAPI introduce any concerns?
3. Would a policy admission hook inside the lifecycle manager be preferable?

[kinwin-ustc]

Logically speaking, I believe that this quota control should be strongly related to the lifecycle transition of the sandbox. When each sandbox changes its state, there should be a callback to verify and modify the quota information

[han2ni3bal-pixel]

@kinwin-ustc Thanks. I agree that the actual lifecycle transition should still be performed by CubeMaster.

However, our intended boundary is slightly different: we only want CubeAPI to integrate with an external Policy/Quota Gateway. We do not want CubeMaster to understand tenants or maintain quota state.

For normal API operations, CubeAPI can enforce quota admission before requesting a lifecycle transition:

• Create: quota reserve -> CubeMaster create -> commit/release
• Resume/Connect: quota reserve -> CubeMaster resume -> commit/release
• Pause/Delete: notify the quota service after success

The problem is specifically auto-resume. Its current path is:

CubeProxy -> cube-lifecycle-manager -> CubeMaster

Even if CubeAPI observes the resulting lifecycle event afterward, it has no opportunity to reject the resume before the sandbox becomes running. At that point, a hard running-sandbox quota may already have been exceeded.

Therefore, we would like either:

1. Auto-resume to call an internal CubeAPI resume endpoint, so CubeAPI can run the same external quota admission flow before calling CubeMaster; or
2. An equivalent pre-transition admission hook that can call the external Policy/Quota Gateway.

CubeMaster would remain responsible for the lifecycle transition itself and would remain unaware of tenant and quota semantics.

Our main requirement is that an external quota decision can reject a resume before the running-state transition, rather than only reconciling usage afterward.

[chenhengqi]

I guess this can be implemented via a webhook. cc @aptacc2421

[kinwin-ustc]

@han2ni3bal-pixel CubeAPI is an external E2B API compatibility layer. It's unreasonable for internal communication to go through this layer. I think a more reasonable approach would be to use webhooks on CubeMaster to perform pre-emptive checks or notifications for changes in the state of each sandbox.

[aptacc2421]

Hi, @han2ni3bal-pixel @kinwin-ustc @chenhengqi

I think @kinwin-ustc is right that internal auto-resume should not route through CubeAPI. I also agree with @han2ni3bal-pixel that the actual lifecycle transition should stay in CubeMaster, and CubeMaster should not become tenant/quota-aware.

The missing capability is a pre-transition admission hook. For auto-resume, the natural owner is CLM because CLM already drives the resume call:

CubeProxy -> CLM -> external quota gateway -> allow/deny -> CubeMaster.Resume

This is different from a post-state notification webhook. It must be synchronous: CLM asks the external gateway before calling CubeMaster, and on deny it skips the resume and returns 503/Retry-After. The external gateway owns tenant mapping and running-count decisions, so CubeMaster stays tenant-agnostic.

CubeMaster can still emit lifecycle events for post-change notifications/reconciliation, but that async event webhook cannot enforce a hard running limit before the transition.

Would this split — a synchronous admission hook in CLM for auto-resume, plus async lifecycle-event webhooks for post-change notifications — fit both of your boundaries?