Skip to content

Commit 033e77c

Browse files
feat(gateway): default-on rate limits and tenant concurrency cap (#45)
* feat(ratelimit): enforce gateway admission Reject exhausted keys before delegation verification. Hold tenant slots until responses and SSE streams finish. Keep admission denials inside the authenticated receipt path. Signed-off-by: Shreyansh Sancheti <43677304+shreyanshjain7174@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(ratelimit): isolate tenant admission Authorize principal and service scopes before tenant charges to prevent cross-tenant quota exhaustion. Bound tenant identities and scan past pinned cache entries. Refund concurrency rejections without rewinding bucket clocks. Release acquired slots during panic unwinding. Signed-off-by: Shreyansh Sancheti <43677304+shreyanshjain7174@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Signed-off-by: Shreyansh Sancheti <43677304+shreyanshjain7174@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 6138192 commit 033e77c

17 files changed

Lines changed: 1778 additions & 53 deletions

‎README.md‎

Lines changed: 50 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -315,6 +315,11 @@ AgentGate is deterministic. Agents never see tokens, calls go only to configured
315315
| `AGENTGATE_ADMIN_SECRET` | Secret for admin API access | Yes |
316316
| `AGENTGATE_PUBLIC_URL` | Base URL used to build the OAuth callback (default `http://localhost:8080`) | No |
317317
| `AGENTGATE_API_KEY_CACHE_TTL` | Verified-key cache duration. Default `30s`; `0` disables caching. | No |
318+
| `AGENTGATE_RATE_KEY_RPS` | Requests per second per API key ID. Default `20`. Zero requires tenant admission disabled. | No |
319+
| `AGENTGATE_RATE_KEY_BURST` | Per-key token bucket burst. Default `40`. Zero requires tenant admission disabled. | No |
320+
| `AGENTGATE_RATE_TENANT_RPS` | Requests per second per tenant. Default `100`; `0` disables this bucket. | No |
321+
| `AGENTGATE_RATE_TENANT_BURST` | Per-tenant token bucket burst. Default `200`; `0` disables this bucket. | No |
322+
| `AGENTGATE_TENANT_MAX_INFLIGHT` | In-flight upstream requests per tenant. Default `32`; `0` disables this cap. | No |
318323
| `AGENTGATE_BISCUIT_MAX_DURATION` | Go duration per Biscuit Datalog world run. Default `250ms`; range `1ms` through `5s`, inclusive. Invalid values stop startup. | No |
319324
| `GITHUB_CLIENT_ID` / `GITHUB_CLIENT_SECRET` | GitHub OAuth credentials | For GitHub OAuth |
320325
| `SLACK_CLIENT_ID` / `SLACK_CLIENT_SECRET` | Slack OAuth credentials | For Slack OAuth |
@@ -332,6 +337,50 @@ SQLite stores a public ID and a peppered HMAC-SHA256, never the secret. Use `POS
332337
Legacy `ag_live_` rows keep their bcrypt hashes. Successful validation adds an indexed HMAC without changing the presented key.
333338
Revoked rows never enter legacy bcrypt scans.
334339

340+
#### Request admission
341+
342+
API keys have no organization ID. The organization store binds admins, not agent keys.
343+
For explicitly scoped keys, tenant identity is the authorized `on_behalf_of` principal.
344+
Such keys share that principal's rate bucket and concurrency cap.
345+
Each key's bucket spans services and human principals.
346+
Wildcard keys allowing `"*"` use 1 key-scoped tenant for every principal.
347+
Different wildcard keys don't share human quotas.
348+
Each key selects at most 64 tenant identities from its fixed scope list.
349+
When the list exceeds 64 entries, only the first 63 retain shared human identity.
350+
The remaining principals share 1 key-scoped fallback tenant.
351+
Human and fallback identities use separate namespaces. No per-key tenant-tracking map is needed.
352+
353+
Rate settings accept finite, non-negative numbers. Bursts and concurrency caps require integers.
354+
Invalid environment values stop startup. Either zero rate or zero burst disables that bucket.
355+
Both key settings must stay positive while tenant rate limiting or tenant concurrency is enabled.
356+
Per-service limits are optional. Set `rate_limit` under a service in the loaded registry YAML.
357+
They apply per API key and service, alongside the global limits.
358+
See [registry rate limits](docs/registry.md#rate-limits) for YAML examples.
359+
360+
The key bucket runs after API key authentication and body validation.
361+
In-memory service and principal scope checks run next.
362+
Scope denials consume only key tokens. Their receipts retain `scope_denied`.
363+
Tenant and service buckets run after scope checks, before delegation verification or vault access.
364+
Rate denials return HTTP 429 with `rate_limited`.
365+
`Retry-After` rounds the reservation delay up to whole seconds, with a minimum of 1.
366+
Concurrency admission runs immediately before upstream dispatch.
367+
Full tenants return HTTP 429 with `concurrency_limited` and `Retry-After: 1`.
368+
Concurrency rejection refunds the key, tenant, and service charges without losing elapsed refill.
369+
Slots remain held through response completion, including SSE streams and receipt commits.
370+
Failures, disconnects, stream termination, and dispatch panics release them.
371+
Every authenticated, schema-valid attempt still commits a receipt before its response, including admission denials.
372+
Both admission errors record policy `rate_limited`. Receipt failures return `receipt_failed` instead.
373+
374+
Admission state is process-local. Replicas don't share quotas.
375+
The combined bucket and semaphore cache holds at most 100000 entries.
376+
Requests trigger idle sweeps at most once per minute. Entries idle for 10 minutes become eviction candidates.
377+
Only full buckets without pending permits or in-flight requests can be evicted.
378+
Slow buckets retain their quota until fully refilled.
379+
At capacity, admission scans up to 64 entries from the LRU tail.
380+
Pinned or depleted entries move to the front. The first evictable entry is replaced.
381+
If none qualifies, admission fails closed with HTTP 429 and `Retry-After: 1`.
382+
Restart the gateway after changing admission settings or service YAML.
383+
335384
#### Biscuit token security caps
336385

337386
Tokens are limited to 4096 bytes and 8 attenuation blocks.
@@ -344,7 +393,7 @@ Authority checks follow the same caps.
344393

345394
These checks run before authorizer construction. Rejections match both `delegation.ErrDenied` and `delegation.ErrTokenShape`.
346395
Errors contain fixed reasons, not token contents.
347-
The structural caps bound fact joins and block processing before the request reaches the rate limiter.
396+
The early rate limiter runs before these checks. Structural caps then bound fact joins and block processing.
348397
Biscuit-go v2.2.0 deadlines do not cover check evaluation and cannot reliably stop timed-out goroutines.
349398
Runtime limits alone do not bound per-request CPU.
350399

‎cmd/agentgw/main.go‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,11 @@ func main() {
6262
logger.Error("load Biscuit limits", "error", err)
6363
os.Exit(1)
6464
}
65+
rateLimits, err := loadRateLimits()
66+
if err != nil {
67+
logger.Error("load rate limits", "error", err)
68+
os.Exit(1)
69+
}
6570

6671
// Load service registry.
6772
reg := registry.New()
@@ -141,7 +146,7 @@ func main() {
141146

142147
ledger := receipt.NewLedger(database, signerStore)
143148
defer ledger.Close()
144-
limiter := ratelimit.New(nil) // no per-service limits configured yet; Allow() is a pass-through
149+
limiter := ratelimit.New(reg.RateLimits(), ratelimit.WithSettings(rateLimits))
145150

146151
delegationRoot, err := delegation.NewRootStore(database, masterKey)
147152
if err != nil {

‎cmd/agentgw/rate_limits.go‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
package main
2+
3+
import (
4+
"fmt"
5+
"math"
6+
"os"
7+
"strconv"
8+
9+
"github.com/Clawdlinux/agentgate/internal/ratelimit"
10+
)
11+
12+
func loadRateLimits() (ratelimit.Settings, error) {
13+
settings := ratelimit.DefaultSettings()
14+
for _, field := range []struct {
15+
name string
16+
target *float64
17+
}{
18+
{"AGENTGATE_RATE_KEY_RPS", &settings.KeyRPS},
19+
{"AGENTGATE_RATE_TENANT_RPS", &settings.TenantRPS},
20+
} {
21+
if value, ok := os.LookupEnv(field.name); ok {
22+
parsed, err := strconv.ParseFloat(value, 64)
23+
if err != nil || parsed < 0 || math.IsNaN(parsed) || math.IsInf(parsed, 0) {
24+
return ratelimit.Settings{}, fmt.Errorf("%s must be a finite non-negative number", field.name)
25+
}
26+
*field.target = parsed
27+
}
28+
}
29+
for _, field := range []struct {
30+
name string
31+
target *int
32+
}{
33+
{"AGENTGATE_RATE_KEY_BURST", &settings.KeyBurst},
34+
{"AGENTGATE_RATE_TENANT_BURST", &settings.TenantBurst},
35+
{"AGENTGATE_TENANT_MAX_INFLIGHT", &settings.TenantMaxInflight},
36+
} {
37+
if value, ok := os.LookupEnv(field.name); ok {
38+
parsed, err := strconv.Atoi(value)
39+
if err != nil || parsed < 0 {
40+
return ratelimit.Settings{}, fmt.Errorf("%s must be a non-negative integer", field.name)
41+
}
42+
*field.target = parsed
43+
}
44+
}
45+
tenantEnabled := (settings.TenantRPS > 0 && settings.TenantBurst > 0) || settings.TenantMaxInflight > 0
46+
if tenantEnabled && (settings.KeyRPS == 0 || settings.KeyBurst == 0) {
47+
return ratelimit.Settings{}, fmt.Errorf(
48+
"AGENTGATE_RATE_KEY_RPS and AGENTGATE_RATE_KEY_BURST must be positive while tenant admission is enabled",
49+
)
50+
}
51+
return settings, nil
52+
}
Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
package main
2+
3+
import (
4+
"strings"
5+
"testing"
6+
)
7+
8+
func TestTenantAdmissionRequiresKeyLimit(t *testing.T) {
9+
for _, name := range []string{"AGENTGATE_RATE_KEY_RPS", "AGENTGATE_RATE_KEY_BURST"} {
10+
for _, enabled := range []string{"rate", "inflight", "neither"} {
11+
t.Run(name+"/"+enabled, func(t *testing.T) {
12+
clearRateEnv(t)
13+
t.Setenv(name, "0")
14+
t.Setenv("AGENTGATE_RATE_TENANT_RPS", "0")
15+
t.Setenv("AGENTGATE_TENANT_MAX_INFLIGHT", "0")
16+
switch enabled {
17+
case "rate":
18+
t.Setenv("AGENTGATE_RATE_TENANT_RPS", "1")
19+
case "inflight":
20+
t.Setenv("AGENTGATE_TENANT_MAX_INFLIGHT", "1")
21+
}
22+
_, err := loadRateLimits()
23+
if enabled == "neither" {
24+
if err != nil {
25+
t.Fatal(err)
26+
}
27+
} else if err == nil || !strings.Contains(err.Error(), name) {
28+
t.Fatalf("expected key limit validation error, got %v", err)
29+
}
30+
})
31+
}
32+
}
33+
}

‎cmd/agentgw/rate_limits_test.go‎

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
package main
2+
3+
import (
4+
"os"
5+
"strings"
6+
"testing"
7+
8+
"github.com/Clawdlinux/agentgate/internal/ratelimit"
9+
)
10+
11+
var rateEnvNames = []string{
12+
"AGENTGATE_RATE_KEY_RPS", "AGENTGATE_RATE_KEY_BURST",
13+
"AGENTGATE_RATE_TENANT_RPS", "AGENTGATE_RATE_TENANT_BURST",
14+
"AGENTGATE_TENANT_MAX_INFLIGHT",
15+
}
16+
17+
func clearRateEnv(t *testing.T) {
18+
t.Helper()
19+
for _, name := range rateEnvNames {
20+
t.Setenv(name, "")
21+
if err := os.Unsetenv(name); err != nil {
22+
t.Fatal(err)
23+
}
24+
}
25+
}
26+
27+
func TestLoadRateLimits(t *testing.T) {
28+
clearRateEnv(t)
29+
settings, err := loadRateLimits()
30+
if err != nil || settings != ratelimit.DefaultSettings() {
31+
t.Fatalf("defaults = %+v, err = %v", settings, err)
32+
}
33+
for _, name := range rateEnvNames {
34+
t.Setenv(name, "0")
35+
}
36+
settings, err = loadRateLimits()
37+
if err != nil || settings != (ratelimit.Settings{}) {
38+
t.Fatalf("disabled = %+v, err = %v", settings, err)
39+
}
40+
t.Setenv("AGENTGATE_RATE_KEY_RPS", "0.5")
41+
t.Setenv("AGENTGATE_RATE_KEY_BURST", "2")
42+
t.Setenv("AGENTGATE_RATE_TENANT_RPS", "12.5")
43+
t.Setenv("AGENTGATE_RATE_TENANT_BURST", "25")
44+
t.Setenv("AGENTGATE_TENANT_MAX_INFLIGHT", "7")
45+
settings, err = loadRateLimits()
46+
want := ratelimit.Settings{KeyRPS: 0.5, KeyBurst: 2, TenantRPS: 12.5, TenantBurst: 25, TenantMaxInflight: 7}
47+
if err != nil || settings != want {
48+
t.Fatalf("overrides = %+v, err = %v", settings, err)
49+
}
50+
}
51+
52+
func TestLoadRateLimitsInvalid(t *testing.T) {
53+
for _, name := range rateEnvNames {
54+
for _, value := range []string{"", "-1", "NaN", "Inf", "1e999", "invalid"} {
55+
t.Run(name+"/"+value, func(t *testing.T) {
56+
clearRateEnv(t)
57+
t.Setenv(name, value)
58+
_, err := loadRateLimits()
59+
if err == nil || !strings.Contains(err.Error(), name) {
60+
t.Fatalf("expected named validation error, got %v", err)
61+
}
62+
})
63+
}
64+
if strings.Contains(name, "RPS") {
65+
continue
66+
}
67+
t.Run(name+"/fraction", func(t *testing.T) {
68+
clearRateEnv(t)
69+
t.Setenv(name, "1.5")
70+
if _, err := loadRateLimits(); err == nil {
71+
t.Fatal("fractional integer accepted")
72+
}
73+
})
74+
}
75+
}

‎docs/registry.md‎

Lines changed: 34 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,41 @@
1-
# Registry response settings
1+
# Registry settings
22

33
Response limits belong under each service's `upstream` settings.
44
Streaming permission belongs on each action.
55

6+
## Rate limits
7+
8+
Optional `rate_limit` settings apply per API key and service.
9+
They combine with the gateway's per-key and per-tenant buckets.
10+
Explicitly scoped keys share human principal tenants. Wildcard keys use key-scoped tenants.
11+
Large scope lists use a key fallback beyond the first 63 principals.
12+
See [request admission](../README.md#request-admission) for identity bounds and startup validation.
13+
The gateway loads them from the same registry YAML as actions.
14+
The separate `configs/agentgate.yaml` example isn't loaded by the gateway.
15+
Restart the gateway after changing these settings.
16+
17+
```yaml
18+
services:
19+
example:
20+
base_url: https://api.example.com
21+
auth: {type: bearer}
22+
rate_limit:
23+
requests_per_second: 25
24+
burst: 50
25+
actions:
26+
list: {method: GET, path: /items}
27+
```
28+
29+
Use at most 1 positive rate field: `requests_per_second`, `requests_per_minute`, or `requests_per_hour`.
30+
Rates must be finite and non-negative. Fractional rates are accepted.
31+
Absent or zero rates disable the service bucket. Omitted or zero `burst` selects 10 tokens.
32+
Negative bursts and conflicting rates fail registry loading.
33+
Rate rejection returns HTTP 429 with `rate_limited` and a whole-second `Retry-After` header.
34+
The gateway commits its receipt before sending that response.
35+
Scope checks run before service and tenant buckets. Concurrency denials refund all token charges.
36+
37+
## Response settings
38+
639
```yaml
740
services:
841
example:

‎internal/gateway/admission.go‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
package gateway
2+
3+
import "github.com/Clawdlinux/agentgate/internal/auth"
4+
5+
// tenantIdentity bounds each key to 64 tenant identities without a separate tracking cache.
6+
// Wildcard keys share one key-scoped tenant across all requested principals.
7+
func tenantIdentity(key *auth.AgentKey, principal string) string {
8+
fallback := "key:" + key.ID
9+
for _, allowed := range key.AllowedUsers {
10+
if allowed == "*" {
11+
return fallback
12+
}
13+
}
14+
limit := 64
15+
if len(key.AllowedUsers) > limit {
16+
limit-- // Reserve one identity for the overflow bucket.
17+
}
18+
for _, allowed := range key.AllowedUsers[:min(limit, len(key.AllowedUsers))] {
19+
if allowed == principal {
20+
return "human:" + principal
21+
}
22+
}
23+
return fallback
24+
}

0 commit comments

Comments
 (0)