feat: add configurable provider RPM limiting - #259
Conversation
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
希望大佬见谅,我是vide的,但是我有本地测试过,好像是能有效果的。(比较悲哀的是,就算我卡rpm在4,也会被老黄429,唉) |
There was a problem hiding this comment.
Pull request overview
This PR adds configurable RPM (requests per minute) rate limiting for logical chat requests, with a global default and per-provider overrides, and applies the limiter at the service entry point so all transports (HTTP/SSE/WebSocket) share the same limit within a VS Code window.
Changes:
- Introduces a token-bucket
RateLimiterwith FIFO token acquisition and acreateRateLimiterhelper. - Wires rate-limit token acquisition + status logging into
UnifyChatServiceand the various provider clients viaApiProvideroptional methods. - Adds UI/config plumbing (schema, forms, descriptions) and localization strings for global and per-provider
rateLimit.rpm.
Reviewed changes
Copilot reviewed 25 out of 26 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/ui/screens/timeout-form-screen.ts | Adds rate-limit RPM field editing to the timeout/network settings UI flow. |
| src/ui/screens/provider-form-screen.ts | Passes rateLimit into the timeout form route for provider editing. |
| src/ui/screens/provider-draft-form-screen.ts | Passes rateLimit into the timeout form route for draft editing. |
| src/ui/router/types.ts | Extends TimeoutFormRoute to include rateLimit. |
| src/ui/provider-fields.ts | Includes rate-limit presence/value in provider “timeout” field description string. |
| src/types.ts | Adds rateLimit?: RateLimitConfig to ProviderConfig with documented semantics. |
| src/service.ts | Acquires a rate-limit token before transport selection and logs the limiter snapshot. |
| src/rate-limit.ts | New token-bucket rate limiter implementation + config type + factory. |
| src/logger.ts | Adds formatting and log suffix for rate-limit token snapshots. |
| src/config-store.ts | Reads global networkSettings.rateLimit and applies as a provider default. |
| src/config-ops.ts | Persists rateLimit as part of provider config keys. |
| src/client/openai/responses-client.ts | Creates and exposes rate limiter for OpenAI Responses provider. |
| src/client/openai/chat-completion-client.ts | Creates and exposes rate limiter for OpenAI Chat Completions provider. |
| src/client/ollama/client.ts | Creates and exposes rate limiter for Ollama provider. |
| src/client/interface.ts | Adds optional getRateLimitStatus / acquireRateLimitToken hooks to ApiProvider. |
| src/client/google/ai-studio-client.ts | Creates and exposes rate limiter for Google AI Studio provider. |
| src/client/github-copilot/client.ts | Creates and exposes rate limiter for GitHub Copilot provider. |
| src/client/anthropic/client.ts | Creates and exposes rate limiter for Anthropic provider. |
| package.nls.zh-cn.json | Adds Chinese configuration descriptions for rate-limit settings. |
| package.nls.json | Adds English configuration descriptions for rate-limit settings. |
| package.json | Adds configuration schema entries for global and per-provider rateLimit.rpm; bumps version. |
| package-lock.json | Updates lockfile version metadata to match package version bump. |
| l10n/bundle.l10n.zh-cn.json | Adds UI strings for RPM rate limit editing (Chinese). |
| l10n/bundle.l10n.json | Adds UI strings for RPM rate limit editing (English). |
| CHANGELOG.md | Adds v7.12.4 changelog entry covering the rate limiter changes. |
| .vscode/指北.md | Adds internal guidance doc describing RPM limiter semantics and maintenance boundaries. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Apply global networkSettings.rateLimit as default when the endpoint | ||
| // does not define its own rate-limit configuration. | ||
| if (provider.rateLimit === undefined) { | ||
| const globalRateLimit = this.readNetworkRateLimit(); | ||
| if (globalRateLimit) { | ||
| provider.rateLimit = globalRateLimit; | ||
| } | ||
| } |
| const client = this.getClient(resolvedProvider); | ||
| await client.acquireRateLimitToken?.(); | ||
| const rateLimitStatus = client.getRateLimitStatus?.(); | ||
|
|
| /** | ||
| * Peek at the current token count without consuming any. | ||
| * | ||
| * Refills first so the returned value reflects tokens earned since | ||
| * the last access. Used for logging / status display. | ||
| */ | ||
| getAvailableTokens(): { available: number; capacity: number } { | ||
| this.refill(); | ||
| return { available: this.tokens, capacity: this.maxTokens }; | ||
| } |
- config-store: normalize provider rateLimit so invalid/empty values (e.g. {}
or non-integer rpm) fall back to the global default, while still honoring an
explicit rpm: 0 override
- rate-limit: accept an optional AbortSignal in acquire(); abort a cancelled
waiter without consuming a token
- service: wire the chat CancellationToken into rate-limit token acquisition so
cancel-while-waiting aborts early
- rate-limit: getAvailableTokens no longer refills, so the logged post-acquire
snapshot is exact instead of slightly inflated
Addressed Copilot review suggestionsCommit 1.
2.
3.
Also updated |
# Conflicts: # package-lock.json # src/client/interface.ts # src/config-ops.ts # src/config-store.ts # src/types.ts
# Conflicts: # CHANGELOG.md # package-lock.json # package.json # src/client/anthropic/client.ts # src/config-store.ts
Summary
rpm: 0to disable a provider's limiterValidation
npm run compilenpm run l10n:checkRateLimiter(60)harness: 52 requests completed FIFO; capacity was 48; the final four completed at approximately 1-second intervals; no snapshot became negativeFixes #242