-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Labels
julesFor Jules AI to work onFor Jules AI to work on
Description
Summary
Reuse HTTP connections for efficiency when collecting from same domain.
Use Case
Collecting 1000 files from same domain - should reuse connections instead of TCP handshake each time.
Implementation
- HTTP/1.1 keep-alive
- HTTP/2 multiplexing
- Connection pool per domain
Configuration
connections:
max_per_host: 6 # Max connections per domain
max_idle: 100 # Max idle connections total
idle_timeout: 90s # Close idle connections after
http2:
enabled: true
max_streams: 100 # Max concurrent streams per connCommands
# Adjust pool size
borg collect website https://example.com --max-connections 10
# Disable keep-alive (for debugging)
borg collect website https://example.com --no-keepalive
# Force HTTP/1.1
borg collect website https://example.com --http1Benefits
- Faster collection (no repeated handshakes)
- Lower server load
- Better rate limit compliance
Acceptance Criteria
- Connection pooling per domain
- HTTP/2 support with multiplexing
- Configurable pool sizes
- Idle connection cleanup
- Connection reuse metrics
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
julesFor Jules AI to work onFor Jules AI to work on