Skip to content

bug: VertexSynthesizer HTTP timeout (120s) too short for large curation prompts #77

Description

@mpeter

Summary

dewey curate fails with context deadline exceeded (Client.Timeout exceeded while awaiting headers) for vaults with rich content. The VertexSynthesizer HTTP client is configured with a 120-second timeout, but large curation extraction prompts (30K-40K input tokens) consistently take 120-260 seconds to complete on Vertex AI.

Reproduction

Configure a knowledge store with 9-10 documents containing rich markdown (dossiers, agreements — ~34K input tokens). Run dewey curate. Observe:

❌ <store>: LLM extraction failed: vertex AI request: Post "https://...": 
context deadline exceeded (Client.Timeout exceeded while awaiting headers)

Timing: the failure occurs at exactly 120 seconds, confirming the HTTP client timeout is the cause.

Root Cause

llm/vertex.go:

client: &http.Client{
    Timeout: 120 * time.Second, // Match OllamaSynthesizer timeout.
},

The Ollama timeout is appropriate for local inference. Vertex AI with Claude Sonnet processing 34K input tokens takes 120-260 seconds — regularly exceeding the cap.

Fix

Increase the Vertex HTTP client timeout to 300 seconds. Vertex API calls have different latency characteristics than local Ollama inference and warrant a longer timeout.

Timeout: 300 * time.Second,

Context

Discovered on Vertex AI with claude-sonnet-4-6@default in us-east5. A direct curl to the same endpoint with the same payload completes in 85-261 seconds depending on load. The 120s Dewey timeout consistently cuts off large extractions.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    Status
    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions