Skip to content

fix(gemini): avoid duplicate structured schema prompt#2277

Merged
nicoloboschi merged 2 commits into
vectorize-io:mainfrom
r266-tech:fix-gemini-structured-schema-prompt
Jun 23, 2026
Merged

fix(gemini): avoid duplicate structured schema prompt#2277
nicoloboschi merged 2 commits into
vectorize-io:mainfrom
r266-tech:fix-gemini-structured-schema-prompt

Conversation

@r266-tech

Copy link
Copy Markdown
Contributor

Summary

  • stops adding the structured-output JSON schema to Gemini system_instruction when native response_schema is already sent
  • keeps response_schema on normal and cached structured calls
  • adds a prompt-schema compatibility retry only for uncached structured calls that return invalid JSON, so cached retries keep their cached context

Fixes #1070.

Tests

  • uv run --project hindsight-api-slim pytest hindsight-api-slim/tests/test_llm_extra_body.py -q
  • uv run --project hindsight-api-slim ruff format --check hindsight-api-slim/hindsight_api/engine/providers/gemini_llm.py hindsight-api-slim/tests/test_llm_extra_body.py
  • uv run --project hindsight-api-slim ruff check hindsight-api-slim/hindsight_api/engine/providers/gemini_llm.py
  • git diff --check

Codex adversarial review approved after reworking the cached retry path so it does not drop cached context.

@koriyoshi2041

Copy link
Copy Markdown
Contributor

I did a quick local check because the only red job is verify-generated-files. The failure appears unrelated to this Gemini change: that run dies while parsing hindsight-docs/src/data/integrations.json with the trailing-comma error that was fixed on main by 55f70e1d.

Targeted checks on this branch passed for me:

uv run --project hindsight-api-slim pytest hindsight-api-slim/tests/test_llm_extra_body.py -q
uv run --project hindsight-api-slim ruff format --check hindsight-api-slim/hindsight_api/engine/providers/gemini_llm.py hindsight-api-slim/tests/test_llm_extra_body.py
uv run --project hindsight-api-slim ruff check hindsight-api-slim/hindsight_api/engine/providers/gemini_llm.py
git diff --check

Result: 19 passed, ruff clean, no diff whitespace issues. A rebase/rerun should be enough to separate this PR from the stale docs failure.

@r266-tech

Copy link
Copy Markdown
Contributor Author

@koriyoshi2041 confirmed — thanks for the targeted verification. The red verify-generated-files is the stale hindsight-docs/src/data/integrations.json trailing-comma parse error fixed on main by 55f70e1, not this change (which only touches gemini_llm.py + test_llm_extra_body.py). I've updated the branch onto current main so the check re-runs against the fixed base.

@nicoloboschi nicoloboschi left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. Verified the structured-output behavior with a Gemini LOCOMO run on conv-26 (this branch, isolated DB): 89.5% accuracy (136/152) and zero invalid-JSON / prompt-fallback events across ~70 ingestion fact-extractions + 152 recall/QA cycles. Native response_schema-only path is robust; removing the duplicate prompt-side schema does not degrade Gemini output.

@nicoloboschi nicoloboschi merged commit 199ae14 into vectorize-io:main Jun 23, 2026
84 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GeminiLLM provider sends structured output schema twice (prompt + native response_schema)

3 participants