Skip to content

fix(providers): clarify structured output completion - #524

Merged
Jason Robert (jrob5756) merged 2 commits into
microsoft:mainfrom
hertznsk:fix/pydantic-ai-final-result-guidance
Sep 11, 2026
Merged

fix(providers): clarify structured output completion#524
Jason Robert (jrob5756) merged 2 commits into
microsoft:mainfrom
hertznsk:fix/pydantic-ai-final-result-guidance

Conversation

@hertznsk

Copy link
Copy Markdown
Contributor

Summary

  • give Pydantic AI's final_result output tool an explicit completion contract
  • keep ToolOutput, schema sanitization, output retries, and provider parity unchanged
  • cover both the shared builder metadata and the serialized OpenAI Chat Completions request

Pydantic AI already exposes the intended customization seam through ToolOutput(..., description=...), but its default description only says that the tool is the final response. Some models behind OpenAI- or Anthropic-compatible endpoints still finish with plain text unless the requirement to call the tool is explicit. This change puts that requirement on the tool definition rather than modifying workflow-authored system prompts.

This is an alternative to the backend-wide PromptedOutput switch proposed in #508. It preserves forced tool output, nested-schema sanitization, parse recovery, telemetry, and continuation message formats.

Validation

  • make check
  • uv run pytest tests/test_providers/test_pydantic_ai_agent_builder.py tests/test_providers/test_pydantic_ai_structured_output.py tests/test_providers/test_openai_http_stub.py -q (84 passed)
  • full non-performance suite: 8919 passed, 49 skipped; two unrelated POSIX permission tests failed because this environment did not produce their expected chmod(0o000) behavior
  • direct builder smoke test confirmed the same final_result description for both backend="openai" and backend="anthropic"

Notes

The description improves adherence for models that support tool calling. It cannot compensate for a gateway that strips tool descriptions or does not implement tool calls.

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@f7f444c). Learn more about missing BASE report.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #524   +/-   ##
=======================================
  Coverage        ?   92.00%           
=======================================
  Files           ?      165           
  Lines           ?    27336           
  Branches        ?        0           
=======================================
  Hits            ?    25150           
  Misses          ?     2186           
  Partials        ?        0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jrob5756 Jason Robert (jrob5756) left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for contributing!

@jrob5756
Jason Robert (jrob5756) merged commit 87c0c14 into microsoft:main Sep 11, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants