Add metadata support to deferred tool exceptions #3345
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
DouweM
requested changes
Nov 6, 2025
DouweM
reviewed
Nov 6, 2025
Enables CallDeferred and ApprovalRequired exceptions to carry arbitrary metadata via an optional `metadata` parameter. The metadata is accessible in DeferredToolRequests.metadata keyed by tool_call_id. This allows tools to: - Provide cost/time estimates for approval decisions - Include task IDs for external execution tracking - Store context about why approval is required - Attach priority or urgency information Backward compatible - metadata defaults to empty dict if not provided.
Extract metadata population logic into separate _populate_deferred_calls helper function to reduce cyclomatic complexity from 16 to 15.
Per Douwe's comments:
1. Store None instead of {} when no metadata provided
2. Don't add tool_call_id to metadata dict when None
3. Update Temporal wrap/unwrap methods to handle metadata
- Updated test assertions to reflect None metadata behavior
- Updated doc example snapshots to show metadata={}
- Fixed codespell issue with table formatting
cjohnhanson
force-pushed
the
feat/deferred-tool-metadata
branch
from
7e17594 to
42cf5b8
Compare
- Replace contrived task_id parameter with realistic tool signatures - Add ComputeDeps class demonstrating dependency injection pattern - Show using ctx.deps to compute metadata from tool arguments - Remove incorrect statement about backwards compatibility - Update test_examples.py to match new realistic example
- Add mock model responses for flight booking example - Update documentation snapshots to include metadata field - Follow existing pattern for deferred tool testing
- Complete test_approval_required_without_metadata() by running agent with approval results to hit the tool implementation line - Complete test_mixed_deferred_tools_with_metadata() by running agent with all deferred tool results to hit tool implementation lines - Remove unreachable branch check in _populate_deferred_calls() - keys are always present in both dicts by construction This achieves 100% coverage for the deferred tool metadata feature.
cjohnhanson
commented
Nov 10, 2025
|
|
||
| _(This example is complete, it can be run "as is" — you'll need to add `asyncio.run(main())` to run `main`)_ | ||
|
|
||
| ## Attaching Metadata to Deferred Tools |
Contributor
Author
There was a problem hiding this comment.
@DouweM I wasn't 100% sure on whether this would benefit from a dedicated example in the docs. I could easily shift this to just be included in the existing examples.
Collaborator
There was a problem hiding this comment.
As much as I appreciate the Meltano reference in the flight destinations, I think we can make the existing examples stronger by using metadata so we should do it there (with one line explanations of the metadata arg, possibly inside one of those # (1) tooltips) instead of in its own section:
- The
CallDeferredexample could have scheduling the task return atask_idthat can then be included in metadata, instead of making theTaskResultaware oftool_call_id. - the
ApprovalRequiredexample could havereason: 'protected'or something that can be displayed to the user.
- Remove 'Assertio' from codespell ignore-words-list (no actual usage found) - Remove unused ML training example test expectations (train_model, process_dataset) that were replaced with flight booking example in commit d4f1613
DouweM
requested changes
Nov 10, 2025
|
|
||
| _(This example is complete, it can be run "as is" — you'll need to add `asyncio.run(main())` to run `main`)_ | ||
|
|
||
| ## Attaching Metadata to Deferred Tools |
Collaborator
There was a problem hiding this comment.
As much as I appreciate the Meltano reference in the flight destinations, I think we can make the existing examples stronger by using metadata so we should do it there (with one line explanations of the metadata arg, possibly inside one of those # (1) tooltips) instead of in its own section:
- The
CallDeferredexample could have scheduling the task return atask_idthat can then be included in metadata, instead of making theTaskResultaware oftool_call_id. - the
ApprovalRequiredexample could havereason: 'protected'or something that can be displayed to the user.
| approvals: list[ToolCallPart] = field(default_factory=list) | ||
| """Tool calls that require human-in-the-loop approval.""" | ||
| metadata: dict[str, dict[str, Any]] = field(default_factory=dict) | ||
| """Metadata for deferred tool calls, keyed by tool_call_id. |
Collaborator
There was a problem hiding this comment.
Suggested change
| """Metadata for deferred tool calls, keyed by tool_call_id. | |
| """Metadata for deferred tool calls, keyed by `tool_call_id`. |
| assert len(result.output.approvals) == 1 | ||
| assert result.output.approvals[0].tool_name == 'requires_approval' | ||
| # When no metadata is provided, the tool_call_id should not be in metadata dict | ||
| tool_call_id = result.output.approvals[0].tool_call_id |
Collaborator
There was a problem hiding this comment.
I think we can revert this to the snapshot -- the metadata type doesn't allow tool_call_id: None anyway
| # When no metadata is provided, the tool_call_id should not be in metadata dict | ||
| tool_call_id = response.calls[0].tool_call_id | ||
| assert tool_call_id not in response.metadata | ||
| assert response.metadata == {} |
Collaborator
There was a problem hiding this comment.
Same as up -- I like snapshots :)
|
|
||
| result = agent.run_sync('Hello') | ||
| assert isinstance(result.output, DeferredToolRequests) | ||
| assert len(result.output.calls) == 1 |
Collaborator
There was a problem hiding this comment.
Unless there's a particular reason not to, lets use snapshots here!
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Enables
CallDeferredandApprovalRequiredexceptions to carry arbitrary metadata via an optionalmetadataparameter. The metadata is accessible inDeferredToolRequests.metadatakeyed bytool_call_id.Backward compatible - metadata defaults to empty dict if not provided.