Skip to content

feat: align function id with tool call response #3111

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

drbh
Copy link
Collaborator

@drbh drbh commented Mar 13, 2025

This PR improve mapping of tool ids to responses and enables tool_call_ids to be included in templates

Changes:

  1. maps input tool.id to tool_calls[0].id for chosen tool if tool.id is sepcified
  2. includes tool_call_id in chat template in the case an id is required

1. map tool id to tool response

import requests
import json

messages = [
    {
        "role": "system",
        "content": "You're a human assistant helping a user with a task. Use the tools to complete the task.",
    },
    {"role": "user", "content": "What's the weather like in Boston today?"},
]

payload = dict(
    messages=messages,
    seed=42,
    max_tokens=40,
    tools=[
        {
            "type": "function",
            "function": {
                "id": "TOOL_ID_01",
                "name": "get_current_weather",
                "description": "Get the current weather",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {
                            "type": "string",
                            "description": "The city and state, e.g. San Francisco, CA",
                        },
                        "format": {
                            "type": "string",
                            "enum": ["celsius", "fahrenheit"],
                            "description": "The temperature unit to use. Infer this from the users location.",
                        },
                    },
                    "required": ["location", "format"],
                },
            },
        }
    ],
)

response = requests.post(
    "http://127.0.0.1:3000/v1/chat/completions",
    json=payload,
    headers={"Content-Type": "application/json"},
)

print(json.dumps(response.json(), indent=2))

returns a response with the specified tool id

{
  "object": "chat.completion",
  "id": "",
  "created": 1741893608,
  "model": "mistralai/Mistral-7B-Instruct-v0.3",
  "system_fingerprint": "3.2.1-dev0-native",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "tool_calls": [
          {
            "id": "TOOL_ID_01",
            "type": "function",
            "function": {
              "description": null,
              "name": "get_current_weather",
              "arguments": "{\"location\":\"Boston, MA\",\"format\":\"celsius\"}"
            }
          }
        ]
      },
      "logprobs": null,
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 261,
    "completion_tokens": 34,
    "total_tokens": 295
  }
}

2. include tool_call_id in chat template

import requests
import json

messages = [
    {
        "role": "system",
        "content": "You're a human assistant helping a user with a task. Use the tools to complete the task.",
    },
    {"role": "user", "content": "What's the weather like in Boston today?"},
    {
        "role": "assistant",
        "tool_calls": [
            {
                "id": "TOOL_ID_1",
                "type": "function",
                "function": {
                    "description": None,
                    "name": "get_current_weather",
                    "arguments": '{"location":"Boston, MA","format":"celsius"}',
                },
            }
        ],
    },
    {
        "role": "tool",
        "tool_call_id": "TOOL_ID_1",
        "content": "-1C",
    },
]

payload = dict(
    messages=messages,
    seed=42,
    max_tokens=40,
    tools=[
        {
            "type": "function",
            "function": {
                "id": "TOOL_ID_1",
                "name": "get_current_weather",
                "description": "Get the current weather",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {
                            "type": "string",
                            "description": "The city and state, e.g. San Francisco, CA",
                        },
                        "format": {
                            "type": "string",
                            "enum": ["celsius", "fahrenheit"],
                            "description": "The temperature unit to use. Infer this from the users location.",
                        },
                    },
                    "required": ["location", "format"],
                },
            },
        }
    ],
    # force a non tool response in the case the last message was a tool response
    tool_choice="none" if messages[-1]["role"] == "tool" else "auto",
)

response = requests.post(
    "http://127.0.0.1:3000/v1/chat/completions",
    json=payload,
    headers={"Content-Type": "application/json"},
)

print(json.dumps(response.json(), indent=2))

returns a natural language response that internal included the tool call id in the prompt

{
  "object": "chat.completion",
  "id": "",
  "created": 1741893739,
  "model": "mistralai/Mistral-7B-Instruct-v0.3",
  "system_fingerprint": "3.2.1-dev0-native",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "The weather in Boston today is -1 degrees Celsius."
      },
      "logprobs": null,
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 100,
    "completion_tokens": 14,
    "total_tokens": 114
  }
}

without these changes the response from mistralai/Mistral-7B-Instruct-v0.3 (which requires an ID of a specific length)

{
  "error": "Template error: syntax error: Tool call IDs should be alphanumeric strings with length 9! (in <string>:81)",
  "error_type": "template_error"
}

@drbh drbh force-pushed the improve-tool-call-and-response-ids branch from 2b1f2a3 to e721574 Compare March 21, 2025 15:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant