feat: add Prompts API to Responses API

[r3v5]

What does this PR do?

The purpose of this PR is to integrate Prompts API to Responses API to achieve full OpenAI compatibility for current Responses API in Llama Stack.

Closes #3321

Test Plan

Manual API testing and running newly added unit tests.

Prerequisites:
uv run --with llama-stack llama stack build --distro starter --image-type venv --run

API Testing:

1. Create response with prompt:

curl --request POST \                                                                      
  --url http://localhost:8321/v1/openai/v1/responses \                  
  --header 'Accept: application/json, text/event-stream' \
  --header 'Content-Type: application/json' \
  --data '{
  "input": "What is the capital of Ireland?",
  "model": "openai/gpt-4o",
  "store": true,
  "prompt": {
      "id": "pmpt_dc6c124c7f1393cd4ddb88ed707ffbfd3d937e644d10052c",
      "version": "2",
      "variables": {
          "area_name": "geography",
          "company_name": "Dummy Company"
      }
  }
}'

Output:
{"created_at":1758562266,"error":null,"id":"resp-f20d3abe-d3cf-416b-9060-e653134d51cf","model":"openai/gpt-4o","object":"response","output":[{"content":[{"text":"The capital of Ireland is Dublin.","type":"output_text","annotations":[]}],"role":"assistant","type":"message","id":"msg_67a2f184-ae00-4f49-b4d2-27dfd22623a6","status":"completed"}],"parallel_tool_calls":false,"previous_response_id":null,"prompt":{"prompt":"You are a helpful {{ area_name }} assistant in {{ company_name }}. Always provide accurate information.","version":2,"prompt_id":"pmpt_dc6c124c7f1393cd4ddb88ed707ffbfd3d937e644d10052c","variables":["area_name","company_name"],"is_default":false},"status":"completed","temperature":null,"text":{"format":{"type":"text"}},"top_p":null,"truncation":null,"user":null}%

Telemetry log:

INFO     2025-09-22 18:45:29,366 console_span_processor:62 telemetry:  17:45:27.575 [INFO] Prompt                                                     
         pmpt_dc6c124c7f1393cd4ddb88ed707ffbfd3d937e644d10052c found and applied                                                                      
         Final prompt text: You are a helpful geography assistant in Dummy Company. Always provide accurate information. 

3. Get current response:

curl --request GET \                                                                                                                                              
  --url http://localhost:8321/v1/openai/v1/responses/resp-f20d3abe-d3cf-416b-9060-e653134d51cf \
  --header 'Accept: application/json'

Output:
{"created_at":1758562266,"error":null,"id":"resp-f20d3abe-d3cf-416b-9060-e653134d51cf","model":"openai/gpt-4o","object":"response","output":[{"content":[{"text":"The capital of Ireland is Dublin.","type":"output_text","annotations":[]}],"role":"assistant","type":"message","id":"msg_67a2f184-ae00-4f49-b4d2-27dfd22623a6","status":"completed"}],"parallel_tool_calls":false,"previous_response_id":null,"prompt":{"prompt":"You are a helpful {{ area_name }} assistant in {{ company_name }}. Always provide accurate information.","version":2,"prompt_id":"pmpt_dc6c124c7f1393cd4ddb88ed707ffbfd3d937e644d10052c","variables":["area_name","company_name"],"is_default":false},"status":"completed","temperature":null,"text":{"format":{"type":"text"}},"top_p":null,"truncation":null,"user":null}%

[mattf]

this is an api change unrelated to how prompts are used in /v1/responses

please review your code assistant output before posting as a PR.

[r3v5]

this is an api change unrelated to how prompts are used in /v1/responses

please review your code assistant output before posting as a PR.

Hi @mattf ! Could you please elaborate on how the prompts should be used in Responses API in your opinion. My understanding was that they should be propagated to Agent’s messages context as OpenAISystemMessageParam

[franciscojavierarceo]

Hey @r3v5 it looks like you've suggested adding prompt_id here where you need to add a Prompt object with an id, version, and variables, which would then be consistent with OpenAI's client usage, as outlined here:

response = client.responses.create(
  prompt={
    "id": "pmpt_68b0c29740048196bd3a6e6ac3c4d0e20ed9a13f0d15bf5e",
    "version": "2",
    "variables": {
        "city": "San Francisco",
        "age": 30,
    }
  }
)

So this is currently incorrect. As @mattf suggested, let's make sure we double check this. Thank you.

[r3v5]

Hey @r3v5 it looks like you've suggested adding prompt_id here where you need to add a Prompt object with an id, version, and variables, which would then be consistent with OpenAI's client usage, as outlined here:

response = client.responses.create(
  prompt={
    "id": "pmpt_68b0c29740048196bd3a6e6ac3c4d0e20ed9a13f0d15bf5e",
    "version": "2",
    "variables": {
        "city": "San Francisco",
        "age": 30,
    }
  }
)

So this is currently incorrect. As @mattf suggested, let's make sure we double check this. Thank you.

Oh yeah, this makes sense. I got it. I will adjust the implementation then

[r3v5]

Version has type string because OpenAI has it like string. Reference is here

[mattf]

@cdoern this is an enhancement to the /openai/v1/responses api, does it match the openai /v1/responses api spec?

[cdoern]

@mattf:

╰─ oasdiff breaking --fail-on ERR \
docs/_static/llama-stack-spec.yaml \
/Users/charliedoern/Downloads/openapi.documented.yml \
--strip-prefix-base "/v1/openai/v1" \
--match-path '(^/v1/openai/v1/responses.*|^/responses.*)'
51 changes: 24 error, 27 warning, 0 info
error	[api-removed-without-deprecation] at docs/_static/llama-stack-spec.yaml
  in API GET /responses
  	api removed without deprecation

error	[request-body-all-of-added] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	added '#/components/schemas/CreateModelResponseProperties, #/components/schemas/ResponseProperties, subschema #3' to the request body 'allOf' list

error	[request-body-type-changed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	the request's body type/format changed from 'object'/'' to ''/''

error	[response-body-type-changed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	the response's body type/format changed from 'object'/'' to ''/'' for status '200'

error	[response-required-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	removed the required property 'created_at' from the response with the '200' status

error	[response-required-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	removed the required property 'id' from the response with the '200' status

error	[response-required-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	removed the required property 'model' from the response with the '200' status

error	[response-required-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	removed the required property 'object' from the response with the '200' status

error	[response-required-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	removed the required property 'output' from the response with the '200' status

error	[response-required-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	removed the required property 'parallel_tool_calls' from the response with the '200' status

error	[response-required-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	removed the required property 'status' from the response with the '200' status

error	[response-required-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	removed the required property 'text' from the response with the '200' status

error	[response-media-type-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API DELETE /responses/{response_id}
  	removed the media type 'application/json' for the response with the status '200'

error	[response-body-type-changed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API GET /responses/{response_id}
  	the response's body type/format changed from 'object'/'' to ''/'' for status '200'

error	[response-required-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API GET /responses/{response_id}
  	removed the required property 'created_at' from the response with the '200' status

error	[response-required-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API GET /responses/{response_id}
  	removed the required property 'id' from the response with the '200' status

error	[response-required-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API GET /responses/{response_id}
  	removed the required property 'model' from the response with the '200' status

error	[response-required-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API GET /responses/{response_id}
  	removed the required property 'object' from the response with the '200' status

error	[response-required-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API GET /responses/{response_id}
  	removed the required property 'output' from the response with the '200' status

error	[response-required-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API GET /responses/{response_id}
  	removed the required property 'parallel_tool_calls' from the response with the '200' status

error	[response-required-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API GET /responses/{response_id}
  	removed the required property 'status' from the response with the '200' status

error	[response-required-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API GET /responses/{response_id}
  	removed the required property 'text' from the response with the '200' status

error	[request-parameter-default-value-added] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API GET /responses/{response_id}/input_items
  	for the 'query' request parameter 'limit', default value '20.00' was added

error	[response-property-type-changed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API GET /responses/{response_id}/input_items
  	the 'object' response's property type/format changed from 'string'/'' to ''/'' for status '200'

warning	[request-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	removed the request property 'include'

warning	[request-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	removed the request property 'input'

warning	[request-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	removed the request property 'instructions'

warning	[request-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	removed the request property 'max_infer_iters'

warning	[request-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	removed the request property 'model'

warning	[request-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	removed the request property 'previous_response_id'

warning	[request-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	removed the request property 'prompt'

warning	[request-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	removed the request property 'store'

warning	[request-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	removed the request property 'stream'

warning	[request-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	removed the request property 'temperature'

warning	[request-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	removed the request property 'text'

warning	[request-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	removed the request property 'tools'

warning	[response-optional-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	removed the optional property 'error' from the response with the '200' status

warning	[response-optional-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	removed the optional property 'previous_response_id' from the response with the '200' status

warning	[response-optional-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	removed the optional property 'prompt' from the response with the '200' status

warning	[response-optional-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	removed the optional property 'temperature' from the response with the '200' status

warning	[response-optional-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	removed the optional property 'top_p' from the response with the '200' status

warning	[response-optional-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	removed the optional property 'truncation' from the response with the '200' status

warning	[response-optional-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API POST /responses
  	removed the optional property 'user' from the response with the '200' status

warning	[response-optional-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API GET /responses/{response_id}
  	removed the optional property 'error' from the response with the '200' status

warning	[response-optional-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API GET /responses/{response_id}
  	removed the optional property 'previous_response_id' from the response with the '200' status

warning	[response-optional-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API GET /responses/{response_id}
  	removed the optional property 'prompt' from the response with the '200' status

warning	[response-optional-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API GET /responses/{response_id}
  	removed the optional property 'temperature' from the response with the '200' status

warning	[response-optional-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API GET /responses/{response_id}
  	removed the optional property 'top_p' from the response with the '200' status

warning	[response-optional-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API GET /responses/{response_id}
  	removed the optional property 'truncation' from the response with the '200' status

warning	[response-optional-property-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API GET /responses/{response_id}
  	removed the optional property 'user' from the response with the '200' status

warning	[request-parameter-removed] at /Users/charliedoern/Downloads/openapi.documented.yml
  in API GET /responses/{response_id}/input_items
  	deleted the 'query' request parameter 'before'
  	This is a warning because some apps may return an error when receiving a parameter that they do not expect. It is recommended to deprecate the parameter first.

there seems to be some breaking changes, BUT these might have existed in main, let me check.

[franciscojavierarceo]

hey @cdoern any update on the main branch check here?