diff --git a/CHANGELOG.md b/CHANGELOG.md index 80dd17fe..11bb8b7e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,24 @@ ## [Unreleased] +## [1.0.0] - 2026-05-04 + +First stable GA release of the HoneyHive Python SDK. Public APIs follow semver from this release forward — breaking changes only on major versions, additive on minor and patch. + +The changes below are relative to `1.0.0rc22`. For the full picture of what shipped during the release-candidate cycle, see the `1.0.0rc1`–`1.0.0rc22` entries. + +### Changed + +- **Tracing: `skip_backend_session_creation=True` now works without an explicit `session_id`** + - Previously the flag only opted out of the init-time backend roundtrip when a `session_id` was also supplied. Setting the flag alone is now supported: the SDK skips the roundtrip, does not mint a UUID, and per-request `create_session(session_id=..., skip_api_call=True)` calls emit spans that ingestion materializes into Session rows on first arrival. + +- **Tracing: `create_session(skip_api_call=True)` propagates `session_name` into baggage** + - When `session_name` is supplied, it now travels with `session_id` across async / threaded boundaries. + +### Removed + +- **`DeleteMetricQuery` removed from `honeyhive.models`** + - Internal query-params type that was exported but unused — no public method accepted or returned it. The public `client.metrics.delete(id=...)` signature is unchanged. + ## [1.0.0rc22] - 2026-05-01 ### Added @@ -42,10 +61,10 @@ ### Fixed -- **Tracing: OTLP exporter preserves attribute types [HHAI-4935]** +- **Tracing: OTLP exporter preserves attribute types** - The exporter previously wrapped every attribute as `{"stringValue": str(value)}`, so numeric and boolean attributes round-tripped through the backend as strings. Now mapped to the correct OTLP `AnyValue` variant. -- **Tracing: OTLP batch export groups spans by instrumentation scope [HHAI-4245]** +- **Tracing: OTLP batch export groups spans by instrumentation scope** - All spans in a batch were previously placed under the first span's scope, causing `BatchSpanProcessor` mixes (e.g. pydantic-ai `chat` + httpx `POST`) to misclassify model events as chains. Spans are now grouped by their actual `instrumentation_scope`. - **Integrations: Anthropic / LangChain + LangGraph / AWS Bedrock audits** @@ -238,7 +257,7 @@ No customer-facing changes. Internal CI/release automation improvements only. ### Added - **✨ API Client: Auto-generated v1 API client from OpenAPI spec** - - Generated Python client from `hive-kube` OpenAPI specification using `openapi-python-generator` + - Generated Python client from the HoneyHive OpenAPI specification using `openapi-python-generator` - Full type-safe Pydantic models for all API requests/responses - Sync and async methods for all endpoints - New endpoint support: batch events, experiment results/comparison, project CRUD diff --git a/openapi/v1.yaml b/openapi/v1.yaml index fa4365ac..c43a044f 100644 --- a/openapi/v1.yaml +++ b/openapi/v1.yaml @@ -2,6 +2,9 @@ openapi: 3.1.0 info: title: HoneyHive API version: 1.1.0 + license: + name: MIT + url: https://opensource.org/licenses/MIT servers: - url: https://api.honeyhive.ai paths: @@ -9,21 +12,18 @@ paths: post: summary: Start a new session operationId: startSession + x-cli-exclude: true + x-ts-sdk-exclude: true tags: - - Session - description: Start a new session. The session field should contain a JSON-encoded session object as a string. + - Sessions + description: | + Start a new session. The request body wraps the session event object under the `session` key, matching the pattern used by POST /events. requestBody: required: true content: application/json: schema: - type: object - required: - - session - properties: - session: - type: string - description: JSON-encoded session object containing session properties (session_id, project, session_name, source, user_properties, etc.) + $ref: '#/components/schemas/StartSessionRequest' responses: '200': description: Session successfully started @@ -31,68 +31,123 @@ paths: application/json: schema: $ref: '#/components/schemas/PostSessionStartResponse' - /v1/sessions/{session_id}: - get: - summary: Get session tree by session ID - operationId: getSession - description: Retrieve a complete session event tree including all nested events and metadata + x-honeyhive-plane: dp + /session/{session_id}/traces: + post: + summary: Add traces to a session + operationId: addSessionTraces + x-cli-exclude: true + x-ts-sdk-exclude: true tags: - Sessions + description: | + Add trace events to an existing session. The field is named `logs` for legacy compatibility with the Go ingestion handler. parameters: - name: session_id in: path required: true schema: type: string - format: uuid - description: Session ID (UUIDv4) + description: Session ID to add traces to + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/AddSessionTracesRequest' responses: '200': - description: Session tree with nested events + description: Traces added successfully content: application/json: schema: - $ref: '#/components/schemas/GetSessionResponse' - '400': - description: 'Missing required scope: org_id' - '404': - description: Session not found - '500': - description: Error fetching session - delete: - summary: Delete all events for a session - operationId: deleteSession - description: Delete all events associated with the given session ID from both events and aggregates tables + $ref: '#/components/schemas/SessionTracesResponse' + x-honeyhive-plane: dp + /events: + post: tags: - - Sessions - parameters: - - name: session_id - in: path - required: true - schema: - type: string - format: uuid - description: Session ID (UUIDv4) + - Events + operationId: createEventLegacy + x-cli-exclude: true + x-ts-sdk-exclude: true + deprecated: true + summary: Create a new event (deprecated) + description: | + Deprecated. Use `POST /v1/events` instead. The legacy route wraps the event object under an `event` key; the v1 route accepts a bare event object. + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/LegacyPostEventRequest' responses: '200': - description: Session deleted successfully + description: Event created successfully content: application/json: schema: - $ref: '#/components/schemas/DeleteSessionResponse' + $ref: '#/components/schemas/PostEventResponse' + example: + event_id: 7f22137a-6911-4ed3-bc36-110f1dde6b66 + success: true '400': - description: Invalid session ID or missing required scope - '500': - description: Error deleting session - /events: + description: Bad request (invalid event data or missing required fields) + x-honeyhive-plane: dp + put: + tags: + - Events + operationId: updateEventLegacy + x-cli-exclude: true + x-ts-sdk-exclude: true + deprecated: true + summary: Update an event (deprecated) + description: | + Deprecated. Use `PUT /v1/events/{event_id}` instead. + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/LegacyUpdateEventRequest' + example: + event_id: 7f22137a-6911-4ed3-bc36-110f1dde6b66 + metadata: + cost: 0.00008 + completion_tokens: 23 + prompt_tokens: 35 + total_tokens: 58 + feedback: + rating: 5 + metrics: + num_words: 2 + outputs: + role: assistant + content: Hello world + config: + template: + - role: system + content: Hello, {{ name }}! + user_properties: + user_id: 691b1f94-d38c-4e92-b051-5e03fee9ff86 + duration: 42 + responses: + '200': + description: Event updated + '400': + description: Bad request + x-honeyhive-plane: dp + /v1/events: post: tags: - Events operationId: createEvent + x-cli-name: create + x-ts-sdk-name: create summary: Create a new event description: | - Create a new event (span) within a session trace. The request body wraps the event in a JSON-encoded string under the `event` key. - **Required properties** within the JSON-encoded event string: + Create a new event (span) within a session trace. The request body is a bare event object (no `event` wrapper). + + **Required properties:** - `event_type` (string) — Must be one of: `chain`, `model`, `tool`, `session`. - `inputs` (object) — Input data for the event. **Auto-generated properties** (provided by the server when omitted): @@ -119,16 +174,7 @@ paths: content: application/json: schema: - type: object - required: - - event - properties: - event: - type: string - description: | - JSON-encoded event object string. Must be valid JSON containing at minimum `event_type` (one of: `chain`, `model`, `tool`, `session`) and `inputs` (object). Fields like `event_id` and `session_id` are auto-generated if not provided. - example: - event: '{"event_type":"model","event_name":"llm_call","inputs":{"prompt":"hello"},"outputs":{"response":"hi"},"config":{"model":"gpt-4","temperature":0.7},"metadata":{"cost":0.00008,"completion_tokens":23,"prompt_tokens":35},"source":"sdk-python","duration":1500,"start_time":1700000000000,"end_time":1700000001500}' + $ref: '#/components/schemas/PostEventRequest' responses: '200': description: Event created successfully @@ -141,44 +187,32 @@ paths: success: true '400': description: Bad request (invalid event data or missing required fields) + x-honeyhive-plane: dp + /v1/events/{event_id}: put: tags: - Events operationId: updateEvent + x-cli-name: update + x-ts-sdk-name: update summary: Update an event + description: | + Update fields on an existing event. Only the provided fields are modified; omitted fields are left unchanged. Extra fields not listed below are accepted by the server but silently ignored. + parameters: + - in: path + name: event_id + required: true + schema: + type: string + format: uuid + description: The unique identifier of the event to update requestBody: required: true content: application/json: schema: - type: object - properties: - event_id: - type: string - metadata: - type: object - additionalProperties: true - feedback: - type: object - additionalProperties: true - metrics: - type: object - additionalProperties: true - outputs: - type: object - additionalProperties: true - config: - type: object - additionalProperties: true - user_properties: - type: object - additionalProperties: true - duration: - type: number - required: - - event_id + $ref: '#/components/schemas/UpdateEventRequest' example: - event_id: 7f22137a-6911-4ed3-bc36-110f1dde6b66 metadata: cost: 0.00008 completion_tokens: 23 @@ -203,299 +237,148 @@ paths: description: Event updated '400': description: Bad request - get: - tags: - - Events - operationId: getEvents - summary: Query events with filters and projections - description: Retrieve events with optional filtering, projections, and pagination - parameters: - - name: dateRange - in: query - required: false - schema: - oneOf: - - type: string - - type: object - properties: - $gte: - type: string - format: date-time - $lte: - type: string - format: date-time - description: Date range filter (ISO string or object with $gte/$lte) - - name: filters - in: query - required: false - schema: - oneOf: - - type: array - items: - type: object - - type: string - description: Array of filter objects or JSON string - - name: projections - in: query - required: false - schema: - oneOf: - - type: array - items: - type: string - - type: string - description: Fields to include in response (array or JSON string) - - name: ignore_order - in: query - required: false - schema: - oneOf: - - type: boolean - - type: string - description: Whether to ignore ordering - - name: limit - in: query - required: false - schema: - oneOf: - - type: integer - - type: string - description: Maximum number of results (default 1000) - - name: page - in: query - required: false - schema: - oneOf: - - type: integer - - type: string - description: Page number (default 1) - - name: evaluation_id - in: query - required: false - schema: - type: string - description: Filter by evaluation ID - responses: - '200': - description: Events retrieved successfully - content: - application/json: - schema: - $ref: '#/components/schemas/GetEventsResponse' - '400': - description: Bad request (missing required scopes or invalid parameters) - /v1/events/chart: - get: - tags: - - Events - operationId: getEventsChart - summary: Get charting data for events - description: Retrieve aggregated chart data for events with optional grouping and bucketing - parameters: - - name: dateRange - in: query - required: false - schema: - oneOf: - - type: string - - type: object - properties: - $gte: - type: string - format: date-time - $lte: - type: string - format: date-time - description: Date range filter (ISO string or object with $gte/$lte) - - name: filters - in: query - required: false - schema: - oneOf: - - type: array - items: - type: object - - type: string - description: Array of filter objects or JSON string - - name: metric - in: query - required: false - schema: - type: string - description: Metric to aggregate (default 'duration') - - name: groupBy - in: query - required: false - schema: - type: string - description: Field to group by - - name: bucket - in: query - required: false - schema: - type: string - enum: - - minute - - minutes - - 1m - - hour - - hours - - 1h - - day - - days - - 1d - - week - - weeks - - 1w - - month - - months - - 1M - description: Time bucket for aggregation (default 'hour') - - name: aggregation - in: query - required: false - schema: - type: string - enum: - - avg - - average - - mean - - p50 - - p75 - - p90 - - p95 - - p99 - - count - - sum - - min - - max - - median - description: Aggregation function (default 'average') - - name: evaluation_id - in: query - required: false - schema: - type: string - description: Filter by evaluation ID - - name: only_experiments - in: query - required: false - schema: - oneOf: - - type: boolean - - type: string - description: Filter to only experiment events - responses: - '200': - description: Chart data retrieved successfully - content: - application/json: - schema: - $ref: '#/components/schemas/GetEventsChartResponse' - '400': - description: Bad request (missing required scopes or invalid parameters) - /v1/events/{id}: - get: - tags: - - Events - operationId: getEventsBySessionId - summary: Get nested events for a session - description: Retrieve all nested events for a specific session ID. The `id` parameter is interpreted as a session_id for this operation. - parameters: - - name: id - in: path - required: true - schema: - type: string - format: uuid - description: Session ID (UUIDv4) - retrieves all events belonging to this session - responses: - '200': - description: Session events retrieved successfully - content: - application/json: - schema: - $ref: '#/components/schemas/GetEventsBySessionIdResponse' - '400': - description: Bad request (missing required scopes or invalid session ID) - delete: + x-honeyhive-plane: dp + /v1/events/export: + post: tags: - Events - operationId: deleteEvent - summary: Delete an event - description: Delete a specific event by event ID. The `id` parameter is interpreted as an event_id for this operation. - parameters: - - name: id - in: path - required: true - schema: - type: string - format: uuid - description: Event ID (UUIDv4) - the specific event to delete + operationId: exportEventsLegacy + x-cli-exclude: true + x-ts-sdk-exclude: true + deprecated: true + summary: Retrieve events based on filters (deprecated) + description: | + Deprecated. Use `POST /v1/events/search` instead. + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/LegacyExportEventsRequest' responses: '200': - description: Event deleted successfully + description: Success content: application/json: schema: - $ref: '#/components/schemas/DeleteEventResponse' - '400': - description: Bad request (missing required scopes or invalid event ID) - /v1/events/export: + $ref: '#/components/schemas/ExportEventsResponse' + x-honeyhive-plane: dp + /v1/events/search: post: tags: - Events - operationId: exportEvents + operationId: searchEvents + x-cli-name: search + x-ts-sdk-name: search summary: Retrieve events based on filters + description: | + Search events via POST with filtering and pagination. This is the primary method for retrieving events from HoneyHive. requestBody: required: true content: application/json: schema: - $ref: '#/components/schemas/GetEventsLegacyRequest' + $ref: '#/components/schemas/SearchEventsRequest' responses: '200': description: Success content: application/json: schema: - $ref: '#/components/schemas/GetEventsLegacyResponse' - /events/model: + $ref: '#/components/schemas/ExportEventsResponse' + x-honeyhive-plane: dp + /v1/events/batch: post: tags: - Events - operationId: createModelEvent - summary: Create a new model event - description: Please refer to our instrumentation guide for detailed information + operationId: createEventBatch + x-cli-name: create-batch + x-ts-sdk-name: createBatch + summary: Create a batch of events + description: | + Create multiple events in a single request. When `single_session` is true, all events share the same session created from `session_properties`. + + **Required properties:** + - `events` (array of event objects) — Each event must include + `event_type` (one of `chain`, `model`, `tool`, `session`) and `inputs`. + + **Optional properties:** + - `single_session` (boolean) — If true, all events share a single session + created from `session_properties`. Defaults to false. + + - `session_properties` (object) — Session metadata used when + `single_session` is true. May include `session_name`, `start_time`, + `metadata`. + + Unknown top-level fields and per-event fields are rejected at the SDK boundary; the legacy aliases `is_single_session`, `session`, and per-event `project` are no longer accepted. requestBody: required: true content: application/json: schema: - $ref: '#/components/schemas/PostModelEventRequest' + $ref: '#/components/schemas/PostEventBatchRequest' responses: '200': - description: Model event created + description: Events created content: application/json: schema: - $ref: '#/components/schemas/PostEventResponse' + $ref: '#/components/schemas/PostEventBatchResponse' + example: + event_ids: + - 7f22137a-6911-4ed3-bc36-110f1dde6b66 + - 7f22137a-6911-4ed3-bc36-110f1dde6b67 + session_id: caf77ace-3417-4da4-944d-f4a0688f3c23 + success: true + '400': + description: Bad request (invalid event data or missing required fields) + '500': + description: Events partially created + x-honeyhive-plane: dp + /events/model: + post: + tags: + - Events + operationId: createModelEvent + x-cli-exclude: true + x-ts-sdk-exclude: true + summary: Create a new model event + description: | + Create a model event. The event_type is automatically set to 'model'. Please refer to our instrumentation guide for detailed information. + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/PostModelEventRequest' + responses: + '200': + description: Model event created + content: + application/json: + schema: + $ref: '#/components/schemas/PostEventResponse' example: event_id: 7f22137a-6911-4ed3-bc36-110f1dde6b66 success: true + x-honeyhive-plane: dp /events/batch: post: tags: - Events - operationId: createEventBatch - summary: Create a batch of events - description: Please refer to our instrumentation guide for detailed information + operationId: createEventBatchLegacy + x-cli-exclude: true + x-ts-sdk-exclude: true + deprecated: true + summary: Create a batch of events (deprecated) + description: | + Deprecated. Use `POST /v1/events/batch` instead. The legacy route accepts the deprecated `is_single_session` and `session` aliases and lets per-event objects carry the deprecated `project` field; the v1 route rejects all three at the SDK boundary. requestBody: required: true content: application/json: schema: - $ref: '#/components/schemas/PostEventBatchRequest' + $ref: '#/components/schemas/LegacyPostEventBatchRequest' responses: '200': description: Events created @@ -511,37 +394,17 @@ paths: success: true '500': description: Events partially created - content: - application/json: - schema: - type: object - properties: - event_ids: - type: array - items: - type: string - errors: - type: array - items: - type: string - description: Any failure messages for events that could not be created - success: - type: boolean - example: - event_ids: - - 7f22137a-6911-4ed3-bc36-110f1dde6b66 - - 7f22137a-6911-4ed3-bc36-110f1dde6b67 - errors: - - Could not create event due to missing inputs - - Could not create event due to missing source - success: true + x-honeyhive-plane: dp /events/model/batch: post: tags: - Events operationId: createModelEventBatch + x-cli-exclude: true + x-ts-sdk-exclude: true summary: Create a batch of model events - description: Please refer to our instrumentation guide for detailed information + description: | + Create multiple model events in a single request. The event_type is automatically set to 'model' for all events. When single_session is true, all events share the same session. Please refer to our instrumentation guide for detailed information. requestBody: required: true content: @@ -562,35 +425,14 @@ paths: success: true '500': description: Model events partially created - content: - application/json: - schema: - type: object - properties: - event_ids: - type: array - items: - type: string - errors: - type: array - items: - type: string - description: Any failure messages for events that could not be created - success: - type: boolean - example: - event_ids: - - 7f22137a-6911-4ed3-bc36-110f1dde6b66 - - 7f22137a-6911-4ed3-bc36-110f1dde6b67 - errors: - - Could not create event due to missing model - - Could not create event due to missing provider - success: true + x-honeyhive-plane: dp /v1/metrics: get: tags: - Metrics operationId: getMetrics + x-cli-name: list + x-ts-sdk-name: list summary: Get all metrics description: Retrieve a list of all metrics parameters: @@ -608,17 +450,18 @@ paths: description: Filter by specific metric ID responses: '200': - description: A list of metrics + description: Metrics retrieved successfully content: application/json: schema: - type: array - items: - $ref: '#/components/schemas/GetMetricsResponse' + $ref: '#/components/schemas/GetMetricsResponse' + x-honeyhive-plane: dp post: tags: - Metrics operationId: createMetric + x-cli-name: create + x-ts-sdk-name: create summary: Create a new metric description: Add a new metric requestBody: @@ -634,12 +477,70 @@ paths: application/json: schema: $ref: '#/components/schemas/CreateMetricResponse' + x-honeyhive-plane: dp + put: + tags: + - Metrics + operationId: updateMetricLegacy + x-cli-exclude: true + x-ts-sdk-exclude: true + deprecated: true + summary: Update an existing metric (deprecated) + description: Deprecated. Use `PUT /v1/metrics/{metric_id}` instead. + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/LegacyUpdateMetricRequest' + responses: + '200': + description: Metric updated successfully + content: + application/json: + schema: + $ref: '#/components/schemas/UpdateMetricResponse' + x-honeyhive-plane: dp + delete: + tags: + - Metrics + operationId: deleteMetricLegacy + x-cli-exclude: true + x-ts-sdk-exclude: true + deprecated: true + summary: Delete a metric (deprecated) + description: Deprecated. Use `DELETE /v1/metrics/{metric_id}` instead. + parameters: + - name: metric_id + in: query + required: true + schema: + type: string + description: Unique identifier of the metric + responses: + '200': + description: Metric deleted successfully + content: + application/json: + schema: + $ref: '#/components/schemas/DeleteMetricResponse' + x-honeyhive-plane: dp + /v1/metrics/{metric_id}: put: tags: - Metrics operationId: updateMetric + x-cli-name: update + x-ts-sdk-name: update summary: Update an existing metric - description: Edit a metric + description: Update a metric's editable fields. Only fields included in the request body are modified. + parameters: + - in: path + name: metric_id + required: true + schema: + type: string + description: The unique identifier of the metric to update requestBody: required: true content: @@ -653,19 +554,22 @@ paths: application/json: schema: $ref: '#/components/schemas/UpdateMetricResponse' + x-honeyhive-plane: dp delete: tags: - Metrics operationId: deleteMetric + x-cli-name: delete + x-ts-sdk-name: delete summary: Delete a metric - description: Remove a metric + description: Remove a metric by id. parameters: - - name: metric_id - in: query + - in: path + name: metric_id required: true schema: type: string - description: Unique identifier of the metric + description: The unique identifier of the metric to delete responses: '200': description: Metric deleted successfully @@ -673,11 +577,14 @@ paths: application/json: schema: $ref: '#/components/schemas/DeleteMetricResponse' - /v1/metrics/run_metric: + x-honeyhive-plane: dp + /v1/metrics/run: post: tags: - Metrics operationId: runMetric + x-cli-name: run + x-ts-sdk-name: run summary: Run a metric evaluation description: Execute a metric on a specific event requestBody: @@ -693,12 +600,41 @@ paths: application/json: schema: $ref: '#/components/schemas/RunMetricResponse' + x-honeyhive-plane: dp + /v1/metrics/run_metric: + post: + tags: + - Metrics + operationId: runMetricLegacy + deprecated: true + x-cli-exclude: true + x-ts-sdk-exclude: true + summary: Run a metric evaluation (deprecated) + description: Deprecated. Use `POST /v1/metrics/run` instead. + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/LegacyRunMetricRequest' + responses: + '200': + description: Metric execution result + content: + application/json: + schema: + $ref: '#/components/schemas/RunMetricResponse' + x-honeyhive-plane: dp /v1/datapoints: get: summary: Retrieve a list of datapoints operationId: getDatapoints + x-cli-name: list + x-ts-sdk-name: list tags: - Datapoints + description: | + Retrieve datapoints, optionally filtered by a list of datapoint IDs or dataset name. parameters: - name: datapoint_ids in: query @@ -721,11 +657,15 @@ paths: application/json: schema: $ref: '#/components/schemas/GetDatapointsResponse' + x-honeyhive-plane: dp post: summary: Create a new datapoint operationId: createDatapoint + x-cli-name: create + x-ts-sdk-name: create tags: - Datapoints + description: Create a single datapoint with inputs, history, ground truth, and metadata. requestBody: required: true content: @@ -739,12 +679,16 @@ paths: application/json: schema: $ref: '#/components/schemas/CreateDatapointResponse' + x-honeyhive-plane: dp /v1/datapoints/batch: post: summary: Create multiple datapoints in batch operationId: batchCreateDatapoints + x-cli-name: create-batch + x-ts-sdk-name: createBatch tags: - Datapoints + description: Create multiple datapoints from events using field mappings and optional filters. requestBody: required: true content: @@ -758,14 +702,18 @@ paths: application/json: schema: $ref: '#/components/schemas/BatchCreateDatapointsResponse' - /v1/datapoints/{id}: + x-honeyhive-plane: dp + /v1/datapoints/{datapoint_id}: get: summary: Retrieve a specific datapoint operationId: getDatapoint + x-cli-name: get + x-ts-sdk-name: get tags: - Datapoints + description: Get a single datapoint by its unique identifier. parameters: - - name: id + - name: datapoint_id in: path required: true schema: @@ -776,23 +724,22 @@ paths: content: application/json: schema: - type: object - properties: - datapoint: - type: array - items: - $ref: '#/components/schemas/GetDatapointResponse' + $ref: '#/components/schemas/GetDatapointResponse' description: Successful response + x-honeyhive-plane: dp put: summary: Update a specific datapoint + description: Update fields on an existing datapoint. Only the provided fields are modified. parameters: - - name: id + - name: datapoint_id in: path required: true schema: type: string description: ID of datapoint to update operationId: updateDatapoint + x-cli-name: update + x-ts-sdk-name: update tags: - Datapoints requestBody: @@ -810,13 +757,17 @@ paths: $ref: '#/components/schemas/UpdateDatapointResponse' '400': description: Error updating datapoint + x-honeyhive-plane: dp delete: summary: Delete a specific datapoint operationId: deleteDatapoint + x-cli-name: delete + x-ts-sdk-name: delete tags: - Datapoints + description: Permanently delete a datapoint by its unique identifier. parameters: - - name: id + - name: datapoint_id in: path required: true schema: @@ -829,12 +780,16 @@ paths: application/json: schema: $ref: '#/components/schemas/DeleteDatapointResponse' + x-honeyhive-plane: dp /v1/datasets: get: tags: - Datasets summary: Get datasets operationId: getDatasets + x-cli-name: list + x-ts-sdk-name: list + description: Retrieve datasets, optionally filtered by dataset ID or name. parameters: - in: query name: dataset_id @@ -855,11 +810,15 @@ paths: application/json: schema: $ref: '#/components/schemas/GetDatasetsResponse' + x-honeyhive-plane: dp post: tags: - Datasets operationId: createDataset + x-cli-name: create + x-ts-sdk-name: create summary: Create a dataset + description: Create a new dataset with an optional name, description, and initial set of datapoint IDs. requestBody: required: true content: @@ -873,11 +832,70 @@ paths: application/json: schema: $ref: '#/components/schemas/CreateDatasetResponse' + x-honeyhive-plane: dp + put: + tags: + - Datasets + operationId: updateDatasetLegacy + x-cli-exclude: true + x-ts-sdk-exclude: true + deprecated: true + summary: Update a dataset (deprecated) + description: Deprecated. Use `PUT /v1/datasets/{dataset_id}` instead. + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/LegacyUpdateDatasetRequest' + responses: + '200': + description: Successful update + content: + application/json: + schema: + $ref: '#/components/schemas/UpdateDatasetResponse' + x-honeyhive-plane: dp + delete: + tags: + - Datasets + operationId: deleteDatasetLegacy + x-cli-exclude: true + x-ts-sdk-exclude: true + deprecated: true + summary: Delete a dataset (deprecated) + description: Deprecated. Use `DELETE /v1/datasets/{dataset_id}` instead. + parameters: + - in: query + name: dataset_id + required: true + schema: + type: string + description: The unique identifier of the dataset to be deleted like `663876ec4611c47f4970f0c3` + responses: + '200': + description: Successful delete + content: + application/json: + schema: + $ref: '#/components/schemas/DeleteDatasetResponse' + x-honeyhive-plane: dp + /v1/datasets/{dataset_id}: put: tags: - Datasets operationId: updateDataset + x-cli-name: update + x-ts-sdk-name: update summary: Update a dataset + description: Update a dataset's name, description, or list of datapoint IDs. + parameters: + - in: path + name: dataset_id + required: true + schema: + type: string + description: The unique identifier of the dataset to update like `663876ec4611c47f4970f0c3` requestBody: required: true content: @@ -891,13 +909,17 @@ paths: application/json: schema: $ref: '#/components/schemas/UpdateDatasetResponse' + x-honeyhive-plane: dp delete: tags: - Datasets operationId: deleteDataset + x-cli-name: delete + x-ts-sdk-name: delete summary: Delete a dataset + description: Permanently delete a dataset by its unique identifier. parameters: - - in: query + - in: path name: dataset_id required: true schema: @@ -910,12 +932,17 @@ paths: application/json: schema: $ref: '#/components/schemas/DeleteDatasetResponse' + x-honeyhive-plane: dp /v1/datasets/{dataset_id}/datapoints: post: tags: - Datasets summary: Add datapoints to a dataset operationId: addDatapoints + x-cli-name: add-datapoints + x-ts-sdk-name: addDatapoints + description: | + Add new datapoints to an existing dataset. Provide raw data objects and a field mapping that specifies which fields map to inputs, ground truth, and history. parameters: - in: path name: dataset_id @@ -936,12 +963,16 @@ paths: application/json: schema: $ref: '#/components/schemas/AddDatapointsResponse' + x-honeyhive-plane: dp /v1/datasets/{dataset_id}/datapoints/{datapoint_id}: delete: tags: - Datasets summary: Remove a datapoint from a dataset operationId: removeDatapoint + x-cli-name: remove-datapoint + x-ts-sdk-name: removeDatapoint + description: Remove a specific datapoint from a dataset. The datapoint itself is not deleted, only dereferenced from the dataset. parameters: - in: path name: dataset_id @@ -962,79 +993,48 @@ paths: application/json: schema: $ref: '#/components/schemas/RemoveDatapointResponse' - /v1/projects: - get: + x-honeyhive-plane: dp + /v1/datasets/{dataset_id}/{datapoint_id}: + delete: tags: - - Projects - summary: Get a list of projects - operationId: getProjects + - Datasets + summary: Remove a datapoint from a dataset (deprecated) + operationId: removeDatapointLegacy + x-cli-exclude: true + x-ts-sdk-exclude: true + deprecated: true + description: Deprecated. Use `DELETE /v1/datasets/{dataset_id}/datapoints/{datapoint_id}` instead. parameters: - - in: query - name: name - required: false + - in: path + name: dataset_id + required: true schema: type: string - description: Filter projects by name - responses: - '200': - description: A list of projects - content: - application/json: - schema: - $ref: '#/components/schemas/GetProjectsResponse' - post: - tags: - - Projects - summary: Create a new project - operationId: createProject - requestBody: - required: true - content: - application/json: - schema: - $ref: '#/components/schemas/PostProjectRequest' - responses: - '200': - description: The created project - content: - application/json: - schema: - $ref: '#/components/schemas/PostProjectResponse' - put: - tags: - - Projects - summary: Update an existing project - operationId: updateProject - requestBody: - required: true - content: - application/json: - schema: - $ref: '#/components/schemas/PutProjectRequest' - responses: - '200': - description: Successfully updated the project - delete: - tags: - - Projects - summary: Delete a project - operationId: deleteProject - parameters: - - in: query - name: name + description: The unique identifier of the dataset + - in: path + name: datapoint_id required: true schema: type: string + description: The unique identifier of the datapoint to remove responses: '200': - description: Project deleted - /v1/runs/schema: + description: Datapoint successfully removed from dataset + content: + application/json: + schema: + $ref: '#/components/schemas/RemoveDatapointResponse' + x-honeyhive-plane: dp + /v1/events/schema: get: - summary: Get experiment runs schema - operationId: getExperimentRunsSchema + summary: Get events schema (deprecated) + operationId: getEventsSchemaLegacy + deprecated: true + x-cli-exclude: true + x-ts-sdk-exclude: true tags: - - Experiments - description: Retrieve the schema and metadata for experiment runs + - Events + description: Deprecated. Use `GET /v1/runs/{run_id}/schema` (single-run) or `GET /v1/runs/schema` (project-wide) instead. parameters: - in: query name: dateRange @@ -1042,16 +1042,7 @@ paths: schema: oneOf: - type: string - - type: object - properties: - $gte: - oneOf: - - type: string - - type: number - $lte: - oneOf: - - type: string - - type: number + - $ref: '#/components/schemas/GetEventsSchemaLegacyDateRangeOneOf1' description: Filter by date range - in: query name: evaluation_id @@ -1061,17 +1052,21 @@ paths: description: Filter by evaluation/run ID responses: '200': - description: Experiment runs schema retrieved successfully + description: Events schema retrieved successfully content: application/json: schema: - $ref: '#/components/schemas/GetExperimentRunsSchemaResponse' + $ref: '#/components/schemas/GetEventsSchemaResponse' + x-honeyhive-plane: dp /v1/runs: post: summary: Create a new evaluation run operationId: createRun + x-cli-name: create-run + x-ts-sdk-name: createRun tags: - Experiments + description: Create a new experiment run to track an evaluation against a dataset. requestBody: required: true content: @@ -1087,11 +1082,16 @@ paths: $ref: '#/components/schemas/PostExperimentRunResponse' '400': description: Invalid input + x-honeyhive-plane: dp get: summary: Get a list of evaluation runs operationId: getRuns + x-cli-name: list-runs + x-ts-sdk-name: listRuns tags: - Experiments + description: | + List experiment runs with optional filtering by dataset, status, name, date range, and specific run IDs. Results are paginated and sortable. parameters: - in: query name: dataset_id @@ -1148,16 +1148,7 @@ paths: schema: oneOf: - type: string - - type: object - properties: - $gte: - oneOf: - - type: string - - type: number - $lte: - oneOf: - - type: string - - type: number + - $ref: '#/components/schemas/GetRunsDateRangeOneOf1' description: Filter by date range - in: query name: sort_by @@ -1190,18 +1181,51 @@ paths: $ref: '#/components/schemas/GetExperimentRunsResponse' '400': description: Error fetching evaluations + x-honeyhive-plane: dp + /v1/runs/schema: + get: + summary: Get events schema across all experiment runs in a project + operationId: getRunsSchema + x-cli-name: get-runs-schema + x-ts-sdk-name: getRunsSchema + tags: + - Experiments + description: Retrieve the aggregated events schema (fields, datasets, mappings) across all experiment runs in the project. + parameters: + - in: query + name: dateRange + required: false + schema: + oneOf: + - type: string + - $ref: '#/components/schemas/GetRunsSchemaDateRangeOneOf1' + description: Filter by date range + responses: + '200': + description: Events schema retrieved successfully + content: + application/json: + schema: + $ref: '#/components/schemas/GetEventsSchemaResponse' + '400': + description: Error fetching events schema + x-honeyhive-plane: dp /v1/runs/{run_id}: get: summary: Get details of an evaluation run operationId: getRun + x-cli-name: get-run + x-ts-sdk-name: getRun tags: - Experiments + description: Retrieve the full details of a single experiment run by its run ID. parameters: - in: path name: run_id required: true schema: type: string + format: uuid responses: '200': description: Successful response @@ -1211,17 +1235,22 @@ paths: $ref: '#/components/schemas/GetExperimentRunResponse' '400': description: Error fetching evaluation + x-honeyhive-plane: dp put: summary: Update an evaluation run operationId: updateRun + x-cli-name: update-run + x-ts-sdk-name: updateRun tags: - Experiments + description: Update fields on an existing experiment run such as name, status, metadata, or results. parameters: - in: path name: run_id required: true schema: type: string + format: uuid requestBody: required: true content: @@ -1237,17 +1266,22 @@ paths: $ref: '#/components/schemas/PutExperimentRunResponse' '400': description: Invalid input + x-honeyhive-plane: dp delete: summary: Delete an evaluation run operationId: deleteRun + x-cli-name: delete-run + x-ts-sdk-name: deleteRun tags: - Experiments + description: Permanently delete an experiment run by its run ID. parameters: - in: path name: run_id required: true schema: type: string + format: uuid responses: '200': description: Successful response @@ -1257,10 +1291,48 @@ paths: $ref: '#/components/schemas/DeleteExperimentRunResponse' '400': description: Error deleting evaluation + x-honeyhive-plane: dp + /v1/runs/{run_id}/schema: + get: + summary: Get events schema for a single experiment run + operationId: getRunSchema + x-cli-name: get-run-schema + x-ts-sdk-name: getRunSchema + tags: + - Experiments + description: Retrieve the events schema (fields, datasets, mappings) for a single experiment run. + parameters: + - name: run_id + in: path + required: true + schema: + type: string + format: uuid + description: Experiment run ID (UUIDv4) + - in: query + name: dateRange + required: false + schema: + oneOf: + - type: string + - $ref: '#/components/schemas/GetRunSchemaDateRangeOneOf1' + description: Filter by date range + responses: + '200': + description: Events schema retrieved successfully + content: + application/json: + schema: + $ref: '#/components/schemas/GetEventsSchemaResponse' + '400': + description: Error fetching events schema + x-honeyhive-plane: dp /v1/runs/{run_id}/metrics: get: summary: Get event metrics for an experiment run operationId: getExperimentRunMetrics + x-cli-name: get-run-metrics + x-ts-sdk-name: getRunMetrics tags: - Experiments description: Retrieve event metrics from ClickHouse for a specific experiment run @@ -1294,22 +1366,19 @@ paths: content: application/json: schema: - type: object - properties: - events: - type: array - items: - type: object - description: Array of event objects with metrics + $ref: '#/components/schemas/GetExperimentRunMetricsResponse' '400': description: Error fetching event metrics + x-honeyhive-plane: dp /v1/runs/{run_id}/result: get: summary: Retrieve experiment result operationId: getExperimentResult + x-cli-exclude: true + x-ts-sdk-exclude: true tags: - Experiments - description: Compute evaluation summary for an experiment run including pass/fail status, metrics, and datapoints + description: 'Compute evaluation summary for an experiment run: pass/fail results, metric aggregations, per-datapoint results, event details, and the experiment run object.' parameters: - name: run_id in: path @@ -1354,10 +1423,13 @@ paths: $ref: '#/components/schemas/GetExperimentRunResultResponse' '400': description: Error processing experiment result - /v1/runs/{new_run_id}/compare-with/{old_run_id}: + x-honeyhive-plane: dp + /v1/runs/{new_run_id}/compare/{old_run_id}: get: summary: Retrieve experiment comparison operationId: getExperimentComparison + x-cli-name: compare-runs + x-ts-sdk-name: compareRuns tags: - Experiments description: Compare metrics and results between two experiment runs @@ -1412,13 +1484,139 @@ paths: $ref: '#/components/schemas/GetExperimentRunCompareResponse' '400': description: Error processing experiment comparison - /v1/runs/compare/events: + x-honeyhive-plane: dp + /v1/runs/{new_run_id}/compare-with/{old_run_id}: get: - summary: Compare events between two experiment runs - operationId: getExperimentCompareEvents + summary: Retrieve experiment comparison (deprecated) + operationId: getExperimentComparisonLegacy + deprecated: true + x-cli-exclude: true + x-ts-sdk-exclude: true tags: - Experiments - description: Retrieve and compare events between two experiment runs for detailed analysis + description: Deprecated. Use `GET /v1/runs/{new_run_id}/compare/{old_run_id}` instead. + parameters: + - name: new_run_id + in: path + required: true + schema: + type: string + format: uuid + description: New experiment run ID to compare (UUIDv4) + - name: old_run_id + in: path + required: true + schema: + type: string + format: uuid + description: Old experiment run ID to compare against (UUIDv4) + - name: aggregate_function + in: query + required: false + schema: + type: string + description: Aggregation function to apply to metrics + - name: filters + in: query + required: false + schema: + oneOf: + - type: string + - type: array + items: + type: object + description: Optional filters to apply (JSON string or array of filter objects) + responses: + '200': + description: Experiment comparison retrieved successfully + content: + application/json: + schema: + $ref: '#/components/schemas/GetExperimentRunCompareResponse' + '400': + description: Error processing experiment comparison + x-honeyhive-plane: dp + /v1/runs/{new_run_id}/compare/{old_run_id}/events: + get: + summary: Compare events between two experiment runs + operationId: getExperimentCompareEvents + x-cli-name: compare-run-events + x-ts-sdk-name: compareRunEvents + tags: + - Experiments + description: Retrieve and compare events between two experiment runs for detailed analysis + parameters: + - name: new_run_id + in: path + required: true + schema: + type: string + format: uuid + description: New experiment run ID (UUIDv4) + - name: old_run_id + in: path + required: true + schema: + type: string + format: uuid + description: Old experiment run ID to compare against (UUIDv4) + - name: event_name + in: query + required: false + schema: + type: string + description: Filter by event name + - name: event_type + in: query + required: false + schema: + type: string + description: Filter by event type + - name: filter + in: query + required: false + schema: + oneOf: + - type: string + - type: object + description: Additional filter criteria (JSON string or object) + - name: limit + in: query + required: false + schema: + type: integer + minimum: 1 + maximum: 1000 + default: 1000 + description: Maximum number of results + - name: page + in: query + required: false + schema: + type: integer + minimum: 1 + default: 1 + description: Page number for pagination + responses: + '200': + description: Event comparison retrieved successfully + content: + application/json: + schema: + $ref: '#/components/schemas/GetExperimentCompareEventsResponse' + '400': + description: Error fetching event comparison + x-honeyhive-plane: dp + /v1/runs/compare/events: + get: + summary: Compare events between two experiment runs (deprecated) + operationId: getExperimentCompareEventsLegacy + deprecated: true + x-cli-exclude: true + x-ts-sdk-exclude: true + tags: + - Experiments + description: Deprecated. Use `GET /v1/runs/{new_run_id}/compare/{old_run_id}/events` instead. parameters: - name: run_id_1 in: query @@ -1477,30 +1675,153 @@ paths: content: application/json: schema: - type: object - properties: - events: - type: array - items: - type: object - description: Array of matched event pairs from both runs - pagination: - type: object - properties: - page: - type: integer - limit: - type: integer - total: - type: integer + $ref: '#/components/schemas/GetExperimentCompareEventsResponse' '400': description: Error fetching event comparison + x-honeyhive-plane: dp + /v1/queues: + get: + tags: + - Queues + operationId: getQueues + x-cli-name: list + x-ts-sdk-name: list + summary: List annotation queues + description: | + List annotation queues for the current project scope, optionally filtered by enabled status. + parameters: + - name: enabled + in: query + required: false + schema: + type: boolean + description: Filter by enabled status + responses: + '200': + description: Queues retrieved successfully + content: + application/json: + schema: + $ref: '#/components/schemas/GetAnnotationQueuesResponse' + x-honeyhive-plane: dp + post: + tags: + - Queues + operationId: createQueue + x-cli-name: create + x-ts-sdk-name: create + summary: Create an annotation queue + description: | + Create a new annotation queue with a name, optional description, filters, and an initial set of event IDs to add. + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/CreateAnnotationQueueRequest' + responses: + '201': + description: Queue created successfully + content: + application/json: + schema: + $ref: '#/components/schemas/CreateAnnotationQueueResponse' + '400': + description: Bad request (invalid queue data or missing required fields) + x-honeyhive-plane: dp + /v1/queues/{queue_id}: + get: + tags: + - Queues + operationId: getQueue + x-cli-name: get + x-ts-sdk-name: get + summary: Get an annotation queue + description: Retrieve a single annotation queue by its unique identifier. + parameters: + - name: queue_id + in: path + required: true + schema: + type: string + description: Annotation queue ID + responses: + '200': + description: Queue retrieved successfully + content: + application/json: + schema: + $ref: '#/components/schemas/GetAnnotationQueueByIdResponse' + '404': + description: Queue not found + x-honeyhive-plane: dp + put: + tags: + - Queues + operationId: updateQueue + x-cli-name: update + x-ts-sdk-name: update + summary: Update an annotation queue + description: | + Update fields on an existing annotation queue. Supports updating name, description, filters, enabled status, and adding/removing events. + parameters: + - name: queue_id + in: path + required: true + schema: + type: string + description: Annotation queue ID + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/UpdateAnnotationQueueRequest' + responses: + '200': + description: Queue updated successfully + content: + application/json: + schema: + $ref: '#/components/schemas/UpdateAnnotationQueueResponse' + '404': + description: Queue not found + x-honeyhive-plane: dp + delete: + tags: + - Queues + operationId: deleteQueue + x-cli-name: delete + x-ts-sdk-name: delete + summary: Delete an annotation queue + description: Soft-delete an annotation queue by its unique identifier. + parameters: + - name: queue_id + in: path + required: true + schema: + type: string + description: Annotation queue ID + responses: + '200': + description: Queue deleted successfully + content: + application/json: + schema: + $ref: '#/components/schemas/DeleteAnnotationQueueResponse' + '404': + description: Queue not found + x-honeyhive-plane: dp /v1/configurations: get: summary: Retrieve a list of configurations operationId: getConfigurations + x-cli-exclude: true + x-ts-sdk-exclude: true + deprecated: true tags: - Configurations + description: List configurations with optional filtering by name, environment, and tags. parameters: - name: name in: query @@ -1522,18 +1843,21 @@ paths: description: Tags to filter configurations responses: '200': - description: An array of configurations + description: Configurations retrieved successfully content: application/json: schema: - type: array - items: - $ref: '#/components/schemas/GetConfigurationsResponse' + $ref: '#/components/schemas/GetConfigurationsResponse' + x-honeyhive-plane: dp post: summary: Create a new configuration operationId: createConfiguration + x-cli-exclude: true + x-ts-sdk-exclude: true + deprecated: true tags: - Configurations + description: Create a new LLM or pipeline configuration with provider, parameters, and environment settings. requestBody: required: true content: @@ -1547,14 +1871,19 @@ paths: application/json: schema: $ref: '#/components/schemas/CreateConfigurationResponse' - /v1/configurations/{id}: + x-honeyhive-plane: dp + /v1/configurations/{configId}: put: summary: Update an existing configuration operationId: updateConfiguration + x-cli-exclude: true + x-ts-sdk-exclude: true + deprecated: true tags: - Configurations + description: Update an existing configuration's name, provider, parameters, environment, or tags. parameters: - - name: id + - name: configId in: path required: true schema: @@ -1573,13 +1902,18 @@ paths: application/json: schema: $ref: '#/components/schemas/UpdateConfigurationResponse' + x-honeyhive-plane: dp delete: summary: Delete a configuration operationId: deleteConfiguration + x-cli-exclude: true + x-ts-sdk-exclude: true + deprecated: true tags: - Configurations + description: Permanently delete a configuration by its unique identifier. parameters: - - name: id + - name: configId in: path required: true schema: @@ -1592,6 +1926,7 @@ paths: application/json: schema: $ref: '#/components/schemas/DeleteConfigurationResponse' + x-honeyhive-plane: dp components: schemas: SelectedFunction: @@ -1682,7 +2017,6 @@ components: enum: - LLM - pipeline - default: LLM provider: type: string parameters: @@ -1715,6 +2049,7 @@ components: required: - id - name + - type - provider - parameters - env @@ -1759,6 +2094,7 @@ components: - provider - parameters additionalProperties: false + description: Request body for POST /configurations UpdateConfigurationRequest: type: object properties: @@ -1796,16 +2132,21 @@ components: required: - name additionalProperties: false + description: Request body for PUT /configurations GetConfigurationsQuery: type: object properties: name: type: string maxLength: 200 + description: Filter configurations by name env: type: string + description: Filter configurations by environment tags: type: string + description: Filter configurations by tags + description: Query parameters for GET /configurations CreateConfigurationResponse: type: object properties: @@ -1817,6 +2158,7 @@ components: required: - acknowledged - insertedId + description: Response for POST /configurations UpdateConfigurationResponse: type: object properties: @@ -1825,7 +2167,9 @@ components: modifiedCount: type: number upsertedId: - type: 'null' + type: + - string + - 'null' upsertedCount: type: number matchedCount: @@ -1836,6 +2180,7 @@ components: - upsertedId - upsertedCount - matchedCount + description: Response for PUT /configurations DeleteConfigurationResponse: type: object properties: @@ -1846,10 +2191,17 @@ components: required: - acknowledged - deletedCount + description: Response for DELETE /configurations GetConfigurationsResponse: - type: array - items: - $ref: '#/components/schemas/ConfigurationItem' + type: object + properties: + configurations: + type: array + items: + $ref: '#/components/schemas/ConfigurationItem' + required: + - configurations + description: Response for GET /configurations DatapointMapping: type: object properties: @@ -1888,25 +2240,21 @@ components: inputs: type: object additionalProperties: {} - default: {} history: type: array items: type: object additionalProperties: {} - default: {} ground_truth: type: - object - 'null' additionalProperties: {} - default: {} metadata: type: - object - 'null' additionalProperties: {} - default: {} linked_event: anyOf: - type: string @@ -1921,7 +2269,10 @@ components: type: string required: - id + - inputs - history + - ground_truth + - metadata - linked_event - created_at GetDatapointsQuery: @@ -1932,18 +2283,23 @@ components: items: type: string minLength: 1 + description: List of datapoint IDs to fetch dataset_name: type: string maxLength: 200 + description: Dataset name to filter by additionalProperties: false + description: Query parameters for GET /datapoints GetDatapointParams: type: object properties: - id: + datapoint_id: type: string minLength: 1 + description: Unique identifier of the datapoint required: - - id + - datapoint_id + description: Path parameters for GET /datapoints/{datapoint_id} CreateDatapointRequest: type: object properties: @@ -1956,16 +2312,13 @@ components: items: type: object additionalProperties: {} - default: {} default: [] ground_truth: type: object additionalProperties: {} - default: {} metadata: type: object additionalProperties: {} - default: {} linked_event: type: string linked_datasets: @@ -1974,27 +2327,24 @@ components: type: string minLength: 1 default: [] + description: Request body for POST /datapoints UpdateDatapointRequest: type: object properties: inputs: type: object additionalProperties: {} - default: {} history: type: array items: type: object additionalProperties: {} - default: {} ground_truth: type: object additionalProperties: {} - default: {} metadata: type: object additionalProperties: {} - default: {} linked_event: type: string linked_datasets: @@ -2002,6 +2352,7 @@ components: items: type: string minLength: 1 + description: Request body for PUT /datapoints/{datapoint_id} UpdateDatapointParams: type: object properties: @@ -2010,6 +2361,7 @@ components: minLength: 1 required: - datapoint_id + description: Path parameters for PUT /datapoints/{datapoint_id} DeleteDatapointParams: type: object properties: @@ -2018,6 +2370,7 @@ components: minLength: 1 required: - datapoint_id + description: Path parameters for DELETE /datapoints/{datapoint_id} BatchCreateDatapointsRequest: type: object properties: @@ -2026,18 +2379,18 @@ components: items: type: string minLength: 1 + deprecated: true mapping: $ref: '#/components/schemas/DatapointMapping' + deprecated: true filters: anyOf: - type: object additionalProperties: {} - default: {} - type: array items: type: object additionalProperties: {} - default: {} dateRange: $ref: '#/components/schemas/BatchDateRange' checkState: @@ -2047,6 +2400,7 @@ components: dataset_id: type: string minLength: 1 + description: Request body for POST /datapoints/batch GetDatapointsResponse: type: object properties: @@ -2056,6 +2410,7 @@ components: $ref: '#/components/schemas/Datapoint' required: - datapoints + description: Response for GET /datapoints GetDatapointResponse: type: object properties: @@ -2065,39 +2420,29 @@ components: $ref: '#/components/schemas/Datapoint' required: - datapoint + description: Response for GET /datapoints/{datapoint_id} CreateDatapointResponse: type: object properties: inserted: type: boolean result: - type: object - properties: - insertedIds: - type: array - items: - type: string - minLength: 1 - required: - - insertedIds + $ref: '#/components/schemas/CreateDatapointResponseResult' required: - inserted - result + description: Response for POST /datapoints UpdateDatapointResponse: type: object properties: updated: type: boolean result: - type: object - properties: - modifiedCount: - type: number - required: - - modifiedCount + $ref: '#/components/schemas/UpdateDatapointResponseResult' required: - updated - result + description: Response for PUT /datapoints/{datapoint_id} DeleteDatapointResponse: type: object properties: @@ -2105,6 +2450,7 @@ components: type: boolean required: - deleted + description: Response for DELETE /datapoints/{datapoint_id} BatchCreateDatapointsResponse: type: object properties: @@ -2118,6 +2464,7 @@ components: required: - inserted - insertedIds + description: Response for POST /datapoints/batch Dataset: type: object properties: @@ -2136,7 +2483,6 @@ components: items: type: string minLength: 1 - default: [] created_at: type: string updated_at: @@ -2144,6 +2490,7 @@ components: required: - id - name + - datapoints CreateDatasetRequest: type: object properties: @@ -2151,40 +2498,82 @@ components: type: string maxLength: 200 default: Untitled Dataset + description: Name of the dataset description: type: string + description: Description of the dataset datapoints: type: array items: type: string minLength: 1 default: [] + description: Initial datapoint IDs to include + description: Request body for POST /datasets UpdateDatasetRequest: + type: object + properties: + name: + type: string + maxLength: 200 + description: New dataset name + description: + type: string + description: New dataset description + datapoints: + type: array + items: + type: string + minLength: 1 + description: Updated list of datapoint IDs + additionalProperties: false + description: Request body for PUT /datasets/{dataset_id} + LegacyUpdateDatasetRequest: type: object properties: dataset_id: type: string minLength: 1 + description: Unique identifier of the dataset to update name: type: string maxLength: 200 + description: New dataset name description: type: string + description: New dataset description datapoints: type: array items: type: string minLength: 1 + description: Updated list of datapoint IDs required: - dataset_id - DeleteDatasetQuery: + description: Request body for PUT /datasets (deprecated — use PUT /datasets/{dataset_id}) + deprecated: true + DeleteDatasetParams: + type: object + properties: + dataset_id: + type: string + minLength: 1 + description: Unique identifier of the dataset to delete + required: + - dataset_id + additionalProperties: false + description: Path parameters for DELETE /datasets/{dataset_id} + LegacyDeleteDatasetQuery: type: object properties: dataset_id: type: string minLength: 1 + description: Unique identifier of the dataset to delete required: - dataset_id + description: Query parameters for DELETE /datasets (deprecated — use DELETE /datasets/{dataset_id}) + deprecated: true AddDatapointsToDatasetRequest: type: object properties: @@ -2193,25 +2582,47 @@ components: items: type: object additionalProperties: {} - default: {} minItems: 1 + description: Array of datapoint data objects to add mapping: $ref: '#/components/schemas/DatapointMapping' + description: Field mapping for inputs, ground truth, and history required: - data - mapping + description: Request body for POST /datasets/{dataset_id}/datapoints RemoveDatapointFromDatasetParams: type: object properties: dataset_id: type: string minLength: 1 + description: Unique identifier of the dataset datapoint_id: type: string minLength: 1 + description: Unique identifier of the datapoint to remove required: - dataset_id - datapoint_id + additionalProperties: false + description: Path parameters for DELETE /datasets/{dataset_id}/datapoints/{datapoint_id} + LegacyRemoveDatapointFromDatasetParams: + type: object + properties: + dataset_id: + type: string + minLength: 1 + description: Unique identifier of the dataset + datapoint_id: + type: string + minLength: 1 + description: Unique identifier of the datapoint to remove + required: + - dataset_id + - datapoint_id + description: Path parameters for DELETE /datasets/{dataset_id}/{datapoint_id} (deprecated — use DELETE /datasets/{dataset_id}/datapoints/{datapoint_id}) + deprecated: true CreateDatasetResponse: type: object properties: @@ -2222,6 +2633,7 @@ components: required: - inserted - result + description: Response for POST /datasets InsertResult: type: object properties: @@ -2237,6 +2649,7 @@ components: $ref: '#/components/schemas/Dataset' required: - result + description: Response for PUT /datasets GetDatasetsResponse: type: object properties: @@ -2246,6 +2659,7 @@ components: $ref: '#/components/schemas/Dataset' required: - datasets + description: Response for GET /datasets DeleteDatasetResponse: type: object properties: @@ -2253,6 +2667,7 @@ components: $ref: '#/components/schemas/DeleteResult' required: - result + description: Response for DELETE /datasets DeleteResult: type: object properties: @@ -2274,6 +2689,7 @@ components: required: - inserted - datapoint_ids + description: Response for POST /datasets/{dataset_id}/datapoints RemoveDatapointResponse: type: object properties: @@ -2284,57 +2700,38 @@ components: required: - dereferenced - message - FilterFieldType: - type: string - enum: - - string - - number - - boolean - - datetime + description: Response for DELETE /datasets/{dataset_id}/datapoints/{datapoint_id} SingleFilter: type: object properties: field: type: string operator: - anyOf: - - type: string - enum: - - exists - - not exists - - is - - is not - - contains - - not contains - - type: string - enum: - - exists - - not exists - - is - - is not - - greater than - - less than - - type: string - enum: - - exists - - not exists - - is - - type: string - enum: - - exists - - not exists - - is - - is not - - after - - before - value: + type: string + enum: + - exists + - not exists + - is + - is not + - contains + - not contains + - greater than + - less than + - after + - before + value: anyOf: - type: string - type: number - type: boolean - type: 'null' type: - $ref: '#/components/schemas/FilterFieldType' + type: string + enum: + - string + - number + - boolean + - datetime required: - field - operator @@ -2344,6 +2741,46 @@ components: type: array items: $ref: '#/components/schemas/SingleFilter' + EventSearchFilter: + type: object + properties: + field: + type: string + operator: + type: string + enum: + - exists + - not exists + - is + - is not + - contains + - not contains + - greater than + - less than + - after + - before + value: + anyOf: + - type: string + - type: number + - type: boolean + - type: 'null' + type: + type: string + enum: + - string + - number + - boolean + - datetime + required: + - field + - operator + - value + additionalProperties: false + EventSearchFiltersArray: + type: array + items: + $ref: '#/components/schemas/EventSearchFilter' Event: type: object properties: @@ -2364,20 +2801,20 @@ components: type: object additionalProperties: {} feedback: - type: object - properties: - ground_truth: {} + $ref: '#/components/schemas/EventFeedback' required: - event_id - project_id - tenant additionalProperties: {} + description: Minimal event object used by evaluation and session endpoints; permissive (passthrough) LegacyEvent: type: object properties: project: type: string - description: Project name + deprecated: true + description: Project name (ignored by server — project is determined from API key scope) project_id: type: string description: Project ID @@ -2394,56 +2831,207 @@ components: - tool - chain - session + description: Type of event (model, tool, chain, or session) event_id: type: string + description: Unique event identifier session_id: type: string + description: Session this event belongs to parent_id: type: string + description: Parent event ID in the trace hierarchy children_ids: type: array items: type: string + description: Child event IDs in the trace hierarchy config: type: object additionalProperties: {} + description: Configuration used for this event inputs: type: object additionalProperties: {} + description: Input data for the event outputs: type: object additionalProperties: {} + description: Output data from the event error: type: - string - 'null' + description: Error message if the event failed start_time: type: number + description: Event start time as Unix milliseconds end_time: type: number + description: Event end time as Unix milliseconds duration: type: number + description: Event duration in milliseconds metadata: type: object additionalProperties: {} + description: Arbitrary metadata for the event feedback: type: object additionalProperties: {} + description: Feedback data associated with the event metrics: type: object additionalProperties: {} + description: Metric values computed for the event user_properties: type: object additionalProperties: {} - required: - - project + description: User properties associated with the event additionalProperties: {} description: Full event object for legacy event creation endpoints + ModelEvent: + type: object + properties: + project: + type: string + deprecated: true + description: Project name (ignored by server — project is determined from API key scope) + project_id: + type: string + description: Project ID + source: + type: string + description: Source of the event (e.g., sdk-python) + event_name: + type: string + description: Name of the event + event_type: + type: string + enum: + - model + - tool + - chain + - session + description: Type of event (model, tool, chain, or session) + event_id: + type: string + description: Unique event identifier + session_id: + type: string + description: Session this event belongs to + parent_id: + type: string + description: Parent event ID in the trace hierarchy + children_ids: + type: array + items: + type: string + description: Child event IDs in the trace hierarchy + config: + type: object + additionalProperties: {} + description: Configuration used for this event + inputs: + type: object + additionalProperties: {} + description: Input data for the event + outputs: + type: object + additionalProperties: {} + description: Output data from the event + error: + type: + - string + - 'null' + deprecated: true + description: Legacy alias — remapped to inputs.error for model events + start_time: + type: number + description: Event start time as Unix milliseconds + end_time: + type: number + description: Event end time as Unix milliseconds + duration: + type: number + deprecated: true + description: Legacy alias — remapped to inputs.duration for model events + metadata: + type: object + additionalProperties: {} + description: Arbitrary metadata for the event + feedback: + type: object + additionalProperties: {} + description: Feedback data associated with the event + metrics: + type: object + additionalProperties: {} + description: Metric values computed for the event + user_properties: + type: object + additionalProperties: {} + description: User properties associated with the event + model_name: + type: string + description: Model identifier (either this or event_name required) + model_version: + type: string + description: Model version string + model: + type: string + deprecated: true + description: Legacy alias for model_name + messages: + type: array + items: {} + deprecated: true + description: Legacy alias — remapped to inputs.messages + response: + deprecated: true + description: Legacy alias — remapped to outputs.response + provider: + type: string + deprecated: true + description: Legacy alias — remapped to inputs.provider + usage: + type: object + additionalProperties: {} + deprecated: true + description: Legacy alias — remapped to inputs.usage + cost: + deprecated: true + description: Legacy alias — remapped to inputs.cost + hyperparameters: + type: object + additionalProperties: {} + deprecated: true + description: Legacy alias — remapped to inputs.hyperparameters + template: + deprecated: true + description: Legacy alias — remapped to inputs.template + template_inputs: + deprecated: true + description: Legacy alias — remapped to inputs.template_inputs + tools: + deprecated: true + description: Legacy alias — remapped to inputs.tools + tool_choice: + deprecated: true + description: Legacy alias — remapped to inputs.tool_choice + response_format: + deprecated: true + description: Legacy alias — remapped to inputs.response_format + additionalProperties: {} + description: Model event object with model-specific fields and legacy aliases SessionProperties: type: object properties: session_name: type: string + start_time: + type: number + description: Session start time as Unix milliseconds user_properties: type: object additionalProperties: {} @@ -2452,331 +3040,298 @@ components: additionalProperties: {} additionalProperties: {} description: Session properties for batch event creation - PostEventRequest: + UpdateEventRequest: type: object properties: - event: + metadata: type: object - properties: - event_id: - type: string - project_id: - type: string - tenant: - type: string - event_name: - type: string - event_type: - type: string - metrics: - type: object - additionalProperties: {} - metadata: - type: object - additionalProperties: {} - feedback: - type: object - properties: - ground_truth: {} - required: - - event_id - - project_id - - tenant additionalProperties: {} - required: - - event - description: Request to create a new event - GetEventsLegacyRequest: - type: object - properties: - project: - type: string - description: Name of the project - filters: - allOf: - - $ref: '#/components/schemas/FiltersArray' - - description: Array of filter conditions - dateRange: + description: Metadata fields to merge into the event + feedback: type: object - properties: - $gte: - type: string - description: ISO String for start of date range - $lte: - type: string - description: ISO String for end of date range - required: - - $gte - - $lte - projections: + additionalProperties: {} + description: Feedback fields to merge into the event + metrics: + type: object + additionalProperties: {} + description: Metric values to merge into the event + outputs: + description: Output data to replace on the event (accepts objects, strings, arrays, or scalars) + config: + type: object + additionalProperties: {} + description: Configuration fields to merge into the event + user_properties: + type: object + additionalProperties: {} + description: User properties to merge into the event + duration: + type: number + description: Event duration in milliseconds + end_time: + type: number + description: Unix timestamp in milliseconds for event end + children_ids: type: array items: type: string - description: Fields to include in the response - limit: - type: number - description: Limit number of results (default 1000, max 7500) - page: - type: number - description: Page number of results (default 1) - required: - - project - - filters - description: Request body for GET /events legacy endpoint - PostModelEventRequest: + description: IDs of child events to set (must be non-empty; an empty array is ignored) + description: Request body for PUT /v1/events/{event_id} + LegacyUpdateEventRequest: type: object properties: - model_event: + event_id: + type: string + description: Event ID to update + metadata: + type: object + additionalProperties: {} + description: Metadata fields to merge into the event + feedback: + type: object + additionalProperties: {} + description: Feedback fields to merge into the event + metrics: + type: object + additionalProperties: {} + description: Metric values to merge into the event + outputs: + description: Output data to replace on the event (accepts objects, strings, arrays, or scalars) + config: + type: object + additionalProperties: {} + description: Configuration fields to merge into the event + user_properties: type: object - properties: - project: - type: string - description: Project name - project_id: - type: string - description: Project ID - source: - type: string - description: Source of the event (e.g., sdk-python) - event_name: - type: string - description: Name of the event - event_type: - type: string - enum: - - model - - tool - - chain - - session - event_id: - type: string - session_id: - type: string - parent_id: - type: string - children_ids: - type: array - items: - type: string - config: - type: object - additionalProperties: {} - inputs: - type: object - additionalProperties: {} - outputs: - type: object - additionalProperties: {} - error: - type: - - string - - 'null' - start_time: - type: number - end_time: - type: number - duration: - type: number - metadata: - type: object - additionalProperties: {} - feedback: - type: object - additionalProperties: {} - metrics: - type: object - additionalProperties: {} - user_properties: - type: object - additionalProperties: {} - required: - - project additionalProperties: {} - description: Full event object for legacy event creation endpoints + description: User properties to merge into the event + duration: + type: number + description: Event duration in milliseconds + end_time: + type: number + description: Unix timestamp in milliseconds for event end + children_ids: + type: array + items: + type: string + description: IDs of child events to set (must be non-empty; an empty array is ignored) required: - - model_event - description: Request body for POST /events/model - PostEventBatchRequest: + - event_id + description: Request to update an existing event (deprecated — use PUT /v1/events/{event_id}) + deprecated: true + PostEventRequest: type: object properties: - events: + project_id: + type: string + description: Project ID + source: + type: string + description: Source of the event (e.g., sdk-python) + event_name: + type: string + description: Name of the event + event_type: + type: string + enum: + - model + - tool + - chain + - session + description: Type of event (model, tool, chain, or session) + event_id: + type: string + description: Unique event identifier + session_id: + type: string + description: Session this event belongs to + parent_id: + type: string + description: Parent event ID in the trace hierarchy + children_ids: type: array items: - type: object - properties: - project: - type: string - description: Project name - project_id: - type: string - description: Project ID - source: - type: string - description: Source of the event (e.g., sdk-python) - event_name: - type: string - description: Name of the event - event_type: - type: string - enum: - - model - - tool - - chain - - session - event_id: - type: string - session_id: - type: string - parent_id: - type: string - children_ids: - type: array - items: - type: string - config: - type: object - additionalProperties: {} - inputs: - type: object - additionalProperties: {} - outputs: - type: object - additionalProperties: {} - error: - type: - - string - - 'null' - start_time: - type: number - end_time: - type: number - duration: - type: number - metadata: - type: object - additionalProperties: {} - feedback: - type: object - additionalProperties: {} - metrics: - type: object - additionalProperties: {} - user_properties: - type: object - additionalProperties: {} - required: - - project - additionalProperties: {} - description: Full event object for legacy event creation endpoints + type: string + description: Child event IDs in the trace hierarchy + config: + type: object + additionalProperties: {} + description: Configuration used for this event + inputs: + type: object + additionalProperties: {} + description: Input data for the event + outputs: + type: object + additionalProperties: {} + description: Output data from the event + error: + type: + - string + - 'null' + description: Error message if the event failed + start_time: + type: number + description: Event start time as Unix milliseconds + end_time: + type: number + description: Event end time as Unix milliseconds + duration: + type: number + description: Event duration in milliseconds + metadata: + type: object + additionalProperties: {} + description: Arbitrary metadata for the event + feedback: + type: object + additionalProperties: {} + description: Feedback data associated with the event + metrics: + type: object + additionalProperties: {} + description: Metric values computed for the event + user_properties: + type: object + additionalProperties: {} + description: User properties associated with the event + required: + - event_type + - inputs + additionalProperties: false + description: Request body for POST /v1/events (bare event object) + LegacyPostEventRequest: + type: object + properties: + event: + $ref: '#/components/schemas/LegacyPostEventRequestEvent' + required: + - event + description: Request body for POST /events (deprecated — use POST /v1/events) + deprecated: true + SearchEventsRequest: + type: object + properties: + filters: + $ref: '#/components/schemas/EventSearchFiltersArray' + description: Array of filter criteria to apply + dateRange: + $ref: '#/components/schemas/SearchEventsRequestDateRange' + limit: + type: integer + minimum: 1 + maximum: 1000 + description: Limit number of results (default 1000, max 1000) + page: + type: integer + minimum: 1 + description: Page number of results (default 1) + ignore_order: + type: boolean + description: 'Deprecated: accepted for SDK back-compat but treated as a no-op. Pagination requires a stable ORDER BY to produce consistent pages, and with the 1000-row cap skipping the sort is not worth the inconsistency. The route always orders by start_time DESC.' + evaluation_id: + type: string + description: Filter by evaluation/experiment run ID + additionalProperties: false + description: Request body for POST /v1/events/search + LegacyExportEventsRequest: + type: object + properties: + filters: + $ref: '#/components/schemas/FiltersArray' + description: Array of filter criteria to apply + dateRange: + $ref: '#/components/schemas/LegacyExportEventsRequestDateRange' + projections: + type: array + items: + type: string + description: Fields to include in the response + limit: + type: number + description: Limit number of results (default 1000, max 7500) + page: + type: number + description: Page number of results (default 1) + ignore_order: + type: boolean + description: If true, skip result ordering for faster queries + evaluation_id: + type: string + description: Filter by evaluation/experiment run ID + description: Request body for POST /v1/events/export + PostModelEventRequest: + type: object + properties: + model_event: + $ref: '#/components/schemas/ModelEvent' + required: + - model_event + description: Request body for POST /events/model + PostEventBatchRequest: + type: object + properties: + events: + type: array + items: + $ref: '#/components/schemas/PostEventRequest' description: Array of events to create - is_single_session: + single_session: type: boolean description: If true, all events share the same session session_properties: - type: object - properties: - session_name: - type: string - user_properties: - type: object - additionalProperties: {} - metadata: - type: object - additionalProperties: {} - additionalProperties: {} - description: Session properties for batch event creation + $ref: '#/components/schemas/SessionProperties' + required: + - events + additionalProperties: false + description: Request body for POST /v1/events/batch + LegacyPostEventBatchRequest: + type: object + properties: + events: + type: array + items: + $ref: '#/components/schemas/LegacyEvent' + description: Array of events to create + single_session: + type: boolean + description: If true, all events share the same session + is_single_session: + type: boolean + deprecated: true + description: Legacy field name for single_session (backward compatibility) + session: + $ref: '#/components/schemas/SessionProperties' + description: Alias for session_properties (backward compatibility) + deprecated: true + session_properties: + $ref: '#/components/schemas/SessionProperties' required: - events - description: Request body for POST /events/batch + description: Request body for POST /events/batch (deprecated — use POST /v1/events/batch) + deprecated: true PostModelEventBatchRequest: type: object properties: model_events: type: array items: - type: object - properties: - project: - type: string - description: Project name - project_id: - type: string - description: Project ID - source: - type: string - description: Source of the event (e.g., sdk-python) - event_name: - type: string - description: Name of the event - event_type: - type: string - enum: - - model - - tool - - chain - - session - event_id: - type: string - session_id: - type: string - parent_id: - type: string - children_ids: - type: array - items: - type: string - config: - type: object - additionalProperties: {} - inputs: - type: object - additionalProperties: {} - outputs: - type: object - additionalProperties: {} - error: - type: - - string - - 'null' - start_time: - type: number - end_time: - type: number - duration: - type: number - metadata: - type: object - additionalProperties: {} - feedback: - type: object - additionalProperties: {} - metrics: - type: object - additionalProperties: {} - user_properties: - type: object - additionalProperties: {} - required: - - project - additionalProperties: {} - description: Full event object for legacy event creation endpoints + $ref: '#/components/schemas/ModelEvent' description: Array of model events to create - is_single_session: + single_session: type: boolean description: If true, all events share the same session + is_single_session: + type: boolean + deprecated: true + description: Legacy field name for single_session (backward compatibility) + session: + $ref: '#/components/schemas/SessionProperties' + description: Alias for session_properties (backward compatibility) + deprecated: true session_properties: - type: object - properties: - session_name: - type: string - user_properties: - type: object - additionalProperties: {} - metadata: - type: object - additionalProperties: {} - additionalProperties: {} - description: Session properties for batch event creation + $ref: '#/components/schemas/SessionProperties' required: - model_events description: Request body for POST /events/model/batch @@ -2787,23 +3342,30 @@ components: $ref: '#/components/schemas/DateRange' filters: $ref: '#/components/schemas/FiltersArray' + description: Event filters to apply projections: type: array items: type: string + description: Fields to include in the response ignore_order: type: boolean + description: If true, skip result ordering for faster queries limit: type: number + description: Maximum number of results to return page: type: number + description: Page number for pagination evaluation_id: type: string + description: Filter by evaluation/experiment run ID description: Query parameters for GET /events DateRange: anyOf: - $ref: '#/components/schemas/RelativeDateRange' - $ref: '#/components/schemas/AbsoluteDateRange' + description: Date range filter RelativeDateRange: type: object properties: @@ -2826,42 +3388,6 @@ components: required: - $gte - $lte - GetEventsChartQuery: - type: object - properties: - dateRange: - $ref: '#/components/schemas/DateRange' - filters: - $ref: '#/components/schemas/FiltersArray' - metric: - type: string - groupBy: - type: string - bucket: - type: string - aggregation: - type: string - evaluation_id: - type: string - only_experiments: - type: boolean - description: Query parameters for GET /events/chart - GetEventsBySessionIdParams: - type: object - properties: - session_id: - type: string - required: - - session_id - description: Path parameters for GET /events/:session_id - DeleteEventParams: - type: object - properties: - event_id: - type: string - required: - - event_id - description: Path parameters for DELETE /events/:event_id PostEventResponse: type: object properties: @@ -2883,206 +3409,36 @@ components: required: - events - totalEvents - GetEventsLegacyResponse: + description: Response for GET /events + ExportEventsResponse: type: object properties: events: type: array items: - type: object - properties: - project: - type: string - description: Project name - project_id: - type: string - description: Project ID - source: - type: string - description: Source of the event (e.g., sdk-python) - event_name: - type: string - description: Name of the event - event_type: - type: string - enum: - - model - - tool - - chain - - session - event_id: - type: string - session_id: - type: string - parent_id: - type: string - children_ids: - type: array - items: - type: string - config: - type: object - additionalProperties: {} - inputs: - type: object - additionalProperties: {} - outputs: - type: object - additionalProperties: {} - error: - type: - - string - - 'null' - start_time: - type: number - end_time: - type: number - duration: - type: number - metadata: - type: object - additionalProperties: {} - feedback: - type: object - additionalProperties: {} - metrics: - type: object - additionalProperties: {} - user_properties: - type: object - additionalProperties: {} - required: - - project - additionalProperties: {} - description: Full event object for legacy event creation endpoints - totalEvents: + $ref: '#/components/schemas/LegacyEvent' + count: type: number required: - events - - totalEvents - description: Response for GET /events legacy endpoint - GetEventsChartResponse: + - count + description: Response for POST /v1/events/search and POST /v1/events/export + PostEventBatchResponse: type: object properties: - events: + event_ids: type: array - items: {} - totalEvents: - type: number + items: + type: string + session_id: + type: string + success: + type: boolean required: - - events - - totalEvents - description: Chart data response for events - GetEventsBySessionIdResponse: - type: object - properties: - event_id: - type: string - event_type: - type: string - enum: - - session - - model - - chain - - tool - event_name: - type: string - children: - type: array - items: {} - start_time: - type: number - end_time: - type: number - duration: - type: number - metadata: - type: object - properties: - num_events: - type: number - num_model_events: - type: number - has_feedback: - type: boolean - cost: - type: number - total_tokens: - type: number - prompt_tokens: - type: number - completion_tokens: - type: number - scope: - type: object - properties: - name: - type: string - additionalProperties: {} - parent_id: - type: string - session_id: - type: string - children_ids: - type: array - items: - type: string - config: {} - inputs: {} - outputs: {} - error: - type: string - source: - type: string - user_properties: {} - metrics: {} - feedback: {} - org_id: - type: string - workspace_id: - type: string - project_id: - type: string - required: - - event_id - - event_type - - event_name - - children - - start_time - - end_time - - duration - - metadata - - org_id - - project_id - additionalProperties: {} - description: Event node in session tree with nested children - DeleteEventResponse: - type: object - properties: - success: - type: boolean - deleted: - type: string - required: - - success - - deleted - description: Response for DELETE /events/:event_id - PostEventBatchResponse: - type: object - properties: - event_ids: - type: array - items: - type: string - session_id: - type: string - success: - type: boolean - required: - - event_ids - - success - description: Response for POST /events/batch - Pagination: + - event_ids + - success + description: Response for POST /events/batch + Pagination: type: object properties: page: @@ -3164,13 +3520,9 @@ components: type: number description: Individual metric values across datapoints passing_range: - allOf: - - $ref: '#/components/schemas/PassingRange' - - description: Threshold range for pass/fail determination + $ref: '#/components/schemas/PassingRange' datapoints: - allOf: - - $ref: '#/components/schemas/MetricDatapoints' - - description: Per-metric pass/fail breakdown by datapoint + $ref: '#/components/schemas/MetricDatapoints' required: - metric_name DatapointResult: @@ -3251,23 +3603,22 @@ components: metadata: type: object additionalProperties: {} - default: {} results: type: object additionalProperties: {} - default: {} + metrics: + type: object + additionalProperties: + type: number event_ids: type: array items: type: string - default: [] configuration: type: object additionalProperties: {} - default: {} is_active: type: boolean - default: true created_at: anyOf: - type: string @@ -3290,6 +3641,11 @@ components: required: - id - run_id + - metadata + - results + - event_ids + - configuration + - is_active - created_at - scope_type - scope_id @@ -3327,6 +3683,47 @@ components: type: number required: - metric_name + EventMetricData: + type: object + properties: + event_name: + type: string + event_type: + type: string + session_id: + type: string + metadata: + $ref: '#/components/schemas/EventMetricDataMetadata' + metrics: + type: object + additionalProperties: + anyOf: + - type: number + - type: boolean + description: Metric name → value, merged from numeric, float, and boolean ClickHouse columns + required: + - event_name + - event_type + - session_id + - metadata + - metrics + ComparableEvent: + type: object + properties: + datapoint_id: + type: string + event_1: + type: object + additionalProperties: {} + description: Full event record from the first run + event_2: + type: object + additionalProperties: {} + description: Full event record from the second run + required: + - datapoint_id + - event_1 + - event_2 PostExperimentRunRequest: type: object properties: @@ -3348,11 +3745,9 @@ components: metadata: type: object additionalProperties: {} - default: {} results: type: object additionalProperties: {} - default: {} dataset_id: type: - string @@ -3365,7 +3760,6 @@ components: configuration: type: object additionalProperties: {} - default: {} evaluators: type: array items: {} @@ -3385,6 +3779,7 @@ components: type: object additionalProperties: $ref: '#/components/schemas/PassingRange' + description: Request body for POST /runs PutExperimentRunRequest: type: object properties: @@ -3403,11 +3798,9 @@ components: metadata: type: object additionalProperties: {} - default: {} results: type: object additionalProperties: {} - default: {} event_ids: type: array items: @@ -3415,7 +3808,6 @@ components: configuration: type: object additionalProperties: {} - default: {} evaluators: type: array items: {} @@ -3432,27 +3824,33 @@ components: type: object additionalProperties: $ref: '#/components/schemas/PassingRange' + description: Request body for PUT /runs/{run_id} GetExperimentRunsQuery: type: object properties: dataset_id: type: string minLength: 1 + description: Filter by dataset ID page: type: integer minimum: 1 default: 1 + description: Page number for pagination limit: type: integer minimum: 1 maximum: 100 default: 20 + description: Number of results per page (max 100) run_ids: type: array items: type: string + description: Filter by specific run IDs name: type: string + description: Filter by run name status: type: string enum: @@ -3461,10 +3859,12 @@ components: - failed - cancelled - running + description: Filter by run status dateRange: anyOf: - type: string - $ref: '#/components/schemas/AbsoluteDateRange' + description: Filter by date range sort_by: type: string enum: @@ -3473,12 +3873,15 @@ components: - name - status default: created_at + description: Field to sort results by sort_order: type: string enum: - asc - desc default: desc + description: Sort order + description: Query parameters for GET /runs GetExperimentRunParams: type: object properties: @@ -3486,76 +3889,181 @@ components: type: string required: - run_id + description: Path parameters for GET /runs/{run_id} GetExperimentRunMetricsQuery: type: object properties: dateRange: type: string + description: Date range filter as JSON string filters: anyOf: - type: string - type: array items: {} + description: Filters to apply to metrics + description: Query parameters for GET /runs/{run_id}/metrics GetExperimentRunResultQuery: type: object properties: aggregate_function: type: string default: average + description: 'Aggregation function to apply (default: average)' filters: anyOf: - type: string - type: array items: {} + description: Filters to apply to results + description: Query parameters for GET /runs/{run_id}/result GetExperimentRunCompareParams: type: object properties: new_run_id: type: string + description: The new run ID to compare old_run_id: type: string + description: The old run ID to compare against required: - new_run_id - old_run_id + additionalProperties: false + description: Path parameters for GET /runs/{new_run_id}/compare/{old_run_id} GetExperimentRunCompareQuery: + type: object + properties: + aggregate_function: + type: string + enum: + - average + - min + - max + - median + - p95 + - p99 + - p90 + - sum + - count + default: average + description: 'Aggregation function to apply (default: average)' + filters: + anyOf: + - type: string + - type: array + items: {} + description: Filters to apply to comparison + additionalProperties: false + description: Query parameters for GET /runs/{new_run_id}/compare/{old_run_id} + LegacyGetExperimentRunCompareParams: + type: object + properties: + new_run_id: + type: string + description: The new run ID to compare + old_run_id: + type: string + description: The old run ID to compare against + required: + - new_run_id + - old_run_id + description: Path parameters for GET /runs/{new_run_id}/compare-with/{old_run_id} (deprecated — use GET /runs/{new_run_id}/compare/{old_run_id}) + deprecated: true + LegacyGetExperimentRunCompareQuery: type: object properties: aggregate_function: type: string default: average + description: 'Aggregation function to apply (default: average)' filters: anyOf: - type: string - type: array items: {} + description: Filters to apply to comparison + description: Query parameters for GET /runs/{new_run_id}/compare-with/{old_run_id} (deprecated — use GET /runs/{new_run_id}/compare/{old_run_id}) + deprecated: true + GetExperimentRunCompareEventsParams: + type: object + properties: + new_run_id: + type: string + description: The new run ID to compare + old_run_id: + type: string + description: The old run ID to compare against + required: + - new_run_id + - old_run_id + additionalProperties: false + description: Path parameters for GET /runs/{new_run_id}/compare/{old_run_id}/events GetExperimentRunCompareEventsQuery: + type: object + properties: + event_name: + type: string + description: Filter by event name + event_type: + type: string + description: Filter by event type + filter: + anyOf: + - type: string + - type: object + additionalProperties: {} + description: Additional filter criteria + limit: + type: integer + exclusiveMinimum: 0 + maximum: 1000 + default: 1000 + description: Maximum number of results (max 1000) + page: + type: integer + exclusiveMinimum: 0 + default: 1 + description: Page number for pagination + additionalProperties: false + description: Query parameters for GET /runs/{new_run_id}/compare/{old_run_id}/events + LegacyGetExperimentRunCompareEventsQuery: type: object properties: run_id_1: type: string + description: First run ID to compare run_id_2: type: string + description: Second run ID to compare event_name: type: string + description: Filter by event name event_type: type: string + description: Filter by event type filter: anyOf: - type: string - type: object additionalProperties: {} + description: Additional filter criteria limit: type: integer exclusiveMinimum: 0 maximum: 1000 default: 1000 + description: Maximum number of results (max 1000) page: type: integer exclusiveMinimum: 0 default: 1 + description: Page number for pagination required: - run_id_1 - run_id_2 + description: Query parameters for GET /runs/compare/events (deprecated — use GET /runs/{new_run_id}/compare/{old_run_id}/events) + deprecated: true DeleteExperimentRunParams: type: object properties: @@ -3563,35 +4071,68 @@ components: type: string required: - run_id - GetExperimentRunsSchemaQuery: + description: Path parameters for DELETE /runs/{run_id} + LegacyGetEventsSchemaQuery: type: object properties: dateRange: anyOf: - type: string - $ref: '#/components/schemas/AbsoluteDateRange' + description: Date range to filter schema by evaluation_id: type: string + description: Filter by evaluation/run ID + description: Query parameters for GET /events/schema (deprecated — use GET /runs/{run_id}/schema or GET /runs/schema) + deprecated: true + GetRunSchemaQuery: + type: object + properties: + dateRange: + anyOf: + - type: string + - $ref: '#/components/schemas/AbsoluteDateRange' + description: Date range to filter schema by + additionalProperties: false + description: Query parameters for GET /runs/{run_id}/schema + GetRunsSchemaQuery: + type: object + properties: + dateRange: + anyOf: + - type: string + - $ref: '#/components/schemas/AbsoluteDateRange' + description: Date range to filter schema by + additionalProperties: false + description: Query parameters for GET /runs/schema PostExperimentRunResponse: type: object properties: - evaluation: {} + evaluation: + $ref: '#/components/schemas/ExperimentRunObject' run_id: type: string required: + - evaluation - run_id + description: Response for POST /runs PutExperimentRunResponse: type: object properties: - evaluation: {} + evaluation: + $ref: '#/components/schemas/ExperimentRunObject' warning: type: string + required: + - evaluation + description: Response for PUT /runs/{run_id} GetExperimentRunsResponse: type: object properties: evaluations: type: array - items: {} + items: + $ref: '#/components/schemas/ExperimentRunObject' pagination: $ref: '#/components/schemas/Pagination' metrics: @@ -3602,10 +4143,15 @@ components: - evaluations - pagination - metrics + description: Response for GET /runs GetExperimentRunResponse: type: object properties: - evaluation: {} + evaluation: + $ref: '#/components/schemas/ExperimentRunObject' + required: + - evaluation + description: Response for GET /runs/{run_id} GetExperimentRunResultResponse: type: object properties: @@ -3644,7 +4190,7 @@ components: - datapoints - event_details - run_object - description: Evaluation summary for an experiment run including pass/fail status, metrics, and datapoints + description: 'Evaluation summary for an experiment run: pass/fail results, metric aggregations, per-datapoint results, event details, and the experiment run object.' GetExperimentRunCompareResponse: type: object properties: @@ -3671,7 +4217,21 @@ components: - old_run - new_run description: Comparison between two experiment runs including metrics, common datapoints, and event details - GetExperimentRunsSchemaResponse: + GetExperimentRunMetricsResponse: + type: object + properties: + events: + type: array + items: + $ref: '#/components/schemas/EventMetricData' + totalEvents: + type: integer + description: Total number of events matching the query + required: + - events + - totalEvents + description: Response for GET /runs/{run_id}/metrics + GetEventsSchemaResponse: type: object properties: fields: @@ -3692,6 +4252,21 @@ components: - fields - datasets - mappings + description: Response for GET /events/schema, GET /runs/{run_id}/schema, and GET /runs/schema + GetExperimentCompareEventsResponse: + type: object + properties: + events: + type: array + items: + $ref: '#/components/schemas/ComparableEvent' + totalEvents: + type: integer + description: Total number of events matching the comparison query + required: + - events + - totalEvents + description: Response for GET /runs/{new_run_id}/compare/{old_run_id}/events DeleteExperimentRunResponse: type: object properties: @@ -3702,6 +4277,7 @@ components: required: - id - deleted + description: Response for DELETE /runs/{run_id} MetricItem: type: object properties: @@ -3719,8 +4295,9 @@ components: type: string minLength: 1 description: - type: string - default: '' + type: + - string + - 'null' return_type: type: string enum: @@ -3728,18 +4305,14 @@ components: - boolean - string - categorical - default: float enabled_in_prod: type: boolean - default: false needs_ground_truth: type: boolean - default: false sampling_percentage: type: number minimum: 0 maximum: 100 - default: 100 model_provider: type: - string @@ -3778,55 +4351,17 @@ components: - array - 'null' items: - type: object - properties: - category: - type: string - maxLength: 200 - score: - type: - - number - - 'null' - required: - - category - - score - additionalProperties: false + $ref: '#/components/schemas/MetricItemCategoriesItem' minItems: 1 child_metrics: type: - array - 'null' items: - type: object - properties: - id: - type: string - minLength: 1 - name: - type: string - maxLength: 200 - weight: - type: number - scale: - type: - - integer - - 'null' - exclusiveMinimum: 0 - required: - - name - - weight - additionalProperties: false + $ref: '#/components/schemas/MetricItemChildMetricsItem' minItems: 1 filters: - type: object - properties: - filterArray: - $ref: '#/components/schemas/FiltersArray' - default: - filterArray: &ref_0 [] - required: - - filterArray - additionalProperties: false + $ref: '#/components/schemas/MetricItemFilters' id: type: string minLength: 1 @@ -3842,6 +4377,12 @@ components: - name - type - criteria + - description + - return_type + - enabled_in_prod + - needs_ground_truth + - sampling_percentage + - filters - id - created_at - updated_at @@ -3922,60 +4463,23 @@ components: - array - 'null' items: - type: object - properties: - category: - type: string - maxLength: 200 - score: - type: - - number - - 'null' - required: - - category - - score - additionalProperties: false + $ref: '#/components/schemas/CreateMetricRequestCategoriesItem' minItems: 1 child_metrics: type: - array - 'null' items: - type: object - properties: - id: - type: string - minLength: 1 - name: - type: string - maxLength: 200 - weight: - type: number - scale: - type: - - integer - - 'null' - exclusiveMinimum: 0 - required: - - name - - weight - additionalProperties: false + $ref: '#/components/schemas/CreateMetricRequestChildMetricsItem' minItems: 1 filters: - type: object - properties: - filterArray: - $ref: '#/components/schemas/FiltersArray' - default: - filterArray: *ref_0 - required: - - filterArray - additionalProperties: false + $ref: '#/components/schemas/CreateMetricRequestFilters' required: - name - type - criteria additionalProperties: false + description: Request body for POST /metrics UpdateMetricRequest: type: object properties: @@ -3993,8 +4497,84 @@ components: type: string minLength: 1 description: + type: + - string + - 'null' + return_type: type: string - default: '' + enum: + - float + - boolean + - string + - categorical + enabled_in_prod: + type: boolean + needs_ground_truth: + type: boolean + sampling_percentage: + type: number + minimum: 0 + maximum: 100 + model_provider: + type: + - string + - 'null' + model_name: + type: + - string + - 'null' + scale: + type: + - integer + - 'null' + exclusiveMinimum: 0 + threshold: + $ref: '#/components/schemas/UpdateMetricRequestThreshold' + categories: + type: array + items: + $ref: '#/components/schemas/UpdateMetricRequestCategoriesItem' + minItems: 1 + child_metrics: + type: array + items: + $ref: '#/components/schemas/UpdateMetricRequestChildMetricsItem' + minItems: 1 + filters: + $ref: '#/components/schemas/UpdateMetricRequestFilters' + additionalProperties: false + description: Request body for PUT /metrics/{metric_id} + UpdateMetricParams: + type: object + properties: + metric_id: + type: string + minLength: 1 + description: Unique identifier of the metric to update + required: + - metric_id + additionalProperties: false + description: Path parameters for PUT /metrics/{metric_id} + LegacyUpdateMetricRequest: + type: object + properties: + name: + type: string + maxLength: 200 + type: + type: string + enum: + - PYTHON + - LLM + - HUMAN + - COMPOSITE + criteria: + type: string + minLength: 1 + description: + type: + - string + - 'null' return_type: type: string enum: @@ -4002,18 +4582,14 @@ components: - boolean - string - categorical - default: float enabled_in_prod: type: boolean - default: false needs_ground_truth: type: boolean - default: false sampling_percentage: type: number minimum: 0 maximum: 100 - default: 100 model_provider: type: - string @@ -4052,217 +4628,92 @@ components: - array - 'null' items: - type: object - properties: - category: - type: string - maxLength: 200 - score: - type: - - number - - 'null' - required: - - category - - score - additionalProperties: false + $ref: '#/components/schemas/LegacyUpdateMetricRequestCategoriesItem' minItems: 1 child_metrics: type: - array - 'null' items: - type: object - properties: - id: - type: string - minLength: 1 - name: - type: string - maxLength: 200 - weight: - type: number - scale: - type: - - integer - - 'null' - exclusiveMinimum: 0 - required: - - name - - weight - additionalProperties: false + $ref: '#/components/schemas/LegacyUpdateMetricRequestChildMetricsItem' minItems: 1 filters: - type: object - properties: - filterArray: - $ref: '#/components/schemas/FiltersArray' - default: - filterArray: *ref_0 - required: - - filterArray - additionalProperties: false + $ref: '#/components/schemas/LegacyUpdateMetricRequestFilters' id: type: string minLength: 1 required: - id additionalProperties: false + description: Request body for PUT /metrics (deprecated — use PUT /metrics/{metric_id}) + deprecated: true GetMetricsQuery: type: object properties: type: type: string + description: Filter metrics by type id: type: string minLength: 1 - DeleteMetricQuery: + description: Filter by metric ID + description: Query parameters for GET /metrics + DeleteMetricParams: + type: object + properties: + metric_id: + type: string + minLength: 1 + description: Unique identifier of the metric to delete + required: + - metric_id + additionalProperties: false + description: Path parameters for DELETE /metrics/{metric_id} + LegacyDeleteMetricQuery: type: object properties: metric_id: type: string minLength: 1 + description: Unique identifier of the metric to delete required: - metric_id + description: Query parameters for DELETE /metrics (deprecated — use DELETE /metrics/{metric_id}) + deprecated: true RunMetricRequest: type: object properties: metric: - type: object - properties: - name: - type: string - maxLength: 200 - type: - type: string - enum: - - PYTHON - - LLM - - HUMAN - - COMPOSITE - criteria: - type: string - minLength: 1 - description: - type: string - default: '' - return_type: - type: string - enum: - - float - - boolean - - string - - categorical - default: float - enabled_in_prod: - type: boolean - default: false - needs_ground_truth: - type: boolean - default: false - sampling_percentage: - type: number - minimum: 0 - maximum: 100 - default: 100 - model_provider: - type: - - string - - 'null' - model_name: - type: - - string - - 'null' - scale: - type: - - integer - - 'null' - exclusiveMinimum: 0 - threshold: - type: - - object - - 'null' - properties: - min: - type: number - max: - type: number - pass_when: - anyOf: - - type: boolean - - type: number - passing_categories: - type: array - items: - type: string - maxLength: 200 - minItems: 1 - additionalProperties: false - categories: - type: - - array - - 'null' - items: - type: object - properties: - category: - type: string - maxLength: 200 - score: - type: - - number - - 'null' - required: - - category - - score - additionalProperties: false - minItems: 1 - child_metrics: - type: - - array - - 'null' - items: - type: object - properties: - id: - type: string - minLength: 1 - name: - type: string - maxLength: 200 - weight: - type: number - scale: - type: - - integer - - 'null' - exclusiveMinimum: 0 - required: - - name - - weight - additionalProperties: false - minItems: 1 - filters: - type: object - properties: - filterArray: - $ref: '#/components/schemas/FiltersArray' - default: - filterArray: *ref_0 - required: - - filterArray - additionalProperties: false - required: - - name - - type - - criteria - additionalProperties: false - event: {} + $ref: '#/components/schemas/RunMetricRequestMetric' + event: + $ref: '#/components/schemas/RunMetricRequestEvent' + required: + - metric + - event + additionalProperties: false + description: Request body for POST /metrics/run + LegacyRunMetricRequest: + type: object + properties: + metric: + $ref: '#/components/schemas/LegacyRunMetricRequestMetric' + event: + $ref: '#/components/schemas/LegacyRunMetricRequestEvent' required: - metric + - event + description: Request body for POST /metrics/run_metric (deprecated — use POST /metrics/run) + deprecated: true GetMetricsResponse: - type: array - items: - $ref: '#/components/schemas/MetricItem' + type: object + properties: + metrics: + type: array + items: + $ref: '#/components/schemas/MetricItem' + required: + - metrics + description: Response for GET /metrics CreateMetricResponse: type: object properties: @@ -4274,6 +4725,7 @@ components: required: - inserted - metric_id + description: Response for POST /metrics UpdateMetricResponse: type: object properties: @@ -4281,6 +4733,7 @@ components: type: boolean required: - updated + description: Response for PUT /metrics DeleteMetricResponse: type: object properties: @@ -4288,7 +4741,30 @@ components: type: boolean required: - deleted - RunMetricResponse: {} + description: Response for DELETE /metrics/{metric_id} + RunMetricResponse: + type: object + properties: + success: + type: boolean + loading: + type: boolean + result: + anyOf: + - type: boolean + - type: number + - type: string + - type: 'null' + explanation: + type: + - string + - 'null' + required: + - success + - loading + - result + - explanation + description: Response for POST /metrics/run ProjectItem: type: object properties: @@ -4403,195 +4879,328 @@ components: - org_id additionalProperties: {} description: Created project - PostSessionRequest: + AnnotationQueue: type: object properties: - event_id: + name: type: string - project_id: + maxLength: 200 + description: type: string - tenant: + maxLength: 1000 + filters: + $ref: '#/components/schemas/AnnotationQueueFilters' + enabled: + type: boolean + id: type: string - event_name: + minLength: 1 + scope_id: type: string - event_type: + scope_type: type: string - metrics: - type: object - additionalProperties: {} - metadata: - type: object - additionalProperties: {} - feedback: - type: object - properties: - ground_truth: {} - required: - - event_id - - project_id - - tenant - additionalProperties: {} - PostSessionStartResponse: - type: object - properties: - event_id: - type: - - string - - 'null' - session_id: - type: - - string - - 'null' - parent_id: + enum: + - system + - controlplane + - dataplane + - org + - workspace + - project + is_active: + type: boolean + created_at: + type: string + format: date-time + updated_at: type: - string - 'null' - children_ids: + format: date-time + required: + - name + - description + - filters + - enabled + - id + - scope_id + - scope_type + - is_active + - created_at + - updated_at + BaseAnnotationQueue: + type: object + properties: + name: + type: string + maxLength: 200 + description: + type: string + maxLength: 1000 + filters: + $ref: '#/components/schemas/BaseAnnotationQueueFilters' + enabled: + type: boolean + required: + - name + - description + - filters + - enabled + CreateAnnotationQueueRequest: + type: object + properties: + name: + type: string + maxLength: 200 + description: + type: string + maxLength: 1000 + default: '' + filters: + $ref: '#/components/schemas/CreateAnnotationQueueRequestFilters' + enabled: + type: boolean + default: true + event_ids: type: array items: type: string default: [] - event_type: - type: - - string - - 'null' - event_name: - type: - - string - - 'null' - config: {} - inputs: {} - outputs: {} - error: - type: - - string - - 'null' - source: - type: - - string - - 'null' - duration: - type: - - number - - 'null' - user_properties: {} - metrics: {} - feedback: {} - metadata: {} - org_id: + required: + - name + UpdateAnnotationQueueRequest: + type: object + properties: + name: type: string - workspace_id: + maxLength: 200 + description: type: string - project_id: + maxLength: 1000 + filters: + $ref: '#/components/schemas/UpdateAnnotationQueueRequestFilters' + enabled: + type: boolean + id: type: string - start_time: + minLength: 1 + add_event_ids: + type: array + items: + type: string + remove_event_ids: + type: array + items: + type: string + required: + - id + GetAnnotationQueuesQuery: + type: object + properties: + enabled: type: - - number + - boolean - 'null' - end_time: + CreateAnnotationQueueResponse: + type: object + properties: + queue: + $ref: '#/components/schemas/CreateAnnotationQueueResponseQueue' + message: + type: string + required: + - queue + - message + UpdateAnnotationQueueResponse: + type: object + properties: + queue: + $ref: '#/components/schemas/UpdateAnnotationQueueResponseQueue' + message: + type: string + required: + - queue + - message + GetAnnotationQueuesResponse: + type: object + properties: + queues: + type: array + items: + $ref: '#/components/schemas/GetAnnotationQueuesResponseQueuesItem' + required: + - queues + GetAnnotationQueueByIdResponse: + type: object + properties: + name: + type: string + maxLength: 200 + description: + type: string + maxLength: 1000 + filters: + $ref: '#/components/schemas/GetAnnotationQueueByIdResponseFilters' + enabled: + type: boolean + id: + type: string + minLength: 1 + scope_id: + type: string + scope_type: + type: string + enum: + - system + - controlplane + - dataplane + - org + - workspace + - project + is_active: + type: boolean + created_at: + type: string + format: date-time + updated_at: type: - - number + - string - 'null' + format: date-time required: - - org_id - - workspace_id - - project_id - additionalProperties: {} - description: Full session event object returned after starting a new session - GetSessionResponse: + - name + - description + - filters + - enabled + - id + - scope_id + - scope_type + - is_active + - created_at + - updated_at + DeleteAnnotationQueueResponse: + type: object + properties: + message: + type: string + required: + - message + PostSessionRequest: type: object properties: event_id: type: string - event_type: + project_id: + type: string + tenant: type: string - enum: - - session - - model - - chain - - tool event_name: type: string - children: - type: array - items: {} - start_time: - type: number - end_time: - type: number - duration: - type: number + event_type: + type: string + metrics: + type: object + additionalProperties: {} metadata: type: object - properties: - num_events: - type: number - num_model_events: - type: number - has_feedback: - type: boolean - cost: - type: number - total_tokens: - type: number - prompt_tokens: - type: number - completion_tokens: - type: number - scope: - type: object - properties: - name: - type: string additionalProperties: {} - parent_id: + feedback: + $ref: '#/components/schemas/PostSessionRequestFeedback' + required: + - event_id + - project_id + - tenant + additionalProperties: {} + description: Minimal event object used by evaluation and session endpoints; permissive (passthrough) + StartSessionRequest: + type: object + properties: + session: + $ref: '#/components/schemas/StartSessionRequestSession' + required: + - session + description: Request to start a new session + AddSessionTracesRequest: + type: object + properties: + logs: + type: array + items: + $ref: '#/components/schemas/LegacyEvent' + required: + - logs + description: Request to add traces to a session + PostSessionStartResponse: + type: object + properties: + event_id: type: string session_id: type: string + parent_id: + type: + - string + - 'null' children_ids: type: array items: type: string + event_type: + type: + - string + - 'null' + event_name: + type: + - string + - 'null' config: {} inputs: {} outputs: {} error: - type: string + type: + - string + - 'null' source: - type: string + type: + - string + - 'null' + duration: + type: + - number + - 'null' user_properties: {} metrics: {} feedback: {} + metadata: {} org_id: type: string workspace_id: type: string project_id: type: string + start_time: + type: + - number + - 'null' + end_time: + type: + - number + - 'null' required: - event_id - - event_type - - event_name - - children - - start_time - - end_time - - duration - - metadata + - session_id - org_id + - workspace_id - project_id additionalProperties: {} - description: Event node in session tree with nested children - DeleteSessionResponse: + description: Full session event object returned after starting a new session + SessionTracesResponse: type: object properties: success: type: boolean - deleted: - type: string required: - success - - deleted - description: Confirmation of session deletion + description: Response from adding traces to a session TODOSchema: type: object properties: @@ -4601,6 +5210,952 @@ components: required: - message description: 'TODO: This is a placeholder schema. Proper Zod schemas need to be created in @hive-kube/iso-core-ts for: Sessions, Events, Projects, and Experiment comparison/result endpoints.' + CreateDatapointResponseResult: + type: object + properties: + insertedIds: + type: array + items: + type: string + minLength: 1 + required: + - insertedIds + UpdateDatapointResponseResult: + type: object + properties: + modifiedCount: + type: number + required: + - modifiedCount + EventFeedback: + type: object + properties: + ground_truth: {} + LegacyPostEventRequestEvent: + type: object + properties: + project: + type: string + deprecated: true + description: Project name (ignored by server — project is determined from API key scope) + project_id: + type: string + description: Project ID + source: + type: string + description: Source of the event (e.g., sdk-python) + event_name: + type: string + description: Name of the event + event_type: + type: string + enum: + - model + - tool + - chain + - session + event_id: + type: string + description: Unique event identifier + session_id: + type: string + description: Session this event belongs to + parent_id: + type: string + description: Parent event ID in the trace hierarchy + children_ids: + type: array + items: + type: string + description: Child event IDs in the trace hierarchy + config: + type: object + additionalProperties: {} + description: Configuration used for this event + inputs: + type: object + additionalProperties: {} + outputs: + type: object + additionalProperties: {} + description: Output data from the event + error: + type: + - string + - 'null' + description: Error message if the event failed + start_time: + type: number + description: Event start time as Unix milliseconds + end_time: + type: number + description: Event end time as Unix milliseconds + duration: + type: number + description: Event duration in milliseconds + metadata: + type: object + additionalProperties: {} + description: Arbitrary metadata for the event + feedback: + type: object + additionalProperties: {} + description: Feedback data associated with the event + metrics: + type: object + additionalProperties: {} + description: Metric values computed for the event + user_properties: + type: object + additionalProperties: {} + description: User properties associated with the event + required: + - event_type + - inputs + additionalProperties: {} + description: Full event object for legacy event creation endpoints + SearchEventsRequestDateRange: + type: object + properties: + start_time: + type: string + format: date-time + description: ISO 8601 timestamp for start of date range (inclusive) + end_time: + type: string + format: date-time + description: ISO 8601 timestamp for end of date range (inclusive) + required: + - start_time + - end_time + additionalProperties: false + LegacyExportEventsRequestDateRange: + type: object + properties: + $gte: + type: string + description: ISO String for start of date range + $lte: + type: string + description: ISO String for end of date range + required: + - $gte + - $lte + EventMetricDataMetadata: + type: object + properties: + datapoint_id: + type: string + MetricItemCategoriesItem: + type: object + properties: + category: + type: string + maxLength: 200 + score: + type: + - number + - 'null' + required: + - category + - score + additionalProperties: false + MetricItemChildMetricsItem: + type: object + properties: + id: + type: string + minLength: 1 + name: + type: string + maxLength: 200 + weight: + type: number + scale: + type: + - integer + - 'null' + exclusiveMinimum: 0 + required: + - name + - weight + additionalProperties: false + MetricItemFilters: + type: object + properties: + filterArray: + $ref: '#/components/schemas/FiltersArray' + required: + - filterArray + additionalProperties: false + CreateMetricRequestCategoriesItem: + type: object + properties: + category: + type: string + maxLength: 200 + score: + type: + - number + - 'null' + required: + - category + - score + additionalProperties: false + CreateMetricRequestChildMetricsItem: + type: object + properties: + id: + type: string + minLength: 1 + name: + type: string + maxLength: 200 + weight: + type: number + scale: + type: + - integer + - 'null' + exclusiveMinimum: 0 + required: + - name + - weight + additionalProperties: false + CreateMetricRequestFilters: + type: object + properties: + filterArray: + $ref: '#/components/schemas/FiltersArray' + default: + filterArray: [] + required: + - filterArray + additionalProperties: false + UpdateMetricRequestThreshold: + type: object + properties: + min: + type: number + max: + type: number + pass_when: + anyOf: + - type: boolean + - type: number + passing_categories: + type: array + items: + type: string + maxLength: 200 + minItems: 1 + additionalProperties: false + UpdateMetricRequestCategoriesItem: + type: object + properties: + category: + type: string + maxLength: 200 + score: + type: + - number + - 'null' + required: + - category + - score + additionalProperties: false + UpdateMetricRequestChildMetricsItem: + type: object + properties: + id: + type: string + minLength: 1 + name: + type: string + maxLength: 200 + weight: + type: number + scale: + type: + - integer + - 'null' + exclusiveMinimum: 0 + required: + - name + - weight + additionalProperties: false + UpdateMetricRequestFilters: + type: object + properties: + filterArray: + $ref: '#/components/schemas/FiltersArray' + required: + - filterArray + additionalProperties: false + LegacyUpdateMetricRequestCategoriesItem: + type: object + properties: + category: + type: string + maxLength: 200 + score: + type: + - number + - 'null' + required: + - category + - score + additionalProperties: false + LegacyUpdateMetricRequestChildMetricsItem: + type: object + properties: + id: + type: string + minLength: 1 + name: + type: string + maxLength: 200 + weight: + type: number + scale: + type: + - integer + - 'null' + exclusiveMinimum: 0 + required: + - name + - weight + additionalProperties: false + LegacyUpdateMetricRequestFilters: + type: object + properties: + filterArray: + $ref: '#/components/schemas/FiltersArray' + required: + - filterArray + additionalProperties: false + RunMetricRequestMetricCategoriesItem: + type: object + properties: + category: + type: string + maxLength: 200 + score: + type: + - number + - 'null' + required: + - category + - score + additionalProperties: false + RunMetricRequestMetricChildMetricsItem: + type: object + properties: + id: + type: string + minLength: 1 + name: + type: string + maxLength: 200 + weight: + type: number + scale: + type: + - integer + - 'null' + exclusiveMinimum: 0 + required: + - name + - weight + additionalProperties: false + RunMetricRequestMetricFilters: + type: object + properties: + filterArray: + $ref: '#/components/schemas/FiltersArray' + default: + filterArray: [] + required: + - filterArray + additionalProperties: false + RunMetricRequestMetric: + type: object + properties: + name: + type: string + maxLength: 200 + type: + type: string + enum: + - LLM + - PYTHON + criteria: + type: string + minLength: 1 + description: + type: string + default: '' + return_type: + type: string + enum: + - float + - boolean + - string + - categorical + default: float + enabled_in_prod: + type: boolean + default: false + needs_ground_truth: + type: boolean + default: false + sampling_percentage: + type: number + minimum: 0 + maximum: 100 + default: 100 + model_provider: + type: + - string + - 'null' + model_name: + type: + - string + - 'null' + scale: + type: + - integer + - 'null' + exclusiveMinimum: 0 + threshold: + type: + - object + - 'null' + properties: + min: + type: number + max: + type: number + pass_when: + anyOf: + - type: boolean + - type: number + passing_categories: + type: array + items: + type: string + maxLength: 200 + minItems: 1 + additionalProperties: false + categories: + type: + - array + - 'null' + items: + $ref: '#/components/schemas/RunMetricRequestMetricCategoriesItem' + minItems: 1 + child_metrics: + type: + - array + - 'null' + items: + $ref: '#/components/schemas/RunMetricRequestMetricChildMetricsItem' + minItems: 1 + filters: + $ref: '#/components/schemas/RunMetricRequestMetricFilters' + required: + - name + - type + - criteria + additionalProperties: false + RunMetricRequestEventFeedback: + type: object + properties: + ground_truth: {} + additionalProperties: {} + RunMetricRequestEvent: + type: object + properties: + event_type: + type: string + event_name: + type: string + inputs: + type: object + additionalProperties: {} + outputs: + type: object + additionalProperties: {} + workspace_id: + type: string + feedback: + $ref: '#/components/schemas/RunMetricRequestEventFeedback' + additionalProperties: {} + LegacyRunMetricRequestMetricCategoriesItem: + type: object + properties: + category: + type: string + maxLength: 200 + score: + type: + - number + - 'null' + required: + - category + - score + additionalProperties: false + LegacyRunMetricRequestMetricChildMetricsItem: + type: object + properties: + id: + type: string + minLength: 1 + name: + type: string + maxLength: 200 + weight: + type: number + scale: + type: + - integer + - 'null' + exclusiveMinimum: 0 + required: + - name + - weight + additionalProperties: false + LegacyRunMetricRequestMetricFilters: + type: object + properties: + filterArray: + $ref: '#/components/schemas/FiltersArray' + default: + filterArray: [] + required: + - filterArray + additionalProperties: false + LegacyRunMetricRequestMetric: + type: object + properties: + name: + type: string + maxLength: 200 + type: + type: string + enum: + - LLM + - PYTHON + criteria: + type: string + minLength: 1 + description: + type: string + default: '' + return_type: + type: string + enum: + - float + - boolean + - string + - categorical + default: float + enabled_in_prod: + type: boolean + default: false + needs_ground_truth: + type: boolean + default: false + sampling_percentage: + type: number + minimum: 0 + maximum: 100 + default: 100 + model_provider: + type: + - string + - 'null' + model_name: + type: + - string + - 'null' + scale: + type: + - integer + - 'null' + exclusiveMinimum: 0 + threshold: + type: + - object + - 'null' + properties: + min: + type: number + max: + type: number + pass_when: + anyOf: + - type: boolean + - type: number + passing_categories: + type: array + items: + type: string + maxLength: 200 + minItems: 1 + additionalProperties: false + categories: + type: + - array + - 'null' + items: + $ref: '#/components/schemas/LegacyRunMetricRequestMetricCategoriesItem' + minItems: 1 + child_metrics: + type: + - array + - 'null' + items: + $ref: '#/components/schemas/LegacyRunMetricRequestMetricChildMetricsItem' + minItems: 1 + filters: + $ref: '#/components/schemas/LegacyRunMetricRequestMetricFilters' + required: + - name + - type + - criteria + additionalProperties: false + LegacyRunMetricRequestEventFeedback: + type: object + properties: + ground_truth: {} + additionalProperties: {} + LegacyRunMetricRequestEvent: + type: object + properties: + event_type: + type: string + event_name: + type: string + inputs: + type: object + additionalProperties: {} + outputs: + type: object + additionalProperties: {} + workspace_id: + type: string + feedback: + $ref: '#/components/schemas/LegacyRunMetricRequestEventFeedback' + additionalProperties: {} + AnnotationQueueFilters: + type: object + properties: + filterArray: + $ref: '#/components/schemas/FiltersArray' + required: + - filterArray + additionalProperties: false + BaseAnnotationQueueFilters: + type: object + properties: + filterArray: + $ref: '#/components/schemas/FiltersArray' + required: + - filterArray + additionalProperties: false + CreateAnnotationQueueRequestFilters: + type: object + properties: + filterArray: + $ref: '#/components/schemas/FiltersArray' + default: + filterArray: [] + required: + - filterArray + additionalProperties: false + UpdateAnnotationQueueRequestFilters: + type: object + properties: + filterArray: + $ref: '#/components/schemas/FiltersArray' + required: + - filterArray + additionalProperties: false + CreateAnnotationQueueResponseQueueFilters: + type: object + properties: + filterArray: + $ref: '#/components/schemas/FiltersArray' + required: + - filterArray + additionalProperties: false + CreateAnnotationQueueResponseQueue: + type: object + properties: + name: + type: string + maxLength: 200 + description: + type: string + maxLength: 1000 + filters: + $ref: '#/components/schemas/CreateAnnotationQueueResponseQueueFilters' + enabled: + type: boolean + id: + type: string + minLength: 1 + scope_id: + type: string + scope_type: + type: string + enum: + - system + - controlplane + - dataplane + - org + - workspace + - project + is_active: + type: boolean + created_at: + type: string + format: date-time + updated_at: + type: + - string + - 'null' + format: date-time + required: + - name + - description + - filters + - enabled + - id + - scope_id + - scope_type + - is_active + - created_at + - updated_at + UpdateAnnotationQueueResponseQueueFilters: + type: object + properties: + filterArray: + $ref: '#/components/schemas/FiltersArray' + required: + - filterArray + additionalProperties: false + UpdateAnnotationQueueResponseQueue: + type: object + properties: + name: + type: string + maxLength: 200 + description: + type: string + maxLength: 1000 + filters: + $ref: '#/components/schemas/UpdateAnnotationQueueResponseQueueFilters' + enabled: + type: boolean + id: + type: string + minLength: 1 + scope_id: + type: string + scope_type: + type: string + enum: + - system + - controlplane + - dataplane + - org + - workspace + - project + is_active: + type: boolean + created_at: + type: string + format: date-time + updated_at: + type: + - string + - 'null' + format: date-time + required: + - name + - description + - filters + - enabled + - id + - scope_id + - scope_type + - is_active + - created_at + - updated_at + GetAnnotationQueuesResponseQueuesItemFilters: + type: object + properties: + filterArray: + $ref: '#/components/schemas/FiltersArray' + required: + - filterArray + additionalProperties: false + GetAnnotationQueuesResponseQueuesItem: + type: object + properties: + name: + type: string + maxLength: 200 + description: + type: string + maxLength: 1000 + filters: + $ref: '#/components/schemas/GetAnnotationQueuesResponseQueuesItemFilters' + enabled: + type: boolean + id: + type: string + minLength: 1 + scope_id: + type: string + scope_type: + type: string + enum: + - system + - controlplane + - dataplane + - org + - workspace + - project + is_active: + type: boolean + created_at: + type: string + format: date-time + updated_at: + type: + - string + - 'null' + format: date-time + required: + - name + - description + - filters + - enabled + - id + - scope_id + - scope_type + - is_active + - created_at + - updated_at + GetAnnotationQueueByIdResponseFilters: + type: object + properties: + filterArray: + $ref: '#/components/schemas/FiltersArray' + required: + - filterArray + additionalProperties: false + PostSessionRequestFeedback: + type: object + properties: + ground_truth: {} + StartSessionRequestSession: + type: object + properties: + session_id: + type: string + description: Client-provided session ID (server generates one if omitted) + session_name: + type: string + description: Display name for the session + event_name: + type: string + description: Fallback name if session_name is not provided + source: + type: string + description: Source of the session (e.g., sdk-python) + start_time: + type: number + description: Session start time as Unix milliseconds + end_time: + type: number + description: Session end time as Unix milliseconds + duration: + type: number + description: Session duration in milliseconds + config: + type: object + additionalProperties: {} + description: Configuration associated with the session + inputs: + type: object + additionalProperties: {} + description: Input data for the session + outputs: + type: object + additionalProperties: {} + description: Output data from the session + metadata: + type: object + additionalProperties: {} + description: Arbitrary metadata for the session + user_properties: + type: object + additionalProperties: {} + description: User properties associated with the session + children_ids: + type: array + items: + type: string + description: IDs of child events in this session + additionalProperties: {} + GetEventsSchemaLegacyDateRangeOneOf1: + type: object + properties: + $gte: + oneOf: + - type: string + - type: number + $lte: + oneOf: + - type: string + - type: number + GetRunsDateRangeOneOf1: + type: object + properties: + $gte: + oneOf: + - type: string + - type: number + $lte: + oneOf: + - type: string + - type: number + GetRunsSchemaDateRangeOneOf1: + type: object + properties: + $gte: + oneOf: + - type: string + - type: number + $lte: + oneOf: + - type: string + - type: number + GetRunSchemaDateRangeOneOf1: + type: object + properties: + $gte: + oneOf: + - type: string + - type: number + $lte: + oneOf: + - type: string + - type: number parameters: {} securitySchemes: BearerAuth: @@ -4608,3 +6163,28 @@ components: scheme: bearer security: - BearerAuth: [] +tags: + - name: Configurations + description: | + Manage prompt configurations — model parameters, message templates, and response settings — that can be versioned and deployed without code changes. + - name: Datapoints + description: | + Manage individual records inside datasets, including batch creation and mapping to source events. + - name: Datasets + description: | + Curate collections of datapoints used as test sets for evaluations and experiments. + - name: Events + description: | + Read and write trace events — the spans that capture every step of an AI application's execution. + - name: Experiments + description: | + Run, retrieve, and compare evaluation runs to measure how prompt or configuration changes affect agent performance. + - name: Metrics + description: | + Define and run evaluators — automated quality checks that score traces against criteria like accuracy, safety, or correctness. + - name: Queues + description: | + Manage annotation queues for human review of traces, turning expert feedback into labeled datasets. + - name: Sessions + description: | + Group related trace events into sessions, the top-level container for a multi-step or multi-service AI interaction. diff --git a/src/honeyhive/__init__.py b/src/honeyhive/__init__.py index 29a53f85..c09c9fde 100644 --- a/src/honeyhive/__init__.py +++ b/src/honeyhive/__init__.py @@ -5,7 +5,7 @@ # Version must be defined BEFORE imports to avoid circular import issues # Version must be semver or semver followed by "a" (alpha), "b" (beta), or "rc" # (release candidate) + a number -__version__ = "1.0.0rc22" +__version__ = "1.0.0" # Main API client from .api import HoneyHive diff --git a/src/honeyhive/_generated/models/DeleteDatasetQuery.py b/src/honeyhive/_generated/models/DeleteDatasetParams.py similarity index 67% rename from src/honeyhive/_generated/models/DeleteDatasetQuery.py rename to src/honeyhive/_generated/models/DeleteDatasetParams.py index 63c04e54..abf23db4 100644 --- a/src/honeyhive/_generated/models/DeleteDatasetQuery.py +++ b/src/honeyhive/_generated/models/DeleteDatasetParams.py @@ -2,13 +2,13 @@ from pydantic import BaseModel, Field -__all__ = ["DeleteDatasetQuery"] +__all__ = ["DeleteDatasetParams"] -class DeleteDatasetQuery(BaseModel): +class DeleteDatasetParams(BaseModel): """ - DeleteDatasetQuery model - Query parameters for DELETE /datasets + DeleteDatasetParams model + Path parameters for DELETE /datasets/{dataset_id} """ model_config = { diff --git a/src/honeyhive/_generated/models/DeleteMetricParams.py b/src/honeyhive/_generated/models/DeleteMetricParams.py new file mode 100644 index 00000000..b5152e02 --- /dev/null +++ b/src/honeyhive/_generated/models/DeleteMetricParams.py @@ -0,0 +1,21 @@ +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field + +__all__ = ["DeleteMetricParams"] + + +class DeleteMetricParams(BaseModel): + """ + DeleteMetricParams model + Path parameters for DELETE /metrics/{metric_id} + """ + + model_config = { + "populate_by_name": True, + "validate_assignment": True, + "extra": "allow", + "protected_namespaces": (), + } + + metric_id: str = Field(validation_alias="metric_id") diff --git a/src/honeyhive/_generated/models/DeleteMetricResponse.py b/src/honeyhive/_generated/models/DeleteMetricResponse.py index 92025aea..bc3ce4cd 100644 --- a/src/honeyhive/_generated/models/DeleteMetricResponse.py +++ b/src/honeyhive/_generated/models/DeleteMetricResponse.py @@ -8,7 +8,7 @@ class DeleteMetricResponse(BaseModel): """ DeleteMetricResponse model - Response for DELETE /metrics + Response for DELETE /metrics/{metric_id} """ model_config = { diff --git a/src/honeyhive/_generated/models/GetEventsSchemaLegacyDateRangeOneOf1.py b/src/honeyhive/_generated/models/GetEventsSchemaLegacyDateRangeOneOf1.py new file mode 100644 index 00000000..0b60d2d5 --- /dev/null +++ b/src/honeyhive/_generated/models/GetEventsSchemaLegacyDateRangeOneOf1.py @@ -0,0 +1,22 @@ +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field + +__all__ = ["GetEventsSchemaLegacyDateRangeOneOf1"] + + +class GetEventsSchemaLegacyDateRangeOneOf1(BaseModel): + """ + GetEventsSchemaLegacyDateRangeOneOf1 model + """ + + model_config = { + "populate_by_name": True, + "validate_assignment": True, + "extra": "allow", + "protected_namespaces": (), + } + + gte: Optional[Union[str, float]] = Field(validation_alias="$gte", default=None) + + lte: Optional[Union[str, float]] = Field(validation_alias="$lte", default=None) diff --git a/src/honeyhive/_generated/models/GetEventsSchemaResponse.py b/src/honeyhive/_generated/models/GetEventsSchemaResponse.py index cc8470a0..db782b6d 100644 --- a/src/honeyhive/_generated/models/GetEventsSchemaResponse.py +++ b/src/honeyhive/_generated/models/GetEventsSchemaResponse.py @@ -10,7 +10,7 @@ class GetEventsSchemaResponse(BaseModel): """ GetEventsSchemaResponse model - Response for GET /events/schema + Response for GET /events/schema, GET /runs/{run_id}/schema, and GET /runs/schema """ model_config = { diff --git a/src/honeyhive/_generated/models/GetExperimentCompareEventsResponse.py b/src/honeyhive/_generated/models/GetExperimentCompareEventsResponse.py index d9bdc350..107d180f 100644 --- a/src/honeyhive/_generated/models/GetExperimentCompareEventsResponse.py +++ b/src/honeyhive/_generated/models/GetExperimentCompareEventsResponse.py @@ -10,7 +10,7 @@ class GetExperimentCompareEventsResponse(BaseModel): """ GetExperimentCompareEventsResponse model - Response for GET /runs/compare/events + Response for GET /runs/{new_run_id}/compare/{old_run_id}/events """ model_config = { diff --git a/src/honeyhive/_generated/models/GetExperimentRunCompareEventsParams.py b/src/honeyhive/_generated/models/GetExperimentRunCompareEventsParams.py new file mode 100644 index 00000000..96b3c676 --- /dev/null +++ b/src/honeyhive/_generated/models/GetExperimentRunCompareEventsParams.py @@ -0,0 +1,23 @@ +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field + +__all__ = ["GetExperimentRunCompareEventsParams"] + + +class GetExperimentRunCompareEventsParams(BaseModel): + """ + GetExperimentRunCompareEventsParams model + Path parameters for GET /runs/{new_run_id}/compare/{old_run_id}/events + """ + + model_config = { + "populate_by_name": True, + "validate_assignment": True, + "extra": "allow", + "protected_namespaces": (), + } + + new_run_id: str = Field(validation_alias="new_run_id") + + old_run_id: str = Field(validation_alias="old_run_id") diff --git a/src/honeyhive/_generated/models/GetExperimentRunCompareEventsQuery.py b/src/honeyhive/_generated/models/GetExperimentRunCompareEventsQuery.py index 5549b0b0..4662c8ec 100644 --- a/src/honeyhive/_generated/models/GetExperimentRunCompareEventsQuery.py +++ b/src/honeyhive/_generated/models/GetExperimentRunCompareEventsQuery.py @@ -8,7 +8,7 @@ class GetExperimentRunCompareEventsQuery(BaseModel): """ GetExperimentRunCompareEventsQuery model - Query parameters for GET /runs/compare/events + Query parameters for GET /runs/{new_run_id}/compare/{old_run_id}/events """ model_config = { @@ -18,10 +18,6 @@ class GetExperimentRunCompareEventsQuery(BaseModel): "protected_namespaces": (), } - run_id_1: str = Field(validation_alias="run_id_1") - - run_id_2: str = Field(validation_alias="run_id_2") - event_name: Optional[str] = Field(validation_alias="event_name", default=None) event_type: Optional[str] = Field(validation_alias="event_type", default=None) diff --git a/src/honeyhive/_generated/models/GetExperimentRunCompareParams.py b/src/honeyhive/_generated/models/GetExperimentRunCompareParams.py index 3548071e..ccb1ca89 100644 --- a/src/honeyhive/_generated/models/GetExperimentRunCompareParams.py +++ b/src/honeyhive/_generated/models/GetExperimentRunCompareParams.py @@ -8,7 +8,7 @@ class GetExperimentRunCompareParams(BaseModel): """ GetExperimentRunCompareParams model - Path parameters for GET /runs/{new_run_id}/compare-with/{old_run_id} + Path parameters for GET /runs/{new_run_id}/compare/{old_run_id} """ model_config = { diff --git a/src/honeyhive/_generated/models/GetExperimentRunCompareQuery.py b/src/honeyhive/_generated/models/GetExperimentRunCompareQuery.py index 267c3812..9a3ae64c 100644 --- a/src/honeyhive/_generated/models/GetExperimentRunCompareQuery.py +++ b/src/honeyhive/_generated/models/GetExperimentRunCompareQuery.py @@ -8,7 +8,7 @@ class GetExperimentRunCompareQuery(BaseModel): """ GetExperimentRunCompareQuery model - Query parameters for GET /runs/{new_run_id}/compare-with/{old_run_id} + Query parameters for GET /runs/{new_run_id}/compare/{old_run_id} """ model_config = { diff --git a/src/honeyhive/_generated/models/GetEventsSchemaDateRangeOneOf1.py b/src/honeyhive/_generated/models/GetRunSchemaDateRangeOneOf1.py similarity index 76% rename from src/honeyhive/_generated/models/GetEventsSchemaDateRangeOneOf1.py rename to src/honeyhive/_generated/models/GetRunSchemaDateRangeOneOf1.py index 25eaf576..bb07e1d2 100644 --- a/src/honeyhive/_generated/models/GetEventsSchemaDateRangeOneOf1.py +++ b/src/honeyhive/_generated/models/GetRunSchemaDateRangeOneOf1.py @@ -2,12 +2,12 @@ from pydantic import BaseModel, Field -__all__ = ["GetEventsSchemaDateRangeOneOf1"] +__all__ = ["GetRunSchemaDateRangeOneOf1"] -class GetEventsSchemaDateRangeOneOf1(BaseModel): +class GetRunSchemaDateRangeOneOf1(BaseModel): """ - GetEventsSchemaDateRangeOneOf1 model + GetRunSchemaDateRangeOneOf1 model """ model_config = { diff --git a/src/honeyhive/_generated/models/GetRunSchemaQuery.py b/src/honeyhive/_generated/models/GetRunSchemaQuery.py new file mode 100644 index 00000000..11342227 --- /dev/null +++ b/src/honeyhive/_generated/models/GetRunSchemaQuery.py @@ -0,0 +1,25 @@ +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field + +from .AbsoluteDateRange import AbsoluteDateRange + +__all__ = ["GetRunSchemaQuery"] + + +class GetRunSchemaQuery(BaseModel): + """ + GetRunSchemaQuery model + Query parameters for GET /runs/{run_id}/schema + """ + + model_config = { + "populate_by_name": True, + "validate_assignment": True, + "extra": "allow", + "protected_namespaces": (), + } + + dateRange: Optional[Union[str, AbsoluteDateRange]] = Field( + validation_alias="dateRange", default=None + ) diff --git a/src/honeyhive/_generated/models/GetRunsSchemaDateRangeOneOf1.py b/src/honeyhive/_generated/models/GetRunsSchemaDateRangeOneOf1.py new file mode 100644 index 00000000..717f1eed --- /dev/null +++ b/src/honeyhive/_generated/models/GetRunsSchemaDateRangeOneOf1.py @@ -0,0 +1,22 @@ +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field + +__all__ = ["GetRunsSchemaDateRangeOneOf1"] + + +class GetRunsSchemaDateRangeOneOf1(BaseModel): + """ + GetRunsSchemaDateRangeOneOf1 model + """ + + model_config = { + "populate_by_name": True, + "validate_assignment": True, + "extra": "allow", + "protected_namespaces": (), + } + + gte: Optional[Union[str, float]] = Field(validation_alias="$gte", default=None) + + lte: Optional[Union[str, float]] = Field(validation_alias="$lte", default=None) diff --git a/src/honeyhive/_generated/models/GetRunsSchemaQuery.py b/src/honeyhive/_generated/models/GetRunsSchemaQuery.py new file mode 100644 index 00000000..4264e082 --- /dev/null +++ b/src/honeyhive/_generated/models/GetRunsSchemaQuery.py @@ -0,0 +1,25 @@ +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field + +from .AbsoluteDateRange import AbsoluteDateRange + +__all__ = ["GetRunsSchemaQuery"] + + +class GetRunsSchemaQuery(BaseModel): + """ + GetRunsSchemaQuery model + Query parameters for GET /runs/schema + """ + + model_config = { + "populate_by_name": True, + "validate_assignment": True, + "extra": "allow", + "protected_namespaces": (), + } + + dateRange: Optional[Union[str, AbsoluteDateRange]] = Field( + validation_alias="dateRange", default=None + ) diff --git a/src/honeyhive/_generated/models/LegacyDeleteDatasetQuery.py b/src/honeyhive/_generated/models/LegacyDeleteDatasetQuery.py new file mode 100644 index 00000000..635b74be --- /dev/null +++ b/src/honeyhive/_generated/models/LegacyDeleteDatasetQuery.py @@ -0,0 +1,21 @@ +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field + +__all__ = ["LegacyDeleteDatasetQuery"] + + +class LegacyDeleteDatasetQuery(BaseModel): + """ + LegacyDeleteDatasetQuery model + Query parameters for DELETE /datasets (deprecated — use DELETE /datasets/{dataset_id}) + """ + + model_config = { + "populate_by_name": True, + "validate_assignment": True, + "extra": "allow", + "protected_namespaces": (), + } + + dataset_id: str = Field(validation_alias="dataset_id") diff --git a/src/honeyhive/_generated/models/LegacyDeleteMetricQuery.py b/src/honeyhive/_generated/models/LegacyDeleteMetricQuery.py new file mode 100644 index 00000000..dd55e4ca --- /dev/null +++ b/src/honeyhive/_generated/models/LegacyDeleteMetricQuery.py @@ -0,0 +1,21 @@ +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field + +__all__ = ["LegacyDeleteMetricQuery"] + + +class LegacyDeleteMetricQuery(BaseModel): + """ + LegacyDeleteMetricQuery model + Query parameters for DELETE /metrics (deprecated — use DELETE /metrics/{metric_id}) + """ + + model_config = { + "populate_by_name": True, + "validate_assignment": True, + "extra": "allow", + "protected_namespaces": (), + } + + metric_id: str = Field(validation_alias="metric_id") diff --git a/src/honeyhive/_generated/models/GetEventsSchemaQuery.py b/src/honeyhive/_generated/models/LegacyGetEventsSchemaQuery.py similarity index 69% rename from src/honeyhive/_generated/models/GetEventsSchemaQuery.py rename to src/honeyhive/_generated/models/LegacyGetEventsSchemaQuery.py index ba78a35e..25f6d80a 100644 --- a/src/honeyhive/_generated/models/GetEventsSchemaQuery.py +++ b/src/honeyhive/_generated/models/LegacyGetEventsSchemaQuery.py @@ -4,13 +4,13 @@ from .AbsoluteDateRange import AbsoluteDateRange -__all__ = ["GetEventsSchemaQuery"] +__all__ = ["LegacyGetEventsSchemaQuery"] -class GetEventsSchemaQuery(BaseModel): +class LegacyGetEventsSchemaQuery(BaseModel): """ - GetEventsSchemaQuery model - Query parameters for GET /events/schema + LegacyGetEventsSchemaQuery model + Query parameters for GET /events/schema (deprecated — use GET /runs/{run_id}/schema or GET /runs/schema) """ model_config = { diff --git a/src/honeyhive/_generated/models/LegacyGetExperimentRunCompareEventsQuery.py b/src/honeyhive/_generated/models/LegacyGetExperimentRunCompareEventsQuery.py new file mode 100644 index 00000000..d896ac0f --- /dev/null +++ b/src/honeyhive/_generated/models/LegacyGetExperimentRunCompareEventsQuery.py @@ -0,0 +1,35 @@ +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field + +__all__ = ["LegacyGetExperimentRunCompareEventsQuery"] + + +class LegacyGetExperimentRunCompareEventsQuery(BaseModel): + """ + LegacyGetExperimentRunCompareEventsQuery model + Query parameters for GET /runs/compare/events (deprecated — use GET /runs/{new_run_id}/compare/{old_run_id}/events) + """ + + model_config = { + "populate_by_name": True, + "validate_assignment": True, + "extra": "allow", + "protected_namespaces": (), + } + + run_id_1: str = Field(validation_alias="run_id_1") + + run_id_2: str = Field(validation_alias="run_id_2") + + event_name: Optional[str] = Field(validation_alias="event_name", default=None) + + event_type: Optional[str] = Field(validation_alias="event_type", default=None) + + filter: Optional[Union[str, Dict[str, Any]]] = Field( + validation_alias="filter", default=None + ) + + limit: Optional[int] = Field(validation_alias="limit", default=None) + + page: Optional[int] = Field(validation_alias="page", default=None) diff --git a/src/honeyhive/_generated/models/LegacyGetExperimentRunCompareParams.py b/src/honeyhive/_generated/models/LegacyGetExperimentRunCompareParams.py new file mode 100644 index 00000000..e22271ba --- /dev/null +++ b/src/honeyhive/_generated/models/LegacyGetExperimentRunCompareParams.py @@ -0,0 +1,23 @@ +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field + +__all__ = ["LegacyGetExperimentRunCompareParams"] + + +class LegacyGetExperimentRunCompareParams(BaseModel): + """ + LegacyGetExperimentRunCompareParams model + Path parameters for GET /runs/{new_run_id}/compare-with/{old_run_id} (deprecated — use GET /runs/{new_run_id}/compare/{old_run_id}) + """ + + model_config = { + "populate_by_name": True, + "validate_assignment": True, + "extra": "allow", + "protected_namespaces": (), + } + + new_run_id: str = Field(validation_alias="new_run_id") + + old_run_id: str = Field(validation_alias="old_run_id") diff --git a/src/honeyhive/_generated/models/LegacyGetExperimentRunCompareQuery.py b/src/honeyhive/_generated/models/LegacyGetExperimentRunCompareQuery.py new file mode 100644 index 00000000..e0575b27 --- /dev/null +++ b/src/honeyhive/_generated/models/LegacyGetExperimentRunCompareQuery.py @@ -0,0 +1,27 @@ +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field + +__all__ = ["LegacyGetExperimentRunCompareQuery"] + + +class LegacyGetExperimentRunCompareQuery(BaseModel): + """ + LegacyGetExperimentRunCompareQuery model + Query parameters for GET /runs/{new_run_id}/compare-with/{old_run_id} (deprecated — use GET /runs/{new_run_id}/compare/{old_run_id}) + """ + + model_config = { + "populate_by_name": True, + "validate_assignment": True, + "extra": "allow", + "protected_namespaces": (), + } + + aggregate_function: Optional[str] = Field( + validation_alias="aggregate_function", default=None + ) + + filters: Optional[Union[str, List[Any]]] = Field( + validation_alias="filters", default=None + ) diff --git a/src/honeyhive/_generated/models/LegacyPostEventBatchRequest.py b/src/honeyhive/_generated/models/LegacyPostEventBatchRequest.py new file mode 100644 index 00000000..aa56d6b6 --- /dev/null +++ b/src/honeyhive/_generated/models/LegacyPostEventBatchRequest.py @@ -0,0 +1,40 @@ +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field + +from .LegacyEvent import LegacyEvent +from .SessionProperties import SessionProperties + +__all__ = ["LegacyPostEventBatchRequest"] + + +class LegacyPostEventBatchRequest(BaseModel): + """ + LegacyPostEventBatchRequest model + Request body for POST /events/batch (deprecated — use POST /v1/events/batch) + """ + + model_config = { + "populate_by_name": True, + "validate_assignment": True, + "extra": "allow", + "protected_namespaces": (), + } + + events: List[LegacyEvent] = Field(validation_alias="events") + + single_session: Optional[bool] = Field( + validation_alias="single_session", default=None + ) + + is_single_session: Optional[bool] = Field( + validation_alias="is_single_session", default=None + ) + + session: Optional[SessionProperties] = Field( + validation_alias="session", default=None + ) + + session_properties: Optional[SessionProperties] = Field( + validation_alias="session_properties", default=None + ) diff --git a/src/honeyhive/_generated/models/LegacyPostEventRequest.py b/src/honeyhive/_generated/models/LegacyPostEventRequest.py new file mode 100644 index 00000000..27d66207 --- /dev/null +++ b/src/honeyhive/_generated/models/LegacyPostEventRequest.py @@ -0,0 +1,23 @@ +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field + +from .LegacyPostEventRequestEvent import LegacyPostEventRequestEvent + +__all__ = ["LegacyPostEventRequest"] + + +class LegacyPostEventRequest(BaseModel): + """ + LegacyPostEventRequest model + Request body for POST /events (deprecated — use POST /v1/events) + """ + + model_config = { + "populate_by_name": True, + "validate_assignment": True, + "extra": "allow", + "protected_namespaces": (), + } + + event: LegacyPostEventRequestEvent = Field(validation_alias="event") diff --git a/src/honeyhive/_generated/models/PostEventRequestEvent.py b/src/honeyhive/_generated/models/LegacyPostEventRequestEvent.py similarity index 94% rename from src/honeyhive/_generated/models/PostEventRequestEvent.py rename to src/honeyhive/_generated/models/LegacyPostEventRequestEvent.py index 3d963f55..27dc7c0a 100644 --- a/src/honeyhive/_generated/models/PostEventRequestEvent.py +++ b/src/honeyhive/_generated/models/LegacyPostEventRequestEvent.py @@ -2,12 +2,12 @@ from pydantic import BaseModel, Field -__all__ = ["PostEventRequestEvent"] +__all__ = ["LegacyPostEventRequestEvent"] -class PostEventRequestEvent(BaseModel): +class LegacyPostEventRequestEvent(BaseModel): """ - PostEventRequestEvent model + LegacyPostEventRequestEvent model Full event object for legacy event creation endpoints """ diff --git a/src/honeyhive/_generated/models/LegacyRemoveDatapointFromDatasetParams.py b/src/honeyhive/_generated/models/LegacyRemoveDatapointFromDatasetParams.py new file mode 100644 index 00000000..5ae553e6 --- /dev/null +++ b/src/honeyhive/_generated/models/LegacyRemoveDatapointFromDatasetParams.py @@ -0,0 +1,23 @@ +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field + +__all__ = ["LegacyRemoveDatapointFromDatasetParams"] + + +class LegacyRemoveDatapointFromDatasetParams(BaseModel): + """ + LegacyRemoveDatapointFromDatasetParams model + Path parameters for DELETE /datasets/{dataset_id}/{datapoint_id} (deprecated — use DELETE /datasets/{dataset_id}/datapoints/{datapoint_id}) + """ + + model_config = { + "populate_by_name": True, + "validate_assignment": True, + "extra": "allow", + "protected_namespaces": (), + } + + dataset_id: str = Field(validation_alias="dataset_id") + + datapoint_id: str = Field(validation_alias="datapoint_id") diff --git a/src/honeyhive/_generated/models/LegacyRunMetricRequest.py b/src/honeyhive/_generated/models/LegacyRunMetricRequest.py new file mode 100644 index 00000000..479b3251 --- /dev/null +++ b/src/honeyhive/_generated/models/LegacyRunMetricRequest.py @@ -0,0 +1,26 @@ +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field + +from .LegacyRunMetricRequestEvent import LegacyRunMetricRequestEvent +from .LegacyRunMetricRequestMetric import LegacyRunMetricRequestMetric + +__all__ = ["LegacyRunMetricRequest"] + + +class LegacyRunMetricRequest(BaseModel): + """ + LegacyRunMetricRequest model + Request body for POST /metrics/run_metric (deprecated — use POST /metrics/run) + """ + + model_config = { + "populate_by_name": True, + "validate_assignment": True, + "extra": "allow", + "protected_namespaces": (), + } + + metric: LegacyRunMetricRequestMetric = Field(validation_alias="metric") + + event: LegacyRunMetricRequestEvent = Field(validation_alias="event") diff --git a/src/honeyhive/_generated/models/LegacyRunMetricRequestEvent.py b/src/honeyhive/_generated/models/LegacyRunMetricRequestEvent.py new file mode 100644 index 00000000..2d7d3a12 --- /dev/null +++ b/src/honeyhive/_generated/models/LegacyRunMetricRequestEvent.py @@ -0,0 +1,34 @@ +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field + +from .LegacyRunMetricRequestEventFeedback import LegacyRunMetricRequestEventFeedback + +__all__ = ["LegacyRunMetricRequestEvent"] + + +class LegacyRunMetricRequestEvent(BaseModel): + """ + LegacyRunMetricRequestEvent model + """ + + model_config = { + "populate_by_name": True, + "validate_assignment": True, + "extra": "allow", + "protected_namespaces": (), + } + + event_type: Optional[str] = Field(validation_alias="event_type", default=None) + + event_name: Optional[str] = Field(validation_alias="event_name", default=None) + + inputs: Optional[Dict[str, Any]] = Field(validation_alias="inputs", default=None) + + outputs: Optional[Dict[str, Any]] = Field(validation_alias="outputs", default=None) + + workspace_id: Optional[str] = Field(validation_alias="workspace_id", default=None) + + feedback: Optional[LegacyRunMetricRequestEventFeedback] = Field( + validation_alias="feedback", default=None + ) diff --git a/src/honeyhive/_generated/models/LegacyRunMetricRequestEventFeedback.py b/src/honeyhive/_generated/models/LegacyRunMetricRequestEventFeedback.py new file mode 100644 index 00000000..42781234 --- /dev/null +++ b/src/honeyhive/_generated/models/LegacyRunMetricRequestEventFeedback.py @@ -0,0 +1,20 @@ +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field + +__all__ = ["LegacyRunMetricRequestEventFeedback"] + + +class LegacyRunMetricRequestEventFeedback(BaseModel): + """ + LegacyRunMetricRequestEventFeedback model + """ + + model_config = { + "populate_by_name": True, + "validate_assignment": True, + "extra": "allow", + "protected_namespaces": (), + } + + ground_truth: Optional[Any] = Field(validation_alias="ground_truth", default=None) diff --git a/src/honeyhive/_generated/models/LegacyRunMetricRequestMetric.py b/src/honeyhive/_generated/models/LegacyRunMetricRequestMetric.py new file mode 100644 index 00000000..d586ba63 --- /dev/null +++ b/src/honeyhive/_generated/models/LegacyRunMetricRequestMetric.py @@ -0,0 +1,64 @@ +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field + +from .LegacyRunMetricRequestMetricFilters import LegacyRunMetricRequestMetricFilters + +__all__ = ["LegacyRunMetricRequestMetric"] + + +class LegacyRunMetricRequestMetric(BaseModel): + """ + LegacyRunMetricRequestMetric model + """ + + model_config = { + "populate_by_name": True, + "validate_assignment": True, + "extra": "allow", + "protected_namespaces": (), + } + + name: str = Field(validation_alias="name") + + type: str = Field(validation_alias="type") + + criteria: str = Field(validation_alias="criteria") + + description: Optional[str] = Field(validation_alias="description", default=None) + + return_type: Optional[str] = Field(validation_alias="return_type", default=None) + + enabled_in_prod: Optional[bool] = Field( + validation_alias="enabled_in_prod", default=None + ) + + needs_ground_truth: Optional[bool] = Field( + validation_alias="needs_ground_truth", default=None + ) + + sampling_percentage: Optional[float] = Field( + validation_alias="sampling_percentage", default=None + ) + + model_provider: Optional[str] = Field( + validation_alias="model_provider", default=None + ) + + model_name: Optional[str] = Field(validation_alias="model_name", default=None) + + scale: Optional[int] = Field(validation_alias="scale", default=None) + + threshold: Optional[Dict[str, Any]] = Field( + validation_alias="threshold", default=None + ) + + categories: Optional[List[Any]] = Field(validation_alias="categories", default=None) + + child_metrics: Optional[List[Any]] = Field( + validation_alias="child_metrics", default=None + ) + + filters: Optional[LegacyRunMetricRequestMetricFilters] = Field( + validation_alias="filters", default=None + ) diff --git a/src/honeyhive/_generated/models/LegacyRunMetricRequestMetricCategoriesItem.py b/src/honeyhive/_generated/models/LegacyRunMetricRequestMetricCategoriesItem.py new file mode 100644 index 00000000..94ea4147 --- /dev/null +++ b/src/honeyhive/_generated/models/LegacyRunMetricRequestMetricCategoriesItem.py @@ -0,0 +1,22 @@ +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field + +__all__ = ["LegacyRunMetricRequestMetricCategoriesItem"] + + +class LegacyRunMetricRequestMetricCategoriesItem(BaseModel): + """ + LegacyRunMetricRequestMetricCategoriesItem model + """ + + model_config = { + "populate_by_name": True, + "validate_assignment": True, + "extra": "allow", + "protected_namespaces": (), + } + + category: str = Field(validation_alias="category") + + score: Optional[float] = Field(validation_alias="score") diff --git a/src/honeyhive/_generated/models/LegacyRunMetricRequestMetricChildMetricsItem.py b/src/honeyhive/_generated/models/LegacyRunMetricRequestMetricChildMetricsItem.py new file mode 100644 index 00000000..11314516 --- /dev/null +++ b/src/honeyhive/_generated/models/LegacyRunMetricRequestMetricChildMetricsItem.py @@ -0,0 +1,26 @@ +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field + +__all__ = ["LegacyRunMetricRequestMetricChildMetricsItem"] + + +class LegacyRunMetricRequestMetricChildMetricsItem(BaseModel): + """ + LegacyRunMetricRequestMetricChildMetricsItem model + """ + + model_config = { + "populate_by_name": True, + "validate_assignment": True, + "extra": "allow", + "protected_namespaces": (), + } + + id: Optional[str] = Field(validation_alias="id", default=None) + + name: str = Field(validation_alias="name") + + weight: float = Field(validation_alias="weight") + + scale: Optional[int] = Field(validation_alias="scale", default=None) diff --git a/src/honeyhive/_generated/models/LegacyRunMetricRequestMetricFilters.py b/src/honeyhive/_generated/models/LegacyRunMetricRequestMetricFilters.py new file mode 100644 index 00000000..2acc8c44 --- /dev/null +++ b/src/honeyhive/_generated/models/LegacyRunMetricRequestMetricFilters.py @@ -0,0 +1,22 @@ +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field + +from .FiltersArray import FiltersArray + +__all__ = ["LegacyRunMetricRequestMetricFilters"] + + +class LegacyRunMetricRequestMetricFilters(BaseModel): + """ + LegacyRunMetricRequestMetricFilters model + """ + + model_config = { + "populate_by_name": True, + "validate_assignment": True, + "extra": "allow", + "protected_namespaces": (), + } + + filterArray: FiltersArray = Field(validation_alias="filterArray") diff --git a/src/honeyhive/_generated/models/LegacyUpdateEventRequest.py b/src/honeyhive/_generated/models/LegacyUpdateEventRequest.py new file mode 100644 index 00000000..8aeac0b6 --- /dev/null +++ b/src/honeyhive/_generated/models/LegacyUpdateEventRequest.py @@ -0,0 +1,47 @@ +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field + +__all__ = ["LegacyUpdateEventRequest"] + + +class LegacyUpdateEventRequest(BaseModel): + """ + LegacyUpdateEventRequest model + Request to update an existing event (deprecated — use PUT /v1/events/{event_id}) + """ + + model_config = { + "populate_by_name": True, + "validate_assignment": True, + "extra": "allow", + "protected_namespaces": (), + } + + event_id: str = Field(validation_alias="event_id") + + metadata: Optional[Dict[str, Any]] = Field( + validation_alias="metadata", default=None + ) + + feedback: Optional[Dict[str, Any]] = Field( + validation_alias="feedback", default=None + ) + + metrics: Optional[Dict[str, Any]] = Field(validation_alias="metrics", default=None) + + outputs: Optional[Any] = Field(validation_alias="outputs", default=None) + + config: Optional[Dict[str, Any]] = Field(validation_alias="config", default=None) + + user_properties: Optional[Dict[str, Any]] = Field( + validation_alias="user_properties", default=None + ) + + duration: Optional[float] = Field(validation_alias="duration", default=None) + + end_time: Optional[float] = Field(validation_alias="end_time", default=None) + + children_ids: Optional[List[str]] = Field( + validation_alias="children_ids", default=None + ) diff --git a/src/honeyhive/_generated/models/LegacyUpdateMetricRequest.py b/src/honeyhive/_generated/models/LegacyUpdateMetricRequest.py new file mode 100644 index 00000000..cd012a4d --- /dev/null +++ b/src/honeyhive/_generated/models/LegacyUpdateMetricRequest.py @@ -0,0 +1,67 @@ +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field + +from .LegacyUpdateMetricRequestFilters import LegacyUpdateMetricRequestFilters + +__all__ = ["LegacyUpdateMetricRequest"] + + +class LegacyUpdateMetricRequest(BaseModel): + """ + LegacyUpdateMetricRequest model + Request body for PUT /metrics (deprecated — use PUT /metrics/{metric_id}) + """ + + model_config = { + "populate_by_name": True, + "validate_assignment": True, + "extra": "allow", + "protected_namespaces": (), + } + + name: Optional[str] = Field(validation_alias="name", default=None) + + type: Optional[str] = Field(validation_alias="type", default=None) + + criteria: Optional[str] = Field(validation_alias="criteria", default=None) + + description: Optional[str] = Field(validation_alias="description", default=None) + + return_type: Optional[str] = Field(validation_alias="return_type", default=None) + + enabled_in_prod: Optional[bool] = Field( + validation_alias="enabled_in_prod", default=None + ) + + needs_ground_truth: Optional[bool] = Field( + validation_alias="needs_ground_truth", default=None + ) + + sampling_percentage: Optional[float] = Field( + validation_alias="sampling_percentage", default=None + ) + + model_provider: Optional[str] = Field( + validation_alias="model_provider", default=None + ) + + model_name: Optional[str] = Field(validation_alias="model_name", default=None) + + scale: Optional[int] = Field(validation_alias="scale", default=None) + + threshold: Optional[Dict[str, Any]] = Field( + validation_alias="threshold", default=None + ) + + categories: Optional[List[Any]] = Field(validation_alias="categories", default=None) + + child_metrics: Optional[List[Any]] = Field( + validation_alias="child_metrics", default=None + ) + + filters: Optional[LegacyUpdateMetricRequestFilters] = Field( + validation_alias="filters", default=None + ) + + id: str = Field(validation_alias="id") diff --git a/src/honeyhive/_generated/models/LegacyUpdateMetricRequestCategoriesItem.py b/src/honeyhive/_generated/models/LegacyUpdateMetricRequestCategoriesItem.py new file mode 100644 index 00000000..3817f4da --- /dev/null +++ b/src/honeyhive/_generated/models/LegacyUpdateMetricRequestCategoriesItem.py @@ -0,0 +1,22 @@ +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field + +__all__ = ["LegacyUpdateMetricRequestCategoriesItem"] + + +class LegacyUpdateMetricRequestCategoriesItem(BaseModel): + """ + LegacyUpdateMetricRequestCategoriesItem model + """ + + model_config = { + "populate_by_name": True, + "validate_assignment": True, + "extra": "allow", + "protected_namespaces": (), + } + + category: str = Field(validation_alias="category") + + score: Optional[float] = Field(validation_alias="score") diff --git a/src/honeyhive/_generated/models/LegacyUpdateMetricRequestChildMetricsItem.py b/src/honeyhive/_generated/models/LegacyUpdateMetricRequestChildMetricsItem.py new file mode 100644 index 00000000..36bae52e --- /dev/null +++ b/src/honeyhive/_generated/models/LegacyUpdateMetricRequestChildMetricsItem.py @@ -0,0 +1,26 @@ +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field + +__all__ = ["LegacyUpdateMetricRequestChildMetricsItem"] + + +class LegacyUpdateMetricRequestChildMetricsItem(BaseModel): + """ + LegacyUpdateMetricRequestChildMetricsItem model + """ + + model_config = { + "populate_by_name": True, + "validate_assignment": True, + "extra": "allow", + "protected_namespaces": (), + } + + id: Optional[str] = Field(validation_alias="id", default=None) + + name: str = Field(validation_alias="name") + + weight: float = Field(validation_alias="weight") + + scale: Optional[int] = Field(validation_alias="scale", default=None) diff --git a/src/honeyhive/_generated/models/LegacyUpdateMetricRequestFilters.py b/src/honeyhive/_generated/models/LegacyUpdateMetricRequestFilters.py new file mode 100644 index 00000000..b1993fe1 --- /dev/null +++ b/src/honeyhive/_generated/models/LegacyUpdateMetricRequestFilters.py @@ -0,0 +1,22 @@ +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field + +from .FiltersArray import FiltersArray + +__all__ = ["LegacyUpdateMetricRequestFilters"] + + +class LegacyUpdateMetricRequestFilters(BaseModel): + """ + LegacyUpdateMetricRequestFilters model + """ + + model_config = { + "populate_by_name": True, + "validate_assignment": True, + "extra": "allow", + "protected_namespaces": (), + } + + filterArray: FiltersArray = Field(validation_alias="filterArray") diff --git a/src/honeyhive/_generated/models/PostEventBatchRequest.py b/src/honeyhive/_generated/models/PostEventBatchRequest.py index 9a484a96..42a761cd 100644 --- a/src/honeyhive/_generated/models/PostEventBatchRequest.py +++ b/src/honeyhive/_generated/models/PostEventBatchRequest.py @@ -2,7 +2,7 @@ from pydantic import BaseModel, Field -from .LegacyEvent import LegacyEvent +from .PostEventRequest import PostEventRequest from .SessionProperties import SessionProperties __all__ = ["PostEventBatchRequest"] @@ -11,7 +11,7 @@ class PostEventBatchRequest(BaseModel): """ PostEventBatchRequest model - Request body for POST /events/batch + Request body for POST /v1/events/batch """ model_config = { @@ -21,20 +21,12 @@ class PostEventBatchRequest(BaseModel): "protected_namespaces": (), } - events: List[LegacyEvent] = Field(validation_alias="events") + events: List[PostEventRequest] = Field(validation_alias="events") single_session: Optional[bool] = Field( validation_alias="single_session", default=None ) - is_single_session: Optional[bool] = Field( - validation_alias="is_single_session", default=None - ) - - session: Optional[SessionProperties] = Field( - validation_alias="session", default=None - ) - session_properties: Optional[SessionProperties] = Field( validation_alias="session_properties", default=None ) diff --git a/src/honeyhive/_generated/models/PostEventRequest.py b/src/honeyhive/_generated/models/PostEventRequest.py index 46b2a507..02418962 100644 --- a/src/honeyhive/_generated/models/PostEventRequest.py +++ b/src/honeyhive/_generated/models/PostEventRequest.py @@ -2,15 +2,13 @@ from pydantic import BaseModel, Field -from .PostEventRequestEvent import PostEventRequestEvent - __all__ = ["PostEventRequest"] class PostEventRequest(BaseModel): """ PostEventRequest model - Request to create a new event + Request body for POST /v1/events (bare event object) """ model_config = { @@ -20,4 +18,48 @@ class PostEventRequest(BaseModel): "protected_namespaces": (), } - event: PostEventRequestEvent = Field(validation_alias="event") + project_id: Optional[str] = Field(validation_alias="project_id", default=None) + + source: Optional[str] = Field(validation_alias="source", default=None) + + event_name: Optional[str] = Field(validation_alias="event_name", default=None) + + event_type: str = Field(validation_alias="event_type") + + event_id: Optional[str] = Field(validation_alias="event_id", default=None) + + session_id: Optional[str] = Field(validation_alias="session_id", default=None) + + parent_id: Optional[str] = Field(validation_alias="parent_id", default=None) + + children_ids: Optional[List[str]] = Field( + validation_alias="children_ids", default=None + ) + + config: Optional[Dict[str, Any]] = Field(validation_alias="config", default=None) + + inputs: Dict[str, Any] = Field(validation_alias="inputs") + + outputs: Optional[Dict[str, Any]] = Field(validation_alias="outputs", default=None) + + error: Optional[str] = Field(validation_alias="error", default=None) + + start_time: Optional[float] = Field(validation_alias="start_time", default=None) + + end_time: Optional[float] = Field(validation_alias="end_time", default=None) + + duration: Optional[float] = Field(validation_alias="duration", default=None) + + metadata: Optional[Dict[str, Any]] = Field( + validation_alias="metadata", default=None + ) + + feedback: Optional[Dict[str, Any]] = Field( + validation_alias="feedback", default=None + ) + + metrics: Optional[Dict[str, Any]] = Field(validation_alias="metrics", default=None) + + user_properties: Optional[Dict[str, Any]] = Field( + validation_alias="user_properties", default=None + ) diff --git a/src/honeyhive/_generated/models/RunMetricRequest.py b/src/honeyhive/_generated/models/RunMetricRequest.py index 48e53652..8d7d06f0 100644 --- a/src/honeyhive/_generated/models/RunMetricRequest.py +++ b/src/honeyhive/_generated/models/RunMetricRequest.py @@ -11,7 +11,7 @@ class RunMetricRequest(BaseModel): """ RunMetricRequest model - Request body for POST /metrics/run_metric + Request body for POST /metrics/run """ model_config = { diff --git a/src/honeyhive/_generated/models/RunMetricResponse.py b/src/honeyhive/_generated/models/RunMetricResponse.py index 9789dee2..8bba4508 100644 --- a/src/honeyhive/_generated/models/RunMetricResponse.py +++ b/src/honeyhive/_generated/models/RunMetricResponse.py @@ -8,7 +8,7 @@ class RunMetricResponse(BaseModel): """ RunMetricResponse model - Response for POST /metrics/run_metric + Response for POST /metrics/run """ model_config = { diff --git a/src/honeyhive/_generated/models/UpdateEventRequest.py b/src/honeyhive/_generated/models/UpdateEventRequest.py index 5a0471cf..b7c6d689 100644 --- a/src/honeyhive/_generated/models/UpdateEventRequest.py +++ b/src/honeyhive/_generated/models/UpdateEventRequest.py @@ -8,7 +8,7 @@ class UpdateEventRequest(BaseModel): """ UpdateEventRequest model - Request to update an existing event + Request body for PUT /v1/events/{event_id} """ model_config = { @@ -18,8 +18,6 @@ class UpdateEventRequest(BaseModel): "protected_namespaces": (), } - event_id: str = Field(validation_alias="event_id") - metadata: Optional[Dict[str, Any]] = Field( validation_alias="metadata", default=None ) diff --git a/src/honeyhive/_generated/models/DeleteMetricQuery.py b/src/honeyhive/_generated/models/UpdateMetricParams.py similarity index 68% rename from src/honeyhive/_generated/models/DeleteMetricQuery.py rename to src/honeyhive/_generated/models/UpdateMetricParams.py index 092a4fe2..b461e991 100644 --- a/src/honeyhive/_generated/models/DeleteMetricQuery.py +++ b/src/honeyhive/_generated/models/UpdateMetricParams.py @@ -2,13 +2,13 @@ from pydantic import BaseModel, Field -__all__ = ["DeleteMetricQuery"] +__all__ = ["UpdateMetricParams"] -class DeleteMetricQuery(BaseModel): +class UpdateMetricParams(BaseModel): """ - DeleteMetricQuery model - Query parameters for DELETE /metrics + UpdateMetricParams model + Path parameters for PUT /metrics/{metric_id} """ model_config = { diff --git a/src/honeyhive/_generated/models/UpdateMetricRequest.py b/src/honeyhive/_generated/models/UpdateMetricRequest.py index f47d5426..a054d71f 100644 --- a/src/honeyhive/_generated/models/UpdateMetricRequest.py +++ b/src/honeyhive/_generated/models/UpdateMetricRequest.py @@ -2,7 +2,10 @@ from pydantic import BaseModel, Field +from .UpdateMetricRequestCategoriesItem import UpdateMetricRequestCategoriesItem +from .UpdateMetricRequestChildMetricsItem import UpdateMetricRequestChildMetricsItem from .UpdateMetricRequestFilters import UpdateMetricRequestFilters +from .UpdateMetricRequestThreshold import UpdateMetricRequestThreshold __all__ = ["UpdateMetricRequest"] @@ -10,7 +13,7 @@ class UpdateMetricRequest(BaseModel): """ UpdateMetricRequest model - Request body for PUT /metrics + Request body for PUT /metrics/{metric_id} """ model_config = { @@ -50,18 +53,18 @@ class UpdateMetricRequest(BaseModel): scale: Optional[int] = Field(validation_alias="scale", default=None) - threshold: Optional[Dict[str, Any]] = Field( + threshold: Optional[UpdateMetricRequestThreshold] = Field( validation_alias="threshold", default=None ) - categories: Optional[List[Any]] = Field(validation_alias="categories", default=None) + categories: Optional[List[Optional[UpdateMetricRequestCategoriesItem]]] = Field( + validation_alias="categories", default=None + ) - child_metrics: Optional[List[Any]] = Field( - validation_alias="child_metrics", default=None + child_metrics: Optional[List[Optional[UpdateMetricRequestChildMetricsItem]]] = ( + Field(validation_alias="child_metrics", default=None) ) filters: Optional[UpdateMetricRequestFilters] = Field( validation_alias="filters", default=None ) - - id: str = Field(validation_alias="id") diff --git a/src/honeyhive/_generated/models/UpdateMetricRequestThreshold.py b/src/honeyhive/_generated/models/UpdateMetricRequestThreshold.py new file mode 100644 index 00000000..78f22ccc --- /dev/null +++ b/src/honeyhive/_generated/models/UpdateMetricRequestThreshold.py @@ -0,0 +1,30 @@ +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field + +__all__ = ["UpdateMetricRequestThreshold"] + + +class UpdateMetricRequestThreshold(BaseModel): + """ + UpdateMetricRequestThreshold model + """ + + model_config = { + "populate_by_name": True, + "validate_assignment": True, + "extra": "allow", + "protected_namespaces": (), + } + + min: Optional[float] = Field(validation_alias="min", default=None) + + max: Optional[float] = Field(validation_alias="max", default=None) + + pass_when: Optional[Union[bool, float]] = Field( + validation_alias="pass_when", default=None + ) + + passing_categories: Optional[List[str]] = Field( + validation_alias="passing_categories", default=None + ) diff --git a/src/honeyhive/_generated/models/__init__.py b/src/honeyhive/_generated/models/__init__.py index d865a4b5..1319b06a 100644 --- a/src/honeyhive/_generated/models/__init__.py +++ b/src/honeyhive/_generated/models/__init__.py @@ -39,11 +39,11 @@ from .DeleteConfigurationResponse import * from .DeleteDatapointParams import * from .DeleteDatapointResponse import * -from .DeleteDatasetQuery import * +from .DeleteDatasetParams import * from .DeleteDatasetResponse import * from .DeleteExperimentRunParams import * from .DeleteExperimentRunResponse import * -from .DeleteMetricQuery import * +from .DeleteMetricParams import * from .DeleteMetricResponse import * from .DeleteResult import * from .Event import * @@ -74,10 +74,10 @@ from .GetDatasetsResponse import * from .GetEventsQuery import * from .GetEventsResponse import * -from .GetEventsSchemaDateRangeOneOf1 import * -from .GetEventsSchemaQuery import * +from .GetEventsSchemaLegacyDateRangeOneOf1 import * from .GetEventsSchemaResponse import * from .GetExperimentCompareEventsResponse import * +from .GetExperimentRunCompareEventsParams import * from .GetExperimentRunCompareEventsQuery import * from .GetExperimentRunCompareParams import * from .GetExperimentRunCompareQuery import * @@ -93,12 +93,38 @@ from .GetMetricsQuery import * from .GetMetricsResponse import * from .GetProjectsResponse import * +from .GetRunSchemaDateRangeOneOf1 import * +from .GetRunSchemaQuery import * from .GetRunsDateRangeOneOf1 import * +from .GetRunsSchemaDateRangeOneOf1 import * +from .GetRunsSchemaQuery import * from .InsertResult import * +from .LegacyDeleteDatasetQuery import * +from .LegacyDeleteMetricQuery import * from .LegacyEvent import * from .LegacyExportEventsRequest import * from .LegacyExportEventsRequestDateRange import * +from .LegacyGetEventsSchemaQuery import * +from .LegacyGetExperimentRunCompareEventsQuery import * +from .LegacyGetExperimentRunCompareParams import * +from .LegacyGetExperimentRunCompareQuery import * +from .LegacyPostEventBatchRequest import * +from .LegacyPostEventRequest import * +from .LegacyPostEventRequestEvent import * +from .LegacyRemoveDatapointFromDatasetParams import * +from .LegacyRunMetricRequest import * +from .LegacyRunMetricRequestEvent import * +from .LegacyRunMetricRequestEventFeedback import * +from .LegacyRunMetricRequestMetric import * +from .LegacyRunMetricRequestMetricCategoriesItem import * +from .LegacyRunMetricRequestMetricChildMetricsItem import * +from .LegacyRunMetricRequestMetricFilters import * from .LegacyUpdateDatasetRequest import * +from .LegacyUpdateEventRequest import * +from .LegacyUpdateMetricRequest import * +from .LegacyUpdateMetricRequestCategoriesItem import * +from .LegacyUpdateMetricRequestChildMetricsItem import * +from .LegacyUpdateMetricRequestFilters import * from .MetricComparison import * from .MetricDatapoints import * from .MetricDetail import * @@ -113,7 +139,6 @@ from .PostEventBatchRequest import * from .PostEventBatchResponse import * from .PostEventRequest import * -from .PostEventRequestEvent import * from .PostEventResponse import * from .PostExperimentRunRequest import * from .PostExperimentRunResponse import * @@ -164,8 +189,10 @@ from .UpdateDatasetRequest import * from .UpdateDatasetResponse import * from .UpdateEventRequest import * +from .UpdateMetricParams import * from .UpdateMetricRequest import * from .UpdateMetricRequestCategoriesItem import * from .UpdateMetricRequestChildMetricsItem import * from .UpdateMetricRequestFilters import * +from .UpdateMetricRequestThreshold import * from .UpdateMetricResponse import * diff --git a/src/honeyhive/_generated/services/Datasets_service.py b/src/honeyhive/_generated/services/Datasets_service.py index 37fd1eca..4940d4ce 100644 --- a/src/honeyhive/_generated/services/Datasets_service.py +++ b/src/honeyhive/_generated/services/Datasets_service.py @@ -114,7 +114,7 @@ def updateDatasetLegacy( ) -def deleteDataset( +def deleteDatasetLegacy( api_config_override: Optional[APIConfig] = None, *, dataset_id: str ) -> DeleteDatasetResponse: api_config = api_config_override if api_config_override else APIConfig() @@ -139,7 +139,7 @@ def deleteDataset( if response.status_code != 200: raise HTTPException( response.status_code, - f"deleteDataset failed with status code: {response.status_code}", + f"deleteDatasetLegacy failed with status code: {response.status_code}", ) else: body = None if 200 == 204 else response.json() @@ -188,6 +188,41 @@ def updateDataset( ) +def deleteDataset( + api_config_override: Optional[APIConfig] = None, *, dataset_id: str +) -> DeleteDatasetResponse: + api_config = api_config_override if api_config_override else APIConfig() + + base_path = api_config.base_path + path = f"/v1/datasets/{dataset_id}" + headers = api_config.get_default_headers() + query_params: Dict[str, Any] = {} + + query_params = { + key: value for (key, value) in query_params.items() if value is not None + } + + with httpx.Client(base_url=base_path, verify=api_config.verify) as client: + response = client.request( + "delete", + httpx.URL(path), + headers=headers, + params=_serialize_query_params(query_params), + ) + + if response.status_code != 200: + raise HTTPException( + response.status_code, + f"deleteDataset failed with status code: {response.status_code}", + ) + else: + body = None if 200 == 204 else response.json() + + return ( + DeleteDatasetResponse(**body) if body is not None else DeleteDatasetResponse() + ) + + def addDatapoints( api_config_override: Optional[APIConfig] = None, *, @@ -236,7 +271,7 @@ def removeDatapoint( api_config = api_config_override if api_config_override else APIConfig() base_path = api_config.base_path - path = f"/v1/datasets/{dataset_id}/{datapoint_id}" + path = f"/v1/datasets/{dataset_id}/datapoints/{datapoint_id}" headers = api_config.get_default_headers() query_params: Dict[str, Any] = {} @@ -265,3 +300,43 @@ def removeDatapoint( if body is not None else RemoveDatapointResponse() ) + + +def removeDatapointLegacy( + api_config_override: Optional[APIConfig] = None, + *, + dataset_id: str, + datapoint_id: str, +) -> RemoveDatapointResponse: + api_config = api_config_override if api_config_override else APIConfig() + + base_path = api_config.base_path + path = f"/v1/datasets/{dataset_id}/{datapoint_id}" + headers = api_config.get_default_headers() + query_params: Dict[str, Any] = {} + + query_params = { + key: value for (key, value) in query_params.items() if value is not None + } + + with httpx.Client(base_url=base_path, verify=api_config.verify) as client: + response = client.request( + "delete", + httpx.URL(path), + headers=headers, + params=_serialize_query_params(query_params), + ) + + if response.status_code != 200: + raise HTTPException( + response.status_code, + f"removeDatapointLegacy failed with status code: {response.status_code}", + ) + else: + body = None if 200 == 204 else response.json() + + return ( + RemoveDatapointResponse(**body) + if body is not None + else RemoveDatapointResponse() + ) diff --git a/src/honeyhive/_generated/services/Events_service.py b/src/honeyhive/_generated/services/Events_service.py index a2a08bda..91b39b4e 100644 --- a/src/honeyhive/_generated/services/Events_service.py +++ b/src/honeyhive/_generated/services/Events_service.py @@ -6,13 +6,81 @@ from ..models import * +def createEventLegacy( + api_config_override: Optional[APIConfig] = None, *, data: LegacyPostEventRequest +) -> PostEventResponse: + api_config = api_config_override if api_config_override else APIConfig() + + base_path = api_config.base_path + path = f"/events" + headers = api_config.get_default_headers() + query_params: Dict[str, Any] = {} + + query_params = { + key: value for (key, value) in query_params.items() if value is not None + } + + with httpx.Client(base_url=base_path, verify=api_config.verify) as client: + response = client.request( + "post", + httpx.URL(path), + headers=headers, + params=_serialize_query_params(query_params), + json=data.model_dump(exclude_none=True), + ) + + if response.status_code != 200: + raise HTTPException( + response.status_code, + f"createEventLegacy failed with status code: {response.status_code}", + ) + else: + body = None if 200 == 204 else response.json() + + return PostEventResponse(**body) if body is not None else PostEventResponse() + + +def updateEventLegacy( + api_config_override: Optional[APIConfig] = None, *, data: LegacyUpdateEventRequest +) -> None: + api_config = api_config_override if api_config_override else APIConfig() + + base_path = api_config.base_path + path = f"/events" + headers = api_config.get_default_headers() + query_params: Dict[str, Any] = {} + + query_params = { + key: value for (key, value) in query_params.items() if value is not None + } + + with httpx.Client(base_url=base_path, verify=api_config.verify) as client: + response = client.request( + "put", + httpx.URL(path), + headers=headers, + params=_serialize_query_params(query_params), + json=data.model_dump(exclude_none=True), + ) + + if response.status_code != 200: + raise HTTPException( + response.status_code, + f"updateEventLegacy failed with status code: {response.status_code}", + ) + else: + body = None if 200 == 204 else response.json() + + return None + + def createEvent( api_config_override: Optional[APIConfig] = None, *, data: PostEventRequest ) -> PostEventResponse: api_config = api_config_override if api_config_override else APIConfig() base_path = api_config.base_path - path = f"/events" + path = f"/v1/events" headers = api_config.get_default_headers() query_params: Dict[str, Any] = {} @@ -41,12 +109,15 @@ def createEvent( def updateEvent( - api_config_override: Optional[APIConfig] = None, *, data: UpdateEventRequest + api_config_override: Optional[APIConfig] = None, + *, + event_id: str, + data: UpdateEventRequest, ) -> None: api_config = api_config_override if api_config_override else APIConfig() base_path = api_config.base_path - path = f"/events" + path = f"/v1/events/{event_id}" headers = api_config.get_default_headers() query_params: Dict[str, Any] = {} @@ -142,6 +213,42 @@ def searchEvents( return ExportEventsResponse(**body) if body is not None else ExportEventsResponse() +def createEventBatch( + api_config_override: Optional[APIConfig] = None, *, data: PostEventBatchRequest +) -> PostEventBatchResponse: + api_config = api_config_override if api_config_override else APIConfig() + + base_path = api_config.base_path + path = f"/v1/events/batch" + headers = api_config.get_default_headers() + query_params: Dict[str, Any] = {} + + query_params = { + key: value for (key, value) in query_params.items() if value is not None + } + + with httpx.Client(base_url=base_path, verify=api_config.verify) as client: + response = client.request( + "post", + httpx.URL(path), + headers=headers, + params=_serialize_query_params(query_params), + json=data.model_dump(exclude_none=True), + ) + + if response.status_code != 200: + raise HTTPException( + response.status_code, + f"createEventBatch failed with status code: {response.status_code}", + ) + else: + body = None if 200 == 204 else response.json() + + return ( + PostEventBatchResponse(**body) if body is not None else PostEventBatchResponse() + ) + + def createModelEvent( api_config_override: Optional[APIConfig] = None, *, data: PostModelEventRequest ) -> PostEventResponse: @@ -176,8 +283,10 @@ def createModelEvent( return PostEventResponse(**body) if body is not None else PostEventResponse() -def createEventBatch( - api_config_override: Optional[APIConfig] = None, *, data: PostEventBatchRequest +def createEventBatchLegacy( + api_config_override: Optional[APIConfig] = None, + *, + data: LegacyPostEventBatchRequest, ) -> PostEventBatchResponse: api_config = api_config_override if api_config_override else APIConfig() @@ -202,7 +311,7 @@ def createEventBatch( if response.status_code != 200: raise HTTPException( response.status_code, - f"createEventBatch failed with status code: {response.status_code}", + f"createEventBatchLegacy failed with status code: {response.status_code}", ) else: body = None if 200 == 204 else response.json() @@ -248,10 +357,10 @@ def createModelEventBatch( ) -def getEventsSchema( +def getEventsSchemaLegacy( api_config_override: Optional[APIConfig] = None, *, - dateRange: Optional[Union[str, GetEventsSchemaDateRangeOneOf1]] = None, + dateRange: Optional[Union[str, GetEventsSchemaLegacyDateRangeOneOf1]] = None, evaluation_id: Optional[str] = None, ) -> GetEventsSchemaResponse: api_config = api_config_override if api_config_override else APIConfig() @@ -279,7 +388,7 @@ def getEventsSchema( if response.status_code != 200: raise HTTPException( response.status_code, - f"getEventsSchema failed with status code: {response.status_code}", + f"getEventsSchemaLegacy failed with status code: {response.status_code}", ) else: body = None if 200 == 204 else response.json() diff --git a/src/honeyhive/_generated/services/Experiments_service.py b/src/honeyhive/_generated/services/Experiments_service.py index a392868a..65f45d72 100644 --- a/src/honeyhive/_generated/services/Experiments_service.py +++ b/src/honeyhive/_generated/services/Experiments_service.py @@ -101,6 +101,45 @@ def createRun( ) +def getRunsSchema( + api_config_override: Optional[APIConfig] = None, + *, + dateRange: Optional[Union[str, GetRunsSchemaDateRangeOneOf1]] = None, +) -> GetEventsSchemaResponse: + api_config = api_config_override if api_config_override else APIConfig() + + base_path = api_config.base_path + path = f"/v1/runs/schema" + headers = api_config.get_default_headers() + query_params: Dict[str, Any] = {"dateRange": dateRange} + + query_params = { + key: value for (key, value) in query_params.items() if value is not None + } + + with httpx.Client(base_url=base_path, verify=api_config.verify) as client: + response = client.request( + "get", + httpx.URL(path), + headers=headers, + params=_serialize_query_params(query_params), + ) + + if response.status_code != 200: + raise HTTPException( + response.status_code, + f"getRunsSchema failed with status code: {response.status_code}", + ) + else: + body = None if 200 == 204 else response.json() + + return ( + GetEventsSchemaResponse(**body) + if body is not None + else GetEventsSchemaResponse() + ) + + def getRun( api_config_override: Optional[APIConfig] = None, *, run_id: str ) -> GetExperimentRunResponse: @@ -216,6 +255,46 @@ def deleteRun( ) +def getRunSchema( + api_config_override: Optional[APIConfig] = None, + *, + run_id: str, + dateRange: Optional[Union[str, GetRunSchemaDateRangeOneOf1]] = None, +) -> GetEventsSchemaResponse: + api_config = api_config_override if api_config_override else APIConfig() + + base_path = api_config.base_path + path = f"/v1/runs/{run_id}/schema" + headers = api_config.get_default_headers() + query_params: Dict[str, Any] = {"dateRange": dateRange} + + query_params = { + key: value for (key, value) in query_params.items() if value is not None + } + + with httpx.Client(base_url=base_path, verify=api_config.verify) as client: + response = client.request( + "get", + httpx.URL(path), + headers=headers, + params=_serialize_query_params(query_params), + ) + + if response.status_code != 200: + raise HTTPException( + response.status_code, + f"getRunSchema failed with status code: {response.status_code}", + ) + else: + body = None if 200 == 204 else response.json() + + return ( + GetEventsSchemaResponse(**body) + if body is not None + else GetEventsSchemaResponse() + ) + + def getExperimentRunMetrics( api_config_override: Optional[APIConfig] = None, *, @@ -312,7 +391,7 @@ def getExperimentComparison( api_config = api_config_override if api_config_override else APIConfig() base_path = api_config.base_path - path = f"/v1/runs/{new_run_id}/compare-with/{old_run_id}" + path = f"/v1/runs/{new_run_id}/compare/{old_run_id}" headers = api_config.get_default_headers() query_params: Dict[str, Any] = { "aggregate_function": aggregate_function, @@ -346,7 +425,103 @@ def getExperimentComparison( ) +def getExperimentComparisonLegacy( + api_config_override: Optional[APIConfig] = None, + *, + new_run_id: str, + old_run_id: str, + aggregate_function: Optional[str] = None, + filters: Optional[Union[str, List[Dict[str, Any]]]] = None, +) -> GetExperimentRunCompareResponse: + api_config = api_config_override if api_config_override else APIConfig() + + base_path = api_config.base_path + path = f"/v1/runs/{new_run_id}/compare-with/{old_run_id}" + headers = api_config.get_default_headers() + query_params: Dict[str, Any] = { + "aggregate_function": aggregate_function, + "filters": filters, + } + + query_params = { + key: value for (key, value) in query_params.items() if value is not None + } + + with httpx.Client(base_url=base_path, verify=api_config.verify) as client: + response = client.request( + "get", + httpx.URL(path), + headers=headers, + params=_serialize_query_params(query_params), + ) + + if response.status_code != 200: + raise HTTPException( + response.status_code, + f"getExperimentComparisonLegacy failed with status code: {response.status_code}", + ) + else: + body = None if 200 == 204 else response.json() + + return ( + GetExperimentRunCompareResponse(**body) + if body is not None + else GetExperimentRunCompareResponse() + ) + + def getExperimentCompareEvents( + api_config_override: Optional[APIConfig] = None, + *, + new_run_id: str, + old_run_id: str, + event_name: Optional[str] = None, + event_type: Optional[str] = None, + filter: Optional[Union[str, Dict[str, Any]]] = None, + limit: Optional[int] = None, + page: Optional[int] = None, +) -> GetExperimentCompareEventsResponse: + api_config = api_config_override if api_config_override else APIConfig() + + base_path = api_config.base_path + path = f"/v1/runs/{new_run_id}/compare/{old_run_id}/events" + headers = api_config.get_default_headers() + query_params: Dict[str, Any] = { + "event_name": event_name, + "event_type": event_type, + "filter": filter, + "limit": limit, + "page": page, + } + + query_params = { + key: value for (key, value) in query_params.items() if value is not None + } + + with httpx.Client(base_url=base_path, verify=api_config.verify) as client: + response = client.request( + "get", + httpx.URL(path), + headers=headers, + params=_serialize_query_params(query_params), + ) + + if response.status_code != 200: + raise HTTPException( + response.status_code, + f"getExperimentCompareEvents failed with status code: {response.status_code}", + ) + else: + body = None if 200 == 204 else response.json() + + return ( + GetExperimentCompareEventsResponse(**body) + if body is not None + else GetExperimentCompareEventsResponse() + ) + + +def getExperimentCompareEventsLegacy( api_config_override: Optional[APIConfig] = None, *, run_id_1: str, @@ -387,7 +562,7 @@ def getExperimentCompareEvents( if response.status_code != 200: raise HTTPException( response.status_code, - f"getExperimentCompareEvents failed with status code: {response.status_code}", + f"getExperimentCompareEventsLegacy failed with status code: {response.status_code}", ) else: body = None if 200 == 204 else response.json() diff --git a/src/honeyhive/_generated/services/Metrics_service.py b/src/honeyhive/_generated/services/Metrics_service.py index 658ba8a8..fd46048e 100644 --- a/src/honeyhive/_generated/services/Metrics_service.py +++ b/src/honeyhive/_generated/services/Metrics_service.py @@ -76,8 +76,8 @@ def createMetric( return CreateMetricResponse(**body) if body is not None else CreateMetricResponse() -def updateMetric( - api_config_override: Optional[APIConfig] = None, *, data: UpdateMetricRequest +def updateMetricLegacy( + api_config_override: Optional[APIConfig] = None, *, data: LegacyUpdateMetricRequest ) -> UpdateMetricResponse: api_config = api_config_override if api_config_override else APIConfig() @@ -102,7 +102,7 @@ def updateMetric( if response.status_code != 200: raise HTTPException( response.status_code, - f"updateMetric failed with status code: {response.status_code}", + f"updateMetricLegacy failed with status code: {response.status_code}", ) else: body = None if 200 == 204 else response.json() @@ -110,7 +110,7 @@ def updateMetric( return UpdateMetricResponse(**body) if body is not None else UpdateMetricResponse() -def deleteMetric( +def deleteMetricLegacy( api_config_override: Optional[APIConfig] = None, *, metric_id: str ) -> DeleteMetricResponse: api_config = api_config_override if api_config_override else APIConfig() @@ -124,6 +124,76 @@ def deleteMetric( key: value for (key, value) in query_params.items() if value is not None } + with httpx.Client(base_url=base_path, verify=api_config.verify) as client: + response = client.request( + "delete", + httpx.URL(path), + headers=headers, + params=_serialize_query_params(query_params), + ) + + if response.status_code != 200: + raise HTTPException( + response.status_code, + f"deleteMetricLegacy failed with status code: {response.status_code}", + ) + else: + body = None if 200 == 204 else response.json() + + return DeleteMetricResponse(**body) if body is not None else DeleteMetricResponse() + + +def updateMetric( + api_config_override: Optional[APIConfig] = None, + *, + metric_id: str, + data: UpdateMetricRequest, +) -> UpdateMetricResponse: + api_config = api_config_override if api_config_override else APIConfig() + + base_path = api_config.base_path + path = f"/v1/metrics/{metric_id}" + headers = api_config.get_default_headers() + query_params: Dict[str, Any] = {} + + query_params = { + key: value for (key, value) in query_params.items() if value is not None + } + + with httpx.Client(base_url=base_path, verify=api_config.verify) as client: + response = client.request( + "put", + httpx.URL(path), + headers=headers, + params=_serialize_query_params(query_params), + json=data.model_dump(exclude_none=True), + ) + + if response.status_code != 200: + raise HTTPException( + response.status_code, + f"updateMetric failed with status code: {response.status_code}", + ) + else: + body = None if 200 == 204 else response.json() + + return UpdateMetricResponse(**body) if body is not None else UpdateMetricResponse() + + +def deleteMetric( + api_config_override: Optional[APIConfig] = None, *, metric_id: str +) -> DeleteMetricResponse: + api_config = api_config_override if api_config_override else APIConfig() + + base_path = api_config.base_path + path = f"/v1/metrics/{metric_id}" + headers = api_config.get_default_headers() + query_params: Dict[str, Any] = {} + + query_params = { + key: value for (key, value) in query_params.items() if value is not None + } + with httpx.Client(base_url=base_path, verify=api_config.verify) as client: response = client.request( "delete", @@ -149,7 +219,7 @@ def runMetric( api_config = api_config_override if api_config_override else APIConfig() base_path = api_config.base_path - path = f"/v1/metrics/run_metric" + path = f"/v1/metrics/run" headers = api_config.get_default_headers() query_params: Dict[str, Any] = {} @@ -175,3 +245,37 @@ def runMetric( body = None if 200 == 204 else response.json() return RunMetricResponse(**body) if body is not None else RunMetricResponse() + + +def runMetricLegacy( + api_config_override: Optional[APIConfig] = None, *, data: LegacyRunMetricRequest +) -> RunMetricResponse: + api_config = api_config_override if api_config_override else APIConfig() + + base_path = api_config.base_path + path = f"/v1/metrics/run_metric" + headers = api_config.get_default_headers() + query_params: Dict[str, Any] = {} + + query_params = { + key: value for (key, value) in query_params.items() if value is not None + } + + with httpx.Client(base_url=base_path, verify=api_config.verify) as client: + response = client.request( + "post", + httpx.URL(path), + headers=headers, + params=_serialize_query_params(query_params), + json=data.model_dump(exclude_none=True), + ) + + if response.status_code != 200: + raise HTTPException( + response.status_code, + f"runMetricLegacy failed with status code: {response.status_code}", + ) + else: + body = None if 200 == 204 else response.json() + + return RunMetricResponse(**body) if body is not None else RunMetricResponse() diff --git a/src/honeyhive/_generated/services/async_Datasets_service.py b/src/honeyhive/_generated/services/async_Datasets_service.py index 05373f84..8e23c1a9 100644 --- a/src/honeyhive/_generated/services/async_Datasets_service.py +++ b/src/honeyhive/_generated/services/async_Datasets_service.py @@ -120,7 +120,7 @@ async def updateDatasetLegacy( ) -async def deleteDataset( +async def deleteDatasetLegacy( api_config_override: Optional[APIConfig] = None, *, dataset_id: str ) -> DeleteDatasetResponse: api_config = api_config_override if api_config_override else APIConfig() @@ -147,7 +147,7 @@ async def deleteDataset( if response.status_code != 200: raise HTTPException( response.status_code, - f"deleteDataset failed with status code: {response.status_code}", + f"deleteDatasetLegacy failed with status code: {response.status_code}", ) else: body = None if 200 == 204 else response.json() @@ -198,6 +198,43 @@ async def updateDataset( ) +async def deleteDataset( + api_config_override: Optional[APIConfig] = None, *, dataset_id: str +) -> DeleteDatasetResponse: + api_config = api_config_override if api_config_override else APIConfig() + + base_path = api_config.base_path + path = f"/v1/datasets/{dataset_id}" + headers = api_config.get_default_headers() + query_params: Dict[str, Any] = {} + + query_params = { + key: value for (key, value) in query_params.items() if value is not None + } + + async with httpx.AsyncClient( + base_url=base_path, verify=api_config.verify + ) as client: + response = await client.request( + "delete", + httpx.URL(path), + headers=headers, + params=_serialize_query_params(query_params), + ) + + if response.status_code != 200: + raise HTTPException( + response.status_code, + f"deleteDataset failed with status code: {response.status_code}", + ) + else: + body = None if 200 == 204 else response.json() + + return ( + DeleteDatasetResponse(**body) if body is not None else DeleteDatasetResponse() + ) + + async def addDatapoints( api_config_override: Optional[APIConfig] = None, *, @@ -248,7 +285,7 @@ async def removeDatapoint( api_config = api_config_override if api_config_override else APIConfig() base_path = api_config.base_path - path = f"/v1/datasets/{dataset_id}/{datapoint_id}" + path = f"/v1/datasets/{dataset_id}/datapoints/{datapoint_id}" headers = api_config.get_default_headers() query_params: Dict[str, Any] = {} @@ -279,3 +316,45 @@ async def removeDatapoint( if body is not None else RemoveDatapointResponse() ) + + +async def removeDatapointLegacy( + api_config_override: Optional[APIConfig] = None, + *, + dataset_id: str, + datapoint_id: str, +) -> RemoveDatapointResponse: + api_config = api_config_override if api_config_override else APIConfig() + + base_path = api_config.base_path + path = f"/v1/datasets/{dataset_id}/{datapoint_id}" + headers = api_config.get_default_headers() + query_params: Dict[str, Any] = {} + + query_params = { + key: value for (key, value) in query_params.items() if value is not None + } + + async with httpx.AsyncClient( + base_url=base_path, verify=api_config.verify + ) as client: + response = await client.request( + "delete", + httpx.URL(path), + headers=headers, + params=_serialize_query_params(query_params), + ) + + if response.status_code != 200: + raise HTTPException( + response.status_code, + f"removeDatapointLegacy failed with status code: {response.status_code}", + ) + else: + body = None if 200 == 204 else response.json() + + return ( + RemoveDatapointResponse(**body) + if body is not None + else RemoveDatapointResponse() + ) diff --git a/src/honeyhive/_generated/services/async_Events_service.py b/src/honeyhive/_generated/services/async_Events_service.py index 943eae81..13b58276 100644 --- a/src/honeyhive/_generated/services/async_Events_service.py +++ b/src/honeyhive/_generated/services/async_Events_service.py @@ -6,13 +6,85 @@ from ..models import * +async def createEventLegacy( + api_config_override: Optional[APIConfig] = None, *, data: LegacyPostEventRequest +) -> PostEventResponse: + api_config = api_config_override if api_config_override else APIConfig() + + base_path = api_config.base_path + path = f"/events" + headers = api_config.get_default_headers() + query_params: Dict[str, Any] = {} + + query_params = { + key: value for (key, value) in query_params.items() if value is not None + } + + async with httpx.AsyncClient( + base_url=base_path, verify=api_config.verify + ) as client: + response = await client.request( + "post", + httpx.URL(path), + headers=headers, + params=_serialize_query_params(query_params), + json=data.model_dump(exclude_none=True), + ) + + if response.status_code != 200: + raise HTTPException( + response.status_code, + f"createEventLegacy failed with status code: {response.status_code}", + ) + else: + body = None if 200 == 204 else response.json() + + return PostEventResponse(**body) if body is not None else PostEventResponse() + + +async def updateEventLegacy( + api_config_override: Optional[APIConfig] = None, *, data: LegacyUpdateEventRequest +) -> None: + api_config = api_config_override if api_config_override else APIConfig() + + base_path = api_config.base_path + path = f"/events" + headers = api_config.get_default_headers() + query_params: Dict[str, Any] = {} + + query_params = { + key: value for (key, value) in query_params.items() if value is not None + } + + async with httpx.AsyncClient( + base_url=base_path, verify=api_config.verify + ) as client: + response = await client.request( + "put", + httpx.URL(path), + headers=headers, + params=_serialize_query_params(query_params), + json=data.model_dump(exclude_none=True), + ) + + if response.status_code != 200: + raise HTTPException( + response.status_code, + f"updateEventLegacy failed with status code: {response.status_code}", + ) + else: + body = None if 200 == 204 else response.json() + + return None + + async def createEvent( api_config_override: Optional[APIConfig] = None, *, data: PostEventRequest ) -> PostEventResponse: api_config = api_config_override if api_config_override else APIConfig() base_path = api_config.base_path - path = f"/events" + path = f"/v1/events" headers = api_config.get_default_headers() query_params: Dict[str, Any] = {} @@ -43,12 +115,15 @@ async def createEvent( async def updateEvent( - api_config_override: Optional[APIConfig] = None, *, data: UpdateEventRequest + api_config_override: Optional[APIConfig] = None, + *, + event_id: str, + data: UpdateEventRequest, ) -> None: api_config = api_config_override if api_config_override else APIConfig() base_path = api_config.base_path - path = f"/events" + path = f"/v1/events/{event_id}" headers = api_config.get_default_headers() query_params: Dict[str, Any] = {} @@ -150,6 +225,44 @@ async def searchEvents( return ExportEventsResponse(**body) if body is not None else ExportEventsResponse() +async def createEventBatch( + api_config_override: Optional[APIConfig] = None, *, data: PostEventBatchRequest +) -> PostEventBatchResponse: + api_config = api_config_override if api_config_override else APIConfig() + + base_path = api_config.base_path + path = f"/v1/events/batch" + headers = api_config.get_default_headers() + query_params: Dict[str, Any] = {} + + query_params = { + key: value for (key, value) in query_params.items() if value is not None + } + + async with httpx.AsyncClient( + base_url=base_path, verify=api_config.verify + ) as client: + response = await client.request( + "post", + httpx.URL(path), + headers=headers, + params=_serialize_query_params(query_params), + json=data.model_dump(exclude_none=True), + ) + + if response.status_code != 200: + raise HTTPException( + response.status_code, + f"createEventBatch failed with status code: {response.status_code}", + ) + else: + body = None if 200 == 204 else response.json() + + return ( + PostEventBatchResponse(**body) if body is not None else PostEventBatchResponse() + ) + + async def createModelEvent( api_config_override: Optional[APIConfig] = None, *, data: PostModelEventRequest ) -> PostEventResponse: @@ -186,8 +299,10 @@ async def createModelEvent( return PostEventResponse(**body) if body is not None else PostEventResponse() -async def createEventBatch( - api_config_override: Optional[APIConfig] = None, *, data: PostEventBatchRequest +async def createEventBatchLegacy( + api_config_override: Optional[APIConfig] = None, + *, + data: LegacyPostEventBatchRequest, ) -> PostEventBatchResponse: api_config = api_config_override if api_config_override else APIConfig() @@ -214,7 +329,7 @@ async def createEventBatch( if response.status_code != 200: raise HTTPException( response.status_code, - f"createEventBatch failed with status code: {response.status_code}", + f"createEventBatchLegacy failed with status code: {response.status_code}", ) else: body = None if 200 == 204 else response.json() @@ -262,10 +377,10 @@ async def createModelEventBatch( ) -async def getEventsSchema( +async def getEventsSchemaLegacy( api_config_override: Optional[APIConfig] = None, *, - dateRange: Optional[Union[str, GetEventsSchemaDateRangeOneOf1]] = None, + dateRange: Optional[Union[str, GetEventsSchemaLegacyDateRangeOneOf1]] = None, evaluation_id: Optional[str] = None, ) -> GetEventsSchemaResponse: api_config = api_config_override if api_config_override else APIConfig() @@ -295,7 +410,7 @@ async def getEventsSchema( if response.status_code != 200: raise HTTPException( response.status_code, - f"getEventsSchema failed with status code: {response.status_code}", + f"getEventsSchemaLegacy failed with status code: {response.status_code}", ) else: body = None if 200 == 204 else response.json() diff --git a/src/honeyhive/_generated/services/async_Experiments_service.py b/src/honeyhive/_generated/services/async_Experiments_service.py index 0ab3c564..d225cda3 100644 --- a/src/honeyhive/_generated/services/async_Experiments_service.py +++ b/src/honeyhive/_generated/services/async_Experiments_service.py @@ -105,6 +105,47 @@ async def createRun( ) +async def getRunsSchema( + api_config_override: Optional[APIConfig] = None, + *, + dateRange: Optional[Union[str, GetRunsSchemaDateRangeOneOf1]] = None, +) -> GetEventsSchemaResponse: + api_config = api_config_override if api_config_override else APIConfig() + + base_path = api_config.base_path + path = f"/v1/runs/schema" + headers = api_config.get_default_headers() + query_params: Dict[str, Any] = {"dateRange": dateRange} + + query_params = { + key: value for (key, value) in query_params.items() if value is not None + } + + async with httpx.AsyncClient( + base_url=base_path, verify=api_config.verify + ) as client: + response = await client.request( + "get", + httpx.URL(path), + headers=headers, + params=_serialize_query_params(query_params), + ) + + if response.status_code != 200: + raise HTTPException( + response.status_code, + f"getRunsSchema failed with status code: {response.status_code}", + ) + else: + body = None if 200 == 204 else response.json() + + return ( + GetEventsSchemaResponse(**body) + if body is not None + else GetEventsSchemaResponse() + ) + + async def getRun( api_config_override: Optional[APIConfig] = None, *, run_id: str ) -> GetExperimentRunResponse: @@ -226,6 +267,48 @@ async def deleteRun( ) +async def getRunSchema( + api_config_override: Optional[APIConfig] = None, + *, + run_id: str, + dateRange: Optional[Union[str, GetRunSchemaDateRangeOneOf1]] = None, +) -> GetEventsSchemaResponse: + api_config = api_config_override if api_config_override else APIConfig() + + base_path = api_config.base_path + path = f"/v1/runs/{run_id}/schema" + headers = api_config.get_default_headers() + query_params: Dict[str, Any] = {"dateRange": dateRange} + + query_params = { + key: value for (key, value) in query_params.items() if value is not None + } + + async with httpx.AsyncClient( + base_url=base_path, verify=api_config.verify + ) as client: + response = await client.request( + "get", + httpx.URL(path), + headers=headers, + params=_serialize_query_params(query_params), + ) + + if response.status_code != 200: + raise HTTPException( + response.status_code, + f"getRunSchema failed with status code: {response.status_code}", + ) + else: + body = None if 200 == 204 else response.json() + + return ( + GetEventsSchemaResponse(**body) + if body is not None + else GetEventsSchemaResponse() + ) + + async def getExperimentRunMetrics( api_config_override: Optional[APIConfig] = None, *, @@ -326,7 +409,7 @@ async def getExperimentComparison( api_config = api_config_override if api_config_override else APIConfig() base_path = api_config.base_path - path = f"/v1/runs/{new_run_id}/compare-with/{old_run_id}" + path = f"/v1/runs/{new_run_id}/compare/{old_run_id}" headers = api_config.get_default_headers() query_params: Dict[str, Any] = { "aggregate_function": aggregate_function, @@ -362,7 +445,107 @@ async def getExperimentComparison( ) +async def getExperimentComparisonLegacy( + api_config_override: Optional[APIConfig] = None, + *, + new_run_id: str, + old_run_id: str, + aggregate_function: Optional[str] = None, + filters: Optional[Union[str, List[Dict[str, Any]]]] = None, +) -> GetExperimentRunCompareResponse: + api_config = api_config_override if api_config_override else APIConfig() + + base_path = api_config.base_path + path = f"/v1/runs/{new_run_id}/compare-with/{old_run_id}" + headers = api_config.get_default_headers() + query_params: Dict[str, Any] = { + "aggregate_function": aggregate_function, + "filters": filters, + } + + query_params = { + key: value for (key, value) in query_params.items() if value is not None + } + + async with httpx.AsyncClient( + base_url=base_path, verify=api_config.verify + ) as client: + response = await client.request( + "get", + httpx.URL(path), + headers=headers, + params=_serialize_query_params(query_params), + ) + + if response.status_code != 200: + raise HTTPException( + response.status_code, + f"getExperimentComparisonLegacy failed with status code: {response.status_code}", + ) + else: + body = None if 200 == 204 else response.json() + + return ( + GetExperimentRunCompareResponse(**body) + if body is not None + else GetExperimentRunCompareResponse() + ) + + async def getExperimentCompareEvents( + api_config_override: Optional[APIConfig] = None, + *, + new_run_id: str, + old_run_id: str, + event_name: Optional[str] = None, + event_type: Optional[str] = None, + filter: Optional[Union[str, Dict[str, Any]]] = None, + limit: Optional[int] = None, + page: Optional[int] = None, +) -> GetExperimentCompareEventsResponse: + api_config = api_config_override if api_config_override else APIConfig() + + base_path = api_config.base_path + path = f"/v1/runs/{new_run_id}/compare/{old_run_id}/events" + headers = api_config.get_default_headers() + query_params: Dict[str, Any] = { + "event_name": event_name, + "event_type": event_type, + "filter": filter, + "limit": limit, + "page": page, + } + + query_params = { + key: value for (key, value) in query_params.items() if value is not None + } + + async with httpx.AsyncClient( + base_url=base_path, verify=api_config.verify + ) as client: + response = await client.request( + "get", + httpx.URL(path), + headers=headers, + params=_serialize_query_params(query_params), + ) + + if response.status_code != 200: + raise HTTPException( + response.status_code, + f"getExperimentCompareEvents failed with status code: {response.status_code}", + ) + else: + body = None if 200 == 204 else response.json() + + return ( + GetExperimentCompareEventsResponse(**body) + if body is not None + else GetExperimentCompareEventsResponse() + ) + + +async def getExperimentCompareEventsLegacy( api_config_override: Optional[APIConfig] = None, *, run_id_1: str, @@ -405,7 +588,7 @@ async def getExperimentCompareEvents( if response.status_code != 200: raise HTTPException( response.status_code, - f"getExperimentCompareEvents failed with status code: {response.status_code}", + f"getExperimentCompareEventsLegacy failed with status code: {response.status_code}", ) else: body = None if 200 == 204 else response.json() diff --git a/src/honeyhive/_generated/services/async_Metrics_service.py b/src/honeyhive/_generated/services/async_Metrics_service.py index aef7eb7c..566dccbc 100644 --- a/src/honeyhive/_generated/services/async_Metrics_service.py +++ b/src/honeyhive/_generated/services/async_Metrics_service.py @@ -80,8 +80,8 @@ async def createMetric( return CreateMetricResponse(**body) if body is not None else CreateMetricResponse() -async def updateMetric( - api_config_override: Optional[APIConfig] = None, *, data: UpdateMetricRequest +async def updateMetricLegacy( + api_config_override: Optional[APIConfig] = None, *, data: LegacyUpdateMetricRequest ) -> UpdateMetricResponse: api_config = api_config_override if api_config_override else APIConfig() @@ -108,7 +108,7 @@ async def updateMetric( if response.status_code != 200: raise HTTPException( response.status_code, - f"updateMetric failed with status code: {response.status_code}", + f"updateMetricLegacy failed with status code: {response.status_code}", ) else: body = None if 200 == 204 else response.json() @@ -116,7 +116,7 @@ async def updateMetric( return UpdateMetricResponse(**body) if body is not None else UpdateMetricResponse() -async def deleteMetric( +async def deleteMetricLegacy( api_config_override: Optional[APIConfig] = None, *, metric_id: str ) -> DeleteMetricResponse: api_config = api_config_override if api_config_override else APIConfig() @@ -130,6 +130,80 @@ async def deleteMetric( key: value for (key, value) in query_params.items() if value is not None } + async with httpx.AsyncClient( + base_url=base_path, verify=api_config.verify + ) as client: + response = await client.request( + "delete", + httpx.URL(path), + headers=headers, + params=_serialize_query_params(query_params), + ) + + if response.status_code != 200: + raise HTTPException( + response.status_code, + f"deleteMetricLegacy failed with status code: {response.status_code}", + ) + else: + body = None if 200 == 204 else response.json() + + return DeleteMetricResponse(**body) if body is not None else DeleteMetricResponse() + + +async def updateMetric( + api_config_override: Optional[APIConfig] = None, + *, + metric_id: str, + data: UpdateMetricRequest, +) -> UpdateMetricResponse: + api_config = api_config_override if api_config_override else APIConfig() + + base_path = api_config.base_path + path = f"/v1/metrics/{metric_id}" + headers = api_config.get_default_headers() + query_params: Dict[str, Any] = {} + + query_params = { + key: value for (key, value) in query_params.items() if value is not None + } + + async with httpx.AsyncClient( + base_url=base_path, verify=api_config.verify + ) as client: + response = await client.request( + "put", + httpx.URL(path), + headers=headers, + params=_serialize_query_params(query_params), + json=data.model_dump(exclude_none=True), + ) + + if response.status_code != 200: + raise HTTPException( + response.status_code, + f"updateMetric failed with status code: {response.status_code}", + ) + else: + body = None if 200 == 204 else response.json() + + return UpdateMetricResponse(**body) if body is not None else UpdateMetricResponse() + + +async def deleteMetric( + api_config_override: Optional[APIConfig] = None, *, metric_id: str +) -> DeleteMetricResponse: + api_config = api_config_override if api_config_override else APIConfig() + + base_path = api_config.base_path + path = f"/v1/metrics/{metric_id}" + headers = api_config.get_default_headers() + query_params: Dict[str, Any] = {} + + query_params = { + key: value for (key, value) in query_params.items() if value is not None + } + async with httpx.AsyncClient( base_url=base_path, verify=api_config.verify ) as client: @@ -157,7 +231,7 @@ async def runMetric( api_config = api_config_override if api_config_override else APIConfig() base_path = api_config.base_path - path = f"/v1/metrics/run_metric" + path = f"/v1/metrics/run" headers = api_config.get_default_headers() query_params: Dict[str, Any] = {} @@ -185,3 +259,39 @@ async def runMetric( body = None if 200 == 204 else response.json() return RunMetricResponse(**body) if body is not None else RunMetricResponse() + + +async def runMetricLegacy( + api_config_override: Optional[APIConfig] = None, *, data: LegacyRunMetricRequest +) -> RunMetricResponse: + api_config = api_config_override if api_config_override else APIConfig() + + base_path = api_config.base_path + path = f"/v1/metrics/run_metric" + headers = api_config.get_default_headers() + query_params: Dict[str, Any] = {} + + query_params = { + key: value for (key, value) in query_params.items() if value is not None + } + + async with httpx.AsyncClient( + base_url=base_path, verify=api_config.verify + ) as client: + response = await client.request( + "post", + httpx.URL(path), + headers=headers, + params=_serialize_query_params(query_params), + json=data.model_dump(exclude_none=True), + ) + + if response.status_code != 200: + raise HTTPException( + response.status_code, + f"runMetricLegacy failed with status code: {response.status_code}", + ) + else: + body = None if 200 == 204 else response.json() + + return RunMetricResponse(**body) if body is not None else RunMetricResponse() diff --git a/src/honeyhive/api/client.py b/src/honeyhive/api/client.py index ceef62aa..6e40fac1 100644 --- a/src/honeyhive/api/client.py +++ b/src/honeyhive/api/client.py @@ -464,7 +464,7 @@ def update(self, request: UpdateDatasetRequest) -> UpdateDatasetResponse: def delete(self, id: str) -> DeleteDatasetResponse: """Delete a dataset.""" - return datasets_svc.deleteDataset(self._api_config, dataset_id=id) + return datasets_svc.deleteDatasetLegacy(self._api_config, dataset_id=id) def add_datapoints( self, dataset_id: str, request: AddDatapointsToDatasetRequest @@ -494,7 +494,7 @@ def remove_datapoint( Returns: RemoveDatapointResponse with dereferenced status and message. """ - return datasets_svc.removeDatapoint( + return datasets_svc.removeDatapointLegacy( self._api_config, dataset_id=dataset_id, datapoint_id=datapoint_id ) @@ -532,7 +532,9 @@ async def update_async( async def delete_async(self, id: str) -> DeleteDatasetResponse: """Delete a dataset asynchronously.""" - return await datasets_svc_async.deleteDataset(self._api_config, dataset_id=id) + return await datasets_svc_async.deleteDatasetLegacy( + self._api_config, dataset_id=id + ) async def add_datapoints_async( self, dataset_id: str, request: AddDatapointsToDatasetRequest @@ -562,7 +564,7 @@ async def remove_datapoint_async( Returns: RemoveDatapointResponse with dereferenced status and message. """ - return await datasets_svc_async.removeDatapoint( + return await datasets_svc_async.removeDatapointLegacy( self._api_config, dataset_id=dataset_id, datapoint_id=datapoint_id ) @@ -749,15 +751,15 @@ def get_by_session_id( def create(self, request: PostEventRequest) -> PostEventResponse: """Create an event.""" - return events_svc.createEvent(self._api_config, data=request) + return events_svc.createEventLegacy(self._api_config, data=request) def update(self, data: UpdateEventRequest) -> None: """Update an event.""" - return events_svc.updateEvent(self._api_config, data=data) + return events_svc.updateEventLegacy(self._api_config, data=data) def create_batch(self, data: PostEventBatchRequest) -> PostEventBatchResponse: """Create events in batch.""" - return events_svc.createEventBatch(self._api_config, data=data) + return events_svc.createEventBatchLegacy(self._api_config, data=data) def export( self, @@ -1053,17 +1055,19 @@ async def get_by_session_id_async( async def create_async(self, request: PostEventRequest) -> PostEventResponse: """Create an event asynchronously.""" - return await events_svc_async.createEvent(self._api_config, data=request) + return await events_svc_async.createEventLegacy(self._api_config, data=request) async def update_async(self, data: UpdateEventRequest) -> None: """Update an event asynchronously.""" - return await events_svc_async.updateEvent(self._api_config, data=data) + return await events_svc_async.updateEventLegacy(self._api_config, data=data) async def create_batch_async( self, data: PostEventBatchRequest ) -> PostEventBatchResponse: """Create events in batch asynchronously.""" - return await events_svc_async.createEventBatch(self._api_config, data=data) + return await events_svc_async.createEventBatchLegacy( + self._api_config, data=data + ) async def export_async( self, @@ -1324,7 +1328,7 @@ def get_schema( dateRange: Filter by date range (string or dict with $gte/$lte). evaluation_id: Filter by evaluation/run ID. """ - return events_svc.getEventsSchema( + return events_svc.getEventsSchemaLegacy( self._api_config, dateRange=dateRange, evaluation_id=evaluation_id ) @@ -1462,7 +1466,7 @@ async def get_schema_async( dateRange: Filter by date range (string or dict with $gte/$lte). evaluation_id: Filter by evaluation/run ID. """ - return await events_svc_async.getEventsSchema( + return await events_svc_async.getEventsSchemaLegacy( self._api_config, dateRange=dateRange, evaluation_id=evaluation_id ) @@ -1620,7 +1624,7 @@ def compare_runs( aggregate_function: Aggregation function to apply. filters: Optional filters to apply. """ - result = experiments_svc.getExperimentComparison( + result = experiments_svc.getExperimentComparisonLegacy( self._api_config, new_run_id=new_run_id, old_run_id=old_run_id, @@ -1666,7 +1670,7 @@ async def compare_runs_async( aggregate_function: Aggregation function to apply. filters: Optional filters to apply. """ - result = await experiments_svc_async.getExperimentComparison( + result = await experiments_svc_async.getExperimentComparisonLegacy( self._api_config, new_run_id=new_run_id, old_run_id=old_run_id, @@ -1706,7 +1710,7 @@ def compare_run_events( limit: Maximum number of results. page: Page number for pagination. """ - return experiments_svc.getExperimentCompareEvents( + return experiments_svc.getExperimentCompareEventsLegacy( self._api_config, run_id_1=new_run_id, run_id_2=old_run_id, @@ -1750,11 +1754,11 @@ def create(self, request: CreateMetricRequest) -> CreateMetricResponse: def update(self, request: UpdateMetricRequest) -> UpdateMetricResponse: """Update a metric.""" - return metrics_svc.updateMetric(self._api_config, data=request) + return metrics_svc.updateMetricLegacy(self._api_config, data=request) def delete(self, id: str) -> DeleteMetricResponse: """Delete a metric.""" - return metrics_svc.deleteMetric(self._api_config, metric_id=id) + return metrics_svc.deleteMetricLegacy(self._api_config, metric_id=id) # Async methods async def list_async( @@ -1783,11 +1787,15 @@ async def create_async(self, request: CreateMetricRequest) -> CreateMetricRespon async def update_async(self, request: UpdateMetricRequest) -> UpdateMetricResponse: """Update a metric asynchronously.""" - return await metrics_svc_async.updateMetric(self._api_config, data=request) + return await metrics_svc_async.updateMetricLegacy( + self._api_config, data=request + ) async def delete_async(self, id: str) -> DeleteMetricResponse: """Delete a metric asynchronously.""" - return await metrics_svc_async.deleteMetric(self._api_config, metric_id=id) + return await metrics_svc_async.deleteMetricLegacy( + self._api_config, metric_id=id + ) # Backwards compatible aliases def get_metric(self, id: str) -> List[MetricItem]: diff --git a/src/honeyhive/config/models/tracer.py b/src/honeyhive/config/models/tracer.py index 271aa818..a822a2de 100644 --- a/src/honeyhive/config/models/tracer.py +++ b/src/honeyhive/config/models/tracer.py @@ -403,8 +403,13 @@ class SessionConfig(BaseHoneyHiveConfig): skip_backend_session_creation: bool = Field( # type: ignore[call-overload] default=False, description=( - "If True and session_id is provided, trust that the session already " - "exists on the backend and skip the init-time session creation call." + "If True, skip the init-time backend session creation call. " + "If a valid session_id is also provided, the SDK trusts that the " + "session already exists on the backend. Otherwise, callers are " + "expected to manage session_ids via per-request " + "create_session(session_id=, skip_api_call=True) calls. " + "Note: an invalid session_id (e.g. non-UUID) triggers the " + "degraded-mode path and still calls the backend." ), ) diff --git a/src/honeyhive/models/__init__.py b/src/honeyhive/models/__init__.py index 21a94239..33914aa6 100644 --- a/src/honeyhive/models/__init__.py +++ b/src/honeyhive/models/__init__.py @@ -210,11 +210,11 @@ class EventExportResponse(BaseModel): DeleteConfigurationResponse, DeleteDatapointParams, DeleteDatapointResponse, + DeleteDatasetParams, DeleteDatasetQuery, DeleteDatasetResponse, DeleteExperimentRunParams, DeleteExperimentRunResponse, - DeleteMetricQuery, DeleteMetricResponse, Event, GetConfigurationsQuery, @@ -301,6 +301,7 @@ class EventExportResponse(BaseModel): "CreateDatasetRequest", "DatapointMapping", "CreateDatasetResponse", + "DeleteDatasetParams", "DeleteDatasetQuery", "DeleteDatasetResponse", "GetDatasetsResponse", @@ -341,7 +342,6 @@ class EventExportResponse(BaseModel): # Metric models "CreateMetricRequest", "CreateMetricResponse", - "DeleteMetricQuery", "DeleteMetricResponse", "GetMetricsQuery", "GetMetricsResponse", diff --git a/src/honeyhive/models/models.py b/src/honeyhive/models/models.py index 1fe91957..8296bc4b 100644 --- a/src/honeyhive/models/models.py +++ b/src/honeyhive/models/models.py @@ -28,11 +28,10 @@ DeleteConfigurationResponse, DeleteDatapointParams, DeleteDatapointResponse, - DeleteDatasetQuery, + DeleteDatasetParams, DeleteDatasetResponse, DeleteExperimentRunParams, DeleteExperimentRunResponse, - DeleteMetricQuery, DeleteMetricResponse, Event, GetConfigurationsQuery, @@ -44,11 +43,7 @@ GetDatasetsResponse, GetEventsQuery, GetEventsResponse, - GetEventsSchemaQuery, GetEventsSchemaResponse, - GetExperimentRunCompareEventsQuery, - GetExperimentRunCompareParams, - GetExperimentRunCompareQuery, GetExperimentRunMetricsQuery, GetExperimentRunParams, GetExperimentRunResponse, @@ -57,13 +52,22 @@ GetExperimentRunsResponse, GetMetricsQuery, GetMetricsResponse, + LegacyDeleteDatasetQuery, LegacyEvent, + LegacyGetEventsSchemaQuery, + LegacyGetExperimentRunCompareEventsQuery, + LegacyGetExperimentRunCompareParams, + LegacyGetExperimentRunCompareQuery, + LegacyPostEventBatchRequest, + LegacyPostEventRequest, + LegacyRemoveDatapointFromDatasetParams, LegacyUpdateDatasetRequest, + LegacyUpdateEventRequest, + LegacyUpdateMetricRequest, MetricItem, Pagination, PostEventBatchRequest, PostEventBatchResponse, - PostEventRequest, PostEventResponse, PostExperimentRunRequest, PostExperimentRunResponse, @@ -71,7 +75,6 @@ PostSessionStartResponse, PutExperimentRunRequest, PutExperimentRunResponse, - RemoveDatapointFromDatasetParams, RemoveDatapointResponse, RunMetricRequest, RunMetricResponse, @@ -83,8 +86,6 @@ UpdateDatapointRequest, UpdateDatapointResponse, UpdateDatasetResponse, - UpdateEventRequest, - UpdateMetricRequest, UpdateMetricResponse, ) @@ -97,13 +98,108 @@ class UpdateDatasetRequest(LegacyUpdateDatasetRequest): pass -# Back-compat aliases for the GET /runs/schema → GET /events/schema rename -# (#3440). The generated symbols moved to GetEventsSchema*; these keep the -# old public names importable from honeyhive.models. +# Public name for metric update requests. The canonical spec renamed this +# to the new PUT /metrics/{metric_id} shape (no id in the body), but the +# Python SDK still calls the legacy PUT /metrics endpoint to avoid breaking +# customers who pass `id` in the request body. +class UpdateMetricRequest(LegacyUpdateMetricRequest): + pass + + +# Public name kept pointing at the legacy events-schema query model. The +# canonical spec forked GET /events/schema into GET /v1/runs/{run_id}/schema +# and GET /v1/runs/schema (HHAI-4990); the generated GetRunSchemaQuery / +# GetRunsSchemaQuery models target those new routes and use `.strict()`, +# but the Python SDK still calls the legacy /v1/events/schema endpoint, so +# existing customer imports keep resolving to a model that mirrors the wire +# shape we actually send (dateRange + evaluation_id, non-strict). +class GetEventsSchemaQuery(LegacyGetEventsSchemaQuery): + pass + + +# Back-compat aliases preserved from the earlier GET /runs/schema → GET /events/schema +# rename (#3440); kept importable here so customer code that still uses the +# old names continues to resolve. GetExperimentRunsSchemaQuery = GetEventsSchemaQuery GetExperimentRunsSchemaResponse = GetEventsSchemaResponse +# Public name preserved for backwards compatibility — was the query schema +# for the legacy DELETE /datasets endpoint, retained here so existing imports +# continue to resolve after the schema fork. +class DeleteDatasetQuery(LegacyDeleteDatasetQuery): + pass + + +# Public name kept pointing at the legacy params schema. The new +# RemoveDatapointFromDatasetParams in the generated module is `.strict()`, +# but on the Python side both shapes are field-identical so existing +# customer imports keep working unchanged. +class RemoveDatapointFromDatasetParams(LegacyRemoveDatapointFromDatasetParams): + pass + + +# Public names kept pointing at the legacy compare-with schemas. The new +# GetExperimentRunCompare* generated models target /v1/runs/{new_run_id}/compare/{old_run_id} +# and add `.strict()`. The Python SDK still calls the legacy compare-with +# endpoint (compare_runs / compare_runs_async) so existing customer imports +# keep resolving to a model that mirrors the wire shape we actually send. +class GetExperimentRunCompareParams(LegacyGetExperimentRunCompareParams): + pass + + +class GetExperimentRunCompareQuery(LegacyGetExperimentRunCompareQuery): + pass + + +# Public name kept pointing at the legacy compare/events query schema. The new +# GetExperimentRunCompareEventsQuery generated model targets +# /v1/runs/{new_run_id}/compare/{old_run_id}/events (no run_ids, `.strict()`), +# but the Python SDK still calls the legacy /v1/runs/compare/events endpoint +# so existing customer imports keep resolving to a model that mirrors the +# wire shape we actually send (run_id_1 / run_id_2 in the query string). +class GetExperimentRunCompareEventsQuery(LegacyGetExperimentRunCompareEventsQuery): + pass + + +# Public name kept pointing at the legacy event-create schema. The new +# generated PostEventRequest targets POST /v1/events with a bare event-object +# body and drops the deprecated `project` field. The Python SDK still calls +# the legacy POST /events endpoint (events.create / events.create_async) so +# existing customer code that constructs PostEventRequest(event={...}) keeps +# resolving to the wrapped wire shape we send. +class PostEventRequest(LegacyPostEventRequest): + pass + + +# Public name kept pointing at the legacy event-batch schema. The new +# generated PostEventBatchRequest (in _generated/models/) targets +# POST /v1/events/batch with a body that, on the Zod / TypeScript-SDK side, +# is `.strict()` and drops the deprecated `is_single_session` and `session` +# aliases at the top level and the deprecated `project` field per event. The +# strictness is a TS-SDK-boundary contract; this Pydantic alias inherits +# `model_config["extra"] = "allow"` from LegacyPostEventBatchRequest and so +# does NOT reject `is_single_session=True` or `session={...}` at construction +# time. That's by design — the public Python alias is a back-compat shim, not +# a v1-strict enforcer. The Python SDK still calls the legacy +# POST /events/batch endpoint (events.create_batch / events.create_batch_async) +# so existing customer code that constructs +# PostEventBatchRequest(events=[...], is_single_session=True, session={...}) +# keeps resolving to the legacy wire shape we send. +class PostEventBatchRequest(LegacyPostEventBatchRequest): + pass + + +# Public name kept pointing at the legacy event-update schema. The new +# generated UpdateEventRequest targets PUT /v1/events/{event_id} (event_id in +# the URL path, not the body). The Python SDK still calls the legacy +# PUT /events endpoint (events.update / events.update_async) so existing +# customer code that constructs UpdateEventRequest(event_id=..., metadata=...) +# keeps resolving to the wire shape we actually send. +class UpdateEventRequest(LegacyUpdateEventRequest): + pass + + # Note: models prefixed with Legacy are intentionally not re-exported — they are # implementation details of the aliases above __all__ = [ @@ -124,11 +220,11 @@ class UpdateDatasetRequest(LegacyUpdateDatasetRequest): "DeleteConfigurationResponse", "DeleteDatapointParams", "DeleteDatapointResponse", + "DeleteDatasetParams", "DeleteDatasetQuery", "DeleteDatasetResponse", "DeleteExperimentRunParams", "DeleteExperimentRunResponse", - "DeleteMetricQuery", "DeleteMetricResponse", "Event", "GetConfigurationsQuery", diff --git a/src/honeyhive/tracer/core/context.py b/src/honeyhive/tracer/core/context.py index d9ae7fed..660fe98b 100644 --- a/src/honeyhive/tracer/core/context.py +++ b/src/honeyhive/tracer/core/context.py @@ -392,6 +392,8 @@ async def chat(message: str): if session_id and skip_api_call: current_ctx = context.get_current() new_ctx = baggage.set_baggage("session_id", session_id, current_ctx) + if session_name: + new_ctx = baggage.set_baggage("session_name", session_name, new_ctx) context.attach(new_ctx) safe_log( @@ -400,6 +402,7 @@ async def chat(message: str): f"Set provided session_id in baggage (no API call): {session_id}", honeyhive_data={ "session_id": session_id, + "session_name": session_name, "storage": "baggage", "source": "provided", "api_call": False, @@ -520,6 +523,8 @@ async def session_middleware(request: Request, call_next): if session_id and skip_api_call: current_ctx = context.get_current() new_ctx = baggage.set_baggage("session_id", session_id, current_ctx) + if session_name: + new_ctx = baggage.set_baggage("session_name", session_name, new_ctx) context.attach(new_ctx) safe_log( @@ -528,6 +533,7 @@ async def session_middleware(request: Request, call_next): f"Set provided session_id in baggage (async, no API): {session_id}", honeyhive_data={ "session_id": session_id, + "session_name": session_name, "storage": "baggage", "source": "provided", "api_call": False, diff --git a/src/honeyhive/tracer/core/operations.py b/src/honeyhive/tracer/core/operations.py index 68646d03..2bc14330 100644 --- a/src/honeyhive/tracer/core/operations.py +++ b/src/honeyhive/tracer/core/operations.py @@ -24,7 +24,7 @@ # Event request is now built as a dict and passed directly to the API # EventType values are now plain strings since we pass dicts to the API -from ..._generated.models import PostEventRequest +from ...models import PostEventRequest from ...utils.logger import is_shutdown_detected, safe_log from ..lifecycle.core import is_new_span_creation_disabled from .base import NoOpSpan diff --git a/src/honeyhive/tracer/instrumentation/initialization.py b/src/honeyhive/tracer/instrumentation/initialization.py index 0590ccef..989b48c1 100644 --- a/src/honeyhive/tracer/instrumentation/initialization.py +++ b/src/honeyhive/tracer/instrumentation/initialization.py @@ -1074,6 +1074,7 @@ def _initialize_session_management(tracer_instance: Any) -> None: # session_id and explicitly request skipping init-time creation. provided_session_id = None has_valid_provided_session_id = False + has_user_provided_session_id = bool(tracer_instance.session_id) skip_backend_session_creation = getattr( tracer_instance.config, "skip_backend_session_creation", False ) @@ -1121,6 +1122,25 @@ def _initialize_session_management(tracer_instance: Any) -> None: "Skipping backend session initialization for provided session_id", honeyhive_data={"session_id": provided_session_id}, ) + elif skip_backend_session_creation and not has_user_provided_session_id: + # No session_id supplied; caller opted out of the init-time roundtrip. + # Set the auto-create flag so the span processor stamps every emitted + # span with honeyhive.session_auto_create=true. Per-request session_ids + # come from explicit create_session() calls; ingestion materializes + # the Session row from the first span carrying each session_id. + # + # NB: the auto-create attribute is only stamped when a session_id is + # in context (baggage or tracer.session_id) — see span_processor.py + # `if session_id:` gate. Spans emitted before any create_session() + # call (module-level instrumentation, framework boot, etc.) carry + # no session_id and fall back to the ingestion-side default. + tracer_instance._session_auto_create = True + safe_log( + tracer_instance, + "debug", + "Skipping backend session initialization; " + "auto-create flag set for spans emitted under this tracer", + ) else: # Always create/initialize session in backend unless the caller # explicitly opts out with a valid provided session_id. diff --git a/src/honeyhive/tracer/processing/span_processor.py b/src/honeyhive/tracer/processing/span_processor.py index e1ab8d9c..ad4adc49 100644 --- a/src/honeyhive/tracer/processing/span_processor.py +++ b/src/honeyhive/tracer/processing/span_processor.py @@ -790,6 +790,22 @@ def on_start(self, span: Span, parent_context: Optional[Context] = None) -> None session_id ) + # Signal ingestion to auto-create the Session row if it doesn't + # exist yet. Stamped on every span; ingestion is idempotent on + # session_id. + if getattr(self.tracer_instance, "_session_auto_create", False): + attributes_to_set["honeyhive.session_auto_create"] = True + # Prefer session_name from baggage (per-request) over the + # tracer-instance value (init-time), matching how + # session_id is resolved. + session_name = baggage.get_baggage("session_name", ctx) + if not session_name: + session_name = getattr( + self.tracer_instance, "session_name", None + ) + if session_name: + attributes_to_set["honeyhive.session_name"] = session_name + # Get other baggage attributes (project, source, etc.) other_baggage_attrs = self._get_basic_baggage_attributes(ctx) # Remove session_id from baggage attrs since we're setting it directly diff --git a/tests/compatibility_matrix/README.md b/tests/compatibility_matrix/README.md index 6020529e..4a042ee6 100644 --- a/tests/compatibility_matrix/README.md +++ b/tests/compatibility_matrix/README.md @@ -101,7 +101,6 @@ export GOOGLE_ADK_API_KEY="your_google_adk_api_key" # Azure OpenAI (Required for: Azure OpenAI tests) export AZURE_OPENAI_ENDPOINT="https://your-resource.openai.azure.com/" export AZURE_OPENAI_API_KEY="your_azure_openai_api_key" -export AZURE_OPENAI_DEPLOYMENT_NAME="your_deployment_name" export AZURE_OPENAI_API_VERSION="2024-02-15-preview" # Optional export AZURE_OPENAI_DEPLOYMENT="gpt-35-turbo" # Optional export AZURE_OPENAI_GPT4_DEPLOYMENT="gpt-4" # Optional diff --git a/tests/compatibility_matrix/env.example b/tests/compatibility_matrix/env.example index 898d162c..444b46cd 100644 --- a/tests/compatibility_matrix/env.example +++ b/tests/compatibility_matrix/env.example @@ -22,7 +22,6 @@ GCP_REGION=us-central1 # Azure OpenAI (Required for: Azure OpenAI tests) AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/ AZURE_OPENAI_API_KEY=your_azure_openai_api_key_here -AZURE_OPENAI_DEPLOYMENT_NAME=your_deployment_name AZURE_OPENAI_API_VERSION=2024-02-15-preview AZURE_OPENAI_DEPLOYMENT=gpt-35-turbo AZURE_OPENAI_GPT4_DEPLOYMENT=gpt-4 diff --git a/tests/compatibility_matrix/run_compatibility_tests.py b/tests/compatibility_matrix/run_compatibility_tests.py index dcde6c20..fc79ac1e 100644 --- a/tests/compatibility_matrix/run_compatibility_tests.py +++ b/tests/compatibility_matrix/run_compatibility_tests.py @@ -89,7 +89,7 @@ def __init__(self): "required_env": [ "AZURE_OPENAI_ENDPOINT", "AZURE_OPENAI_API_KEY", - "AZURE_OPENAI_DEPLOYMENT_NAME", + "AZURE_OPENAI_DEPLOYMENT", ], }, "test_openinference_anthropic.py": { @@ -136,7 +136,7 @@ def __init__(self): "required_env": [ "AZURE_OPENAI_ENDPOINT", "AZURE_OPENAI_API_KEY", - "AZURE_OPENAI_DEPLOYMENT_NAME", + "AZURE_OPENAI_DEPLOYMENT", ], }, "test_traceloop_anthropic.py": { diff --git a/tests/compatibility_matrix/test_openinference_azure_openai.py b/tests/compatibility_matrix/test_openinference_azure_openai.py index 21fd0855..67693352 100644 --- a/tests/compatibility_matrix/test_openinference_azure_openai.py +++ b/tests/compatibility_matrix/test_openinference_azure_openai.py @@ -20,7 +20,7 @@ def test_azure_openai_integration(): azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT") azure_key = os.getenv("AZURE_OPENAI_API_KEY") azure_version = os.getenv("AZURE_OPENAI_API_VERSION", "2024-02-15-preview") - deployment_name = os.getenv("AZURE_OPENAI_DEPLOYMENT_NAME") + deployment_name = os.getenv("AZURE_OPENAI_DEPLOYMENT") if not all([api_key, project, azure_endpoint, azure_key, deployment_name]): print("❌ Missing required environment variables:") @@ -28,7 +28,7 @@ def test_azure_openai_integration(): print(" - HH_PROJECT (HoneyHive project)") print(" - AZURE_OPENAI_ENDPOINT (Azure OpenAI endpoint)") print(" - AZURE_OPENAI_API_KEY (Azure OpenAI API key)") - print(" - AZURE_OPENAI_DEPLOYMENT_NAME (Azure deployment name)") + print(" - AZURE_OPENAI_DEPLOYMENT (Azure deployment name)") print( " - AZURE_OPENAI_API_VERSION (optional, defaults to 2024-02-15-preview)" ) diff --git a/tests/integration/test_azure_openai_integration.py b/tests/integration/test_azure_openai_integration.py index e8a4d4d5..25d761a1 100644 --- a/tests/integration/test_azure_openai_integration.py +++ b/tests/integration/test_azure_openai_integration.py @@ -1,11 +1,12 @@ """ Azure OpenAI Integration Tests -Tests Azure OpenAI integration with HoneyHive using Traceloop instrumentor. -Based on examples/integrations/traceloop_azure_openai_example.py. +Tests Azure OpenAI integration with HoneyHive using both the Traceloop +(opentelemetry-instrumentation-openai) and OpenInference +(openinference-instrumentation-openai) instrumentors. Requirements: - pip install honeyhive opentelemetry-instrumentation-openai openai + pip install honeyhive opentelemetry-instrumentation-openai openinference-instrumentation-openai openai Environment Variables: HH_API_KEY: HoneyHive API key @@ -152,3 +153,117 @@ def process_with_azure(prompt: str) -> str: finally: instrumentor.uninstrument() + + +class TestOpenInferenceAzureOpenAIIntegration: + """Test Azure OpenAI integration via OpenInference instrumentor.""" + + @pytest.fixture(autouse=True) + def setup(self): + """Check if dependencies are available.""" + pytest.importorskip("openai") + pytest.importorskip("openinference.instrumentation.openai") + + def test_basic_chat_completion(self): + """Test basic Azure OpenAI chat completion is traced via OpenInference. + + Verifies: + - AzureOpenAI client initializes with endpoint/key + - Chat completion returns valid response + - Response has non-empty content + - Traces are exported via flush() + """ + from openai import AzureOpenAI + from openinference.instrumentation.openai import OpenAIInstrumentor + + from honeyhive import HoneyHiveTracer + + tracer = HoneyHiveTracer.init( + project=os.getenv("HH_PROJECT", "azure-openai-integration-test"), + session_name="test_openinference_basic_chat_completion", + source="pytest", + ) + + instrumentor = OpenAIInstrumentor() + instrumentor.instrument(tracer_provider=tracer.provider) + + try: + client = AzureOpenAI( + api_key=os.getenv("AZURE_OPENAI_API_KEY"), + api_version="2024-02-01", + azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"), + ) + + deployment = os.getenv("AZURE_OPENAI_DEPLOYMENT", "gpt-35-turbo") + response = client.chat.completions.create( + model=deployment, + messages=[{"role": "user", "content": "Say 'test' and nothing else."}], + max_tokens=10, + ) + + assert response.choices[0].message.content is not None + assert len(response.choices[0].message.content) > 0 + assert response.usage.total_tokens > 0 + + tracer.flush() + + finally: + instrumentor.uninstrument() + + def test_chat_with_enrichment(self): + """Test Azure OpenAI with span enrichment via OpenInference. + + Verifies: + - @trace decorator wraps Azure OpenAI call + - enrich_span() adds deployment metadata + - enrich_span() adds token usage metrics + - Nested spans are properly linked + """ + from openai import AzureOpenAI + from openinference.instrumentation.openai import OpenAIInstrumentor + + from honeyhive import HoneyHiveTracer, enrich_span, trace + + tracer = HoneyHiveTracer.init( + project=os.getenv("HH_PROJECT", "azure-openai-integration-test"), + session_name="test_openinference_chat_with_enrichment", + source="pytest", + ) + + instrumentor = OpenAIInstrumentor() + instrumentor.instrument(tracer_provider=tracer.provider) + + try: + + @trace(event_type="tool") + def process_with_azure(prompt: str) -> str: + enrich_span( + metadata={"deployment": os.getenv("AZURE_OPENAI_DEPLOYMENT")} + ) + + client = AzureOpenAI( + api_key=os.getenv("AZURE_OPENAI_API_KEY"), + api_version="2024-02-01", + azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"), + ) + + deployment = os.getenv("AZURE_OPENAI_DEPLOYMENT", "gpt-35-turbo") + response = client.chat.completions.create( + model=deployment, + messages=[{"role": "user", "content": prompt}], + max_tokens=20, + ) + + result = response.choices[0].message.content + enrich_span(metrics={"tokens": response.usage.total_tokens}) + return result + + result = process_with_azure( + "Say 'azure openinference test' and nothing else." + ) + assert result is not None + + tracer.flush() + + finally: + instrumentor.uninstrument() diff --git a/tests/integration/test_experiments_integration.py b/tests/integration/test_experiments_integration.py index 4b7e5bea..09f7ce44 100644 --- a/tests/integration/test_experiments_integration.py +++ b/tests/integration/test_experiments_integration.py @@ -23,7 +23,7 @@ from honeyhive import HoneyHive, enrich_span, trace from honeyhive.experiments import compare_runs, evaluate -from honeyhive.models import CreateDatapointRequest, CreateDatasetRequest +from honeyhive.models import CreateDatapointRequest, CreateDatasetRequest, EventFilter @pytest.mark.integration @@ -664,21 +664,20 @@ def test_managed_dataset_evaluation( # Helper: Extract ID from object def get_id_from_object(obj: Any) -> Optional[str]: """Extract ID from object/dict with multiple field support.""" - if isinstance(obj, dict): - for key in ("_id", "id"): + for key in ("insertedId", "inserted_id", "_id", "id"): + if isinstance(obj, dict): id_value = obj.get(key) - if id_value: - return str(id_value) - return None - - if hasattr(obj, "_id"): - id_value = getattr(obj, "_id", None) - if id_value: - return str(id_value) - if hasattr(obj, "id"): - id_value = getattr(obj, "id", None) + else: + id_value = getattr(obj, key, None) if id_value: return str(id_value) + ids_list = ( + obj.get("insertedIds") + if isinstance(obj, dict) + else getattr(obj, "insertedIds", None) + ) + if ids_list: + return str(ids_list[0]) return None # Helper: Create dataset and get ID @@ -694,43 +693,7 @@ def create_dataset_and_get_id(name: str) -> str: print("✅ Dataset created") created_result = getattr(created, "result", None) - dataset_obj: Any = created - if created_result is not None: - dataset_obj = ( - created_result[0] - if isinstance(created_result, list) and created_result - else created_result - ) - - dataset_name_value = ( - dataset_obj.get("name") - if isinstance(dataset_obj, dict) - else getattr(dataset_obj, "name", None) - ) - if dataset_name_value: - print(f" Name: {dataset_name_value}") - - dataset_project_value = ( - dataset_obj.get("project") - if isinstance(dataset_obj, dict) - else getattr(dataset_obj, "project", None) - ) - if dataset_project_value: - print(f" Project: {dataset_project_value}") - - # Try to get ID from response - ds_id = get_id_from_object(dataset_obj) or get_id_from_object(created) - - if not ds_id: - # Fallback: search by name - print(" Searching by name...") - datasets = integration_client.datasets.list_datasets(name=name) - for ds in datasets: - if getattr(ds, "name", None) == name: - ds_id = get_id_from_object(ds) - if ds_id: - print(f" Found: {ds_id}") - break + ds_id = get_id_from_object(created_result) if created_result else None assert ds_id, f"Could not get dataset_id for {name}" print(f" ID: {ds_id}") @@ -780,7 +743,7 @@ def create_and_track_datapoint( metadata=None, ) created = integration_client.datapoints.create_datapoint(datapoint_request) - datapoint_id = get_id_from_object(created) + datapoint_id = get_id_from_object(getattr(created, "result", None)) print(f"✅ Datapoint {idx} created") if datapoint_id: print(f" ID: {datapoint_id}") @@ -1070,25 +1033,24 @@ def accuracy_evaluator( def _fetch_all_session_events( integration_client: HoneyHive, event_ids: list, real_project: str ) -> list: - """Fetch all events for given session IDs.""" + """Fetch all events for given session IDs via /events/export.""" + del real_project all_events = [] for session_id in event_ids: try: - # Convert UUID to string for EventFilter - # (backend returns UUIDType objects) session_id_str = str(session_id) - # TODO: EventFilter doesn't exist in v1, need to update to v1 API - # events_response = integration_client.events.get_events( - # project=real_project, - # filters=[ - # EventFilter( - # field="session_id", value=session_id_str, operator="is" - # ), - # ], - # ) - # Placeholder response until v1 API is implemented - events_response = {"events": []} - session_events = events_response.get("events", []) + events_response = integration_client.events.export( + filters=[ + EventFilter( + field="session_id", + operator="is", + value=session_id_str, + type="string", + ) + ], + limit=1000, + ) + session_events = list(getattr(events_response, "events", []) or []) all_events.extend(session_events) print( f" ✅ Session {session_id_str[:16]}... " diff --git a/tests/unit/test_session_baggage_propagation.py b/tests/unit/test_session_baggage_propagation.py new file mode 100644 index 00000000..1374ec95 --- /dev/null +++ b/tests/unit/test_session_baggage_propagation.py @@ -0,0 +1,100 @@ +"""Tests for session_name baggage propagation in create_session(skip_api_call=True). + +When skip_backend_session_creation is enabled and callers manage session_ids +per-request via create_session(session_id=..., session_name=..., skip_api_call=True), +session_name must propagate through baggage so per-request names override the +init-time tracer.session_name on emitted spans. +""" + +from typing import Any +from unittest.mock import Mock, patch + +import pytest + +from honeyhive.tracer.core.context import TracerContextMixin + + +class _FakeTracerContextMixin(TracerContextMixin): + """Minimal concrete TracerContextMixin for exercising create_session().""" + + def __init__(self) -> None: + self.client: Any = None + self.session_api: Any = None + self._session_id = None + self._baggage_lock = Mock() + self._cache_manager = None + self.propagator = Mock() + + def _normalize_attribute_key_dynamically(self, key: str) -> str: + return key + + def _normalize_attribute_value_dynamically(self, value: Any) -> Any: + return value + + +@pytest.fixture +def mixin() -> _FakeTracerContextMixin: + return _FakeTracerContextMixin() + + +@patch("honeyhive.tracer.core.context.context.attach") +@patch("honeyhive.tracer.core.context.baggage.set_baggage") +@patch("honeyhive.tracer.core.context.context.get_current") +def test_skip_api_call_propagates_session_name_via_baggage( + mock_get_current: Mock, + mock_set_baggage: Mock, + mock_attach: Mock, + mixin: _FakeTracerContextMixin, +) -> None: + """skip_api_call=True with a session_name must propagate the name via + baggage, so per-request session_names don't get masked by the init-time + tracer.session_name when skip_backend_session_creation is enabled.""" + mock_current_ctx = Mock(name="current_ctx") + mock_ctx_with_id = Mock(name="ctx_with_id") + mock_ctx_with_name = Mock(name="ctx_with_name") + mock_get_current.return_value = mock_current_ctx + mock_set_baggage.side_effect = [mock_ctx_with_id, mock_ctx_with_name] + + result = mixin.create_session( + session_id="provided-session-id", + session_name="per-request-name", + skip_api_call=True, + ) + + assert result == "provided-session-id" + assert mock_set_baggage.call_count == 2 + mock_set_baggage.assert_any_call( + "session_id", "provided-session-id", mock_current_ctx + ) + mock_set_baggage.assert_any_call( + "session_name", "per-request-name", mock_ctx_with_id + ) + mock_attach.assert_called_once_with(mock_ctx_with_name) + + +@patch("honeyhive.tracer.core.context.context.attach") +@patch("honeyhive.tracer.core.context.baggage.set_baggage") +@patch("honeyhive.tracer.core.context.context.get_current") +def test_skip_api_call_no_session_name_does_not_set_baggage_name( + mock_get_current: Mock, + mock_set_baggage: Mock, + mock_attach: Mock, + mixin: _FakeTracerContextMixin, +) -> None: + """When session_name is not passed, the session_name baggage entry must + not be set so the span processor falls back to tracer.session_name.""" + mock_current_ctx = Mock(name="current_ctx") + mock_ctx_with_id = Mock(name="ctx_with_id") + mock_get_current.return_value = mock_current_ctx + mock_set_baggage.return_value = mock_ctx_with_id + + result = mixin.create_session( + session_id="provided-session-id", + skip_api_call=True, + ) + + assert result == "provided-session-id" + assert mock_set_baggage.call_count == 1 + mock_set_baggage.assert_called_once_with( + "session_id", "provided-session-id", mock_current_ctx + ) diff --git a/tests/unit/test_tracer_instrumentation_initialization.py b/tests/unit/test_tracer_instrumentation_initialization.py index 5b0ba4cb..fc6fb984 100644 --- a/tests/unit/test_tracer_instrumentation_initialization.py +++ b/tests/unit/test_tracer_instrumentation_initialization.py @@ -1370,6 +1370,31 @@ def test__initialize_session_management_invalid_session_id_still_creates( mock_create.assert_called_once_with(self.mock_tracer) mock_log.assert_called() + @patch("honeyhive.tracer.instrumentation.initialization._create_new_session") + @patch("honeyhive.tracer.instrumentation.initialization.uuid") + @patch("honeyhive.tracer.instrumentation.initialization.HoneyHive") + @patch("honeyhive.tracer.instrumentation.initialization.safe_log") + def test__initialize_session_management_skips_backend_creation_no_session_id( + self, mock_log: Any, mock_client: Any, mock_uuid: Any, mock_create: Any + ) -> None: + """Test skip_backend_session_creation=True without a session_id sets the + auto-create flag and skips the backend call without minting a UUID.""" + mock_client_instance = Mock() + mock_client.return_value = mock_client_instance + self.mock_tracer.session_id = None + self.mock_tracer.config.skip_backend_session_creation = True + + initialization._initialize_session_management(self.mock_tracer) + + assert self.mock_tracer.client == mock_client_instance + assert self.mock_tracer._session_auto_create is True + # No UUID minted at init — session_id stays None until per-request + # create_session() sets one in baggage. + assert self.mock_tracer.session_id is None + mock_uuid.uuid4.assert_not_called() + mock_create.assert_not_called() + mock_log.assert_called() + # ======================================================================== # Tests for _validate_configuration_gracefully # ======================================================================== diff --git a/tests/unit/test_tracer_processing_span_processor.py b/tests/unit/test_tracer_processing_span_processor.py index 83fcfce4..67b141fd 100644 --- a/tests/unit/test_tracer_processing_span_processor.py +++ b/tests/unit/test_tracer_processing_span_processor.py @@ -776,6 +776,120 @@ def test_on_start_exception_handling(self, mock_safe_log: Mock) -> None: mock_safe_log.assert_called() + @patch("honeyhive.utils.logger.safe_log") + def test_on_start_stamps_auto_create_when_flag_is_true( + self, mock_safe_log: Mock + ) -> None: + """When tracer._session_auto_create is True, the processor stamps + honeyhive.session_auto_create=True on the span.""" + mock_tracer = Mock(spec=HoneyHiveTracer) + mock_tracer.session_id = "tracer-session" + mock_tracer.session_name = "my-session" + mock_tracer._session_auto_create = True + processor = HoneyHiveSpanProcessor(tracer_instance=mock_tracer) + + mock_span = Mock(spec=Span) + mock_span.name = "test_span" + mock_span.get_span_context.return_value = Mock(span_id=12345) + mock_context = Mock(spec=Context) + + processor.on_start(mock_span, mock_context) + + attribute_calls = { + call.args[0]: call.args[1] + for call in mock_span.set_attribute.call_args_list + if len(call.args) == 2 + } + assert attribute_calls.get("honeyhive.session_auto_create") is True + assert attribute_calls.get("honeyhive.session_name") == "my-session" + + @patch("honeyhive.utils.logger.safe_log") + def test_on_start_does_not_stamp_auto_create_when_flag_is_false( + self, mock_safe_log: Mock + ) -> None: + """When tracer._session_auto_create is not set, the processor does NOT + stamp honeyhive.session_auto_create on the span.""" + mock_tracer = Mock(spec=HoneyHiveTracer) + mock_tracer.session_id = "tracer-session" + mock_tracer.session_name = None + # _session_auto_create deliberately unset (getattr default = False) + del mock_tracer._session_auto_create + processor = HoneyHiveSpanProcessor(tracer_instance=mock_tracer) + + mock_span = Mock(spec=Span) + mock_span.name = "test_span" + mock_span.get_span_context.return_value = Mock(span_id=12345) + mock_context = Mock(spec=Context) + + processor.on_start(mock_span, mock_context) + + attribute_calls = { + call.args[0]: call.args[1] + for call in mock_span.set_attribute.call_args_list + if len(call.args) == 2 + } + assert "honeyhive.session_auto_create" not in attribute_calls + + @patch("honeyhive.tracer.processing.span_processor.baggage.get_baggage") + @patch("honeyhive.utils.logger.safe_log") + def test_on_start_prefers_baggage_session_name_over_tracer_instance( + self, mock_safe_log: Mock, mock_get_baggage: Mock + ) -> None: + """When both baggage and tracer instance carry session_name, the + per-request baggage value wins (mirrors session_id precedence).""" + mock_tracer = Mock(spec=HoneyHiveTracer) + mock_tracer.session_id = "tracer-session" + mock_tracer.session_name = "init-time-name" + mock_tracer._session_auto_create = True + processor = HoneyHiveSpanProcessor(tracer_instance=mock_tracer) + + mock_span = Mock(spec=Span) + mock_span.name = "test_span" + mock_span.get_span_context.return_value = Mock(span_id=12345) + mock_context = Mock(spec=Context) + + mock_get_baggage.side_effect = lambda key, ctx: { + "session_id": "tracer-session", + "session_name": "per-request-name", + }.get(key) + + processor.on_start(mock_span, mock_context) + + attribute_calls = { + call.args[0]: call.args[1] + for call in mock_span.set_attribute.call_args_list + if len(call.args) == 2 + } + assert attribute_calls.get("honeyhive.session_auto_create") is True + assert attribute_calls.get("honeyhive.session_name") == "per-request-name" + + @patch("honeyhive.utils.logger.safe_log") + def test_on_start_stamps_auto_create_but_not_name_when_session_name_is_none( + self, mock_safe_log: Mock + ) -> None: + """When tracer._session_auto_create is True and session_name is None, + only the auto-create flag is stamped (no honeyhive.session_name).""" + mock_tracer = Mock(spec=HoneyHiveTracer) + mock_tracer.session_id = "tracer-session" + mock_tracer.session_name = None + mock_tracer._session_auto_create = True + processor = HoneyHiveSpanProcessor(tracer_instance=mock_tracer) + + mock_span = Mock(spec=Span) + mock_span.name = "test_span" + mock_span.get_span_context.return_value = Mock(span_id=12345) + mock_context = Mock(spec=Context) + + processor.on_start(mock_span, mock_context) + + attribute_calls = { + call.args[0]: call.args[1] + for call in mock_span.set_attribute.call_args_list + if len(call.args) == 2 + } + assert attribute_calls.get("honeyhive.session_auto_create") is True + assert "honeyhive.session_name" not in attribute_calls + class TestHoneyHiveSpanProcessorOnEnd: """Test on_end method functionality with all conditional branches.""" diff --git a/tox.ini b/tox.ini index d0d268d6..ce135647 100644 --- a/tox.ini +++ b/tox.ini @@ -233,6 +233,9 @@ passenv = AWS_SESSION_TOKEN AWS_DEFAULT_REGION BEDROCK_MODEL_ID + AZURE_OPENAI_API_KEY + AZURE_OPENAI_ENDPOINT + AZURE_OPENAI_DEPLOYMENT CI GITHUB_ACTIONS @@ -242,8 +245,8 @@ deps = {[testenv]deps} openai>=1.0.0 openinference-instrumentation-openai>=0.1.0 - opentelemetry-instrumentation-openai>=0.46.0 -commands = + opentelemetry-instrumentation-openai>=0.58.0,<1.0.0 +commands = pytest tests/integration/test_openai_integration.py {posargs} --asyncio-mode=auto --tb=short setenv = PYTHONPATH = {toxinidir}/src @@ -334,6 +337,26 @@ passenv = BEDROCK_MODEL_ID CI +[testenv:integrations-azure-openai] +description = run Azure OpenAI integration tests only (requires Azure credentials) +deps = + {[testenv]deps} + openai>=1.0.0 + openinference-instrumentation-openai>=0.1.0 + opentelemetry-instrumentation-openai>=0.58.0,<1.0.0 +commands = + pytest tests/integration/test_azure_openai_integration.py {posargs} --asyncio-mode=auto --tb=short +setenv = + PYTHONPATH = {toxinidir}/src +passenv = + HH_API_KEY + HH_PROJECT + HH_API_URL + AZURE_OPENAI_API_KEY + AZURE_OPENAI_ENDPOINT + AZURE_OPENAI_DEPLOYMENT + CI + [testenv:integrations-evaluate] description = run evaluate() integration tests only deps = @@ -457,7 +480,6 @@ passenv = # Azure OpenAI configuration AZURE_OPENAI_ENDPOINT AZURE_OPENAI_API_KEY - AZURE_OPENAI_DEPLOYMENT_NAME AZURE_OPENAI_API_VERSION AZURE_OPENAI_DEPLOYMENT AZURE_OPENAI_GPT4_DEPLOYMENT