Skip to content

Commit 20d702e

Browse files
committed
Small documentation update to include how to _use_ InferenceObjective in the API summary
1 parent 198e6ca commit 20d702e

File tree

1 file changed

+5
-1
lines changed

1 file changed

+5
-1
lines changed

site-src/api-types/inferenceobjective.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,10 @@
99

1010
The **InferenceObjective** API defines a set of serving objectives of the specific request it is associated with. This CRD currently houses only `Priority` but will be expanded to include fields such as SLO attainment.
1111

12+
## Usage
13+
14+
To associate a request to the InferencePool with a specific InferenceObjective, the system uses a specific header: `x-gateway-inference-objective` with the value of the header set to the InferenceObjective metadata name. So the calling client must set the header key/value on the request to associate the selected InferenceObjective. If no InferenceObjective is selected, default values are used.
15+
1216
## Spec
1317

14-
The full spec of the InferenceModel is defined [here](/reference/x-spec/#inferenceobjective).
18+
The full spec of the InferenceObjective is defined [here](/reference/x-spec/#inferenceobjective).

0 commit comments

Comments
 (0)