1
1
# Passing information through context
2
2
3
- When working with LLMs, there are often different categories of information
4
- being handled simultaneously:
3
+ There are different categories of information that an LLM may be handling
4
+ simultaneously:
5
5
6
6
- ** Input:** Information that is directly relevant to guide the LLM's response
7
- for a particular call such as the text that needs to be summarized.
8
- - ** Generation Context:** Information that is relevant to the LLM but isn't
9
- specific to the call, such as the current time or a user's name.
7
+ for a particular call. An example of this is the text that needs to be
8
+ summarized.
9
+ - ** Generation Context:** Information that is relevant to the LLM, but isn't
10
+ specific to the call. An example of this is the current time or a user's name.
10
11
- ** Execution Context:** Information that is important to the code surrounding
11
- the LLM call but not to the LLM itself, e.g. a user's current auth token.
12
+ the LLM call but not to the LLM itself. An example of this is a user's
13
+ current auth token.
12
14
13
15
Genkit provides a consistent ` context ` object that can propagate generation and
14
16
execution context throughout the process. This context is made available to all
15
17
actions including [ flows] ( flows ) , [ tools] ( tool-calling ) , and
16
18
[ prompts] ( dotprompt ) .
17
19
18
20
Context is automatically propagated to all actions called within the scope of
19
- execution - context passed to a flow will be made available to prompts executed
20
- within the flow. Context passed to the ` generate() ` method will be available to
21
+ execution: Context passed to a flow is made available to prompts executed
22
+ within the flow. Context passed to the ` generate() ` method is available to
21
23
tools called within the generation loop.
22
24
23
25
## Why is context important?
@@ -27,7 +29,7 @@ LLM that it needs to complete a task. This is important for multiple reasons:
27
29
28
30
- The less extraneous information the LLM has, the more likely it is to perform
29
31
well at its task.
30
- - If an LLM needs to pass around information like user or account ids to tools,
32
+ - If an LLM needs to pass around information like user or account IDs to tools,
31
33
it can potentially be tricked into leaking information.
32
34
33
35
Context gives you a side channel of information that can be used by any of your
@@ -37,7 +39,7 @@ allow you to restrict tool queries to the current user's available scope.
37
39
## Context structure
38
40
39
41
Context must be an object, but its properties are yours to decide. In some
40
- situations Genkit will automatically populate context. For example, when using
42
+ situations Genkit automatically populates context. For example, when using
41
43
[ persistent sessions] ( chat ) the ` state ` property is automatically added to
42
44
context.
43
45
@@ -55,12 +57,12 @@ user. We recommend adding auth context in the following format:
55
57
}
56
58
```
57
59
58
- The context object can store any information that you might need to know somewhere
59
- else in the flow of execution.
60
+ The context object can store any information that you might need to know
61
+ somewhere else in the flow of execution.
60
62
61
63
## Use context in an action
62
64
63
- To use context within an action, you can access the provided context helper
65
+ To use context within an action, you can access the context helper
64
66
that is automatically supplied to your function definition:
65
67
66
68
* {Flow}
@@ -149,7 +151,7 @@ when calling the action.
149
151
150
152
By default , when you provide context it is automatically propagated to all
151
153
actions called as a result of your original call . If your flow calls other
152
- flows , or your generation calls tools , the same context will be provided .
154
+ flows , or your generation calls tools , the same context is provided .
153
155
154
156
If you wish to override context within an action , you can pass a different
155
157
context object to replace the existing one :
@@ -167,6 +169,6 @@ const myFlow = ai.defineFlow({
167
169
});
168
170
` ` `
169
171
170
- When context is replaced it propagates the same way . In the above example ,
172
+ When context is replaced , it propagates the same way . In this example ,
171
173
any actions that ` otherFlow ` called during its execution would inherit the
172
- overridden context .
174
+ overridden context .
0 commit comments