Skip to content

feat: Recommend AppMap recordings inline in chat responses #2268

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions packages/navie/src/agents/explain-agent.ts
Original file line number Diff line number Diff line change
Expand Up @@ -263,6 +263,47 @@ classDiagram
</example>
`;

const APPMAP_HINT_PROMPT = `
## Recommending AppMap Recordings

As you provide your response, consider whether your statements could be improved if the user had provided
additional context in the form of an AppMap recording - essentially, runtime traces. If so, add an
\`appmap\` XML tag following the statement that could be improved. This will prompt the user to provide such information; the tags
will be used to provide specialized UI to help the user accomplish this task. You can use many of these tags spread throughout your
response text as you need, each associated with a specific statement that could be made more accurate with this data.

Provide a \`prompt\` attribute to the \`appmap\` tag, which will be used to initiate a conversation with an assistant to help
the user create AppMaps. You should use \`@observe\` prefix in the prompt which will choose the correct assistant.

You can additionally provide a \`reasoning\` attribute to the \`appmap\` tag, which will be used to explain
how an AppMap recording would provide additional context.

Make sure that all recommendations for AppMap recordings are kept only to these tags ONLY. Do not mention AppMap in the
running text of your response unless the user has specifically asked for it.

Note recording AppMaps is currently only supported in Ruby, Python, Java and JavaScript (server-side only, ie. Node.js) applications.
Do not recommend AppMap recordings for other languages or environments.

Examples:

\`\`\`markdown
If you suspect redundant calls to the \`foo\` method, you should investigate further. Check to see if the \`foo\` method is being
called more than once <appmap prompt="@observe record and analyze tests that involve the foo method to identify redundant calls" />.
This will help you identify and eliminate inefficiencies in your code.
\`\`\`

\`\`\`markdown
The JWT token might not be properly stored or transmitted in subsequent requests. This could lead to issues with authentication
or session management. <appmap reasoning="An AppMap trace would show if the JWT token is being properly included in request headers after login"
prompt="@observe record a session including the authentication flow, focusing on login and subsequent API requests to verify token usage" />. Ensuring proper token handling is critical for secure communication.
\`\`\`

\`\`\`markdown
Authentication filters can sometimes cause unexpected behavior. Check that your authentication filter isn't accidentally catching the login endpoints themselves <appmap reasoning="An AppMap trace would reveal if the auth filter is intercepting login requests, causing a loop"
prompt="@observe record both authenticated and unauthenticated HTTP requests, including login attempts, to identify potential filter misconfigurations" />. This will help you avoid potential infinite loops or access issues.
\`\`\`
`;

export default class ExplainAgent implements Agent {
public temperature = undefined;

Expand All @@ -284,6 +325,16 @@ export default class ExplainAgent implements Agent {
if (hasLabel(options.contextLabels, ContextV2.ContextLabelName.GenerateDiagram))
this.history.addEvent(new PromptInteractionEvent('agent', 'system', DIAGRAM_FORMAT_PROMPT));

if (
// Do not prompt for AppMap recordings if the user is greeting or chatting.
!options.contextLabels?.find((label) =>
[ContextV2.ContextLabelName.Greeting, ContextV2.ContextLabelName.Chatting].includes(
label.name
)
)
)
this.history.addEvent(new PromptInteractionEvent('agent', 'system', APPMAP_HINT_PROMPT));

this.history.addEvent(
new PromptInteractionEvent(
PromptType.Question,
Expand Down
16 changes: 11 additions & 5 deletions packages/navie/test/agents/explain-agent.spec.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,4 @@
import ExplainAgent from '../../src/agents/explain-agent';
import InteractionHistory, {
ContextItemRequestor,
isPromptEvent,
} from '../../src/interaction-history';
import ApplyContextService from '../../src/services/apply-context-service';
import VectorTermsService from '../../src/services/vector-terms-service';
import LookupContextService from '../../src/services/lookup-context-service';
Expand All @@ -12,8 +8,8 @@ import { CHARACTERS_PER_TOKEN } from '../../src/message';
import { UserOptions } from '../../src/lib/parse-options';
import ContextService from '../../src/services/context-service';
import MermaidFixerService from '../../src/services/mermaid-fixer-service';
import mermaid from 'mermaid';
import { ContextV2 } from '../../src/context';
import InteractionHistory, { ContextItemRequestor } from '../../src/interaction-history';

describe('@explain agent', () => {
let interactionHistory: InteractionHistory;
Expand Down Expand Up @@ -114,6 +110,11 @@ describe('@explain agent', () => {
role: 'system',
name: 'agent',
},
{
type: 'prompt',
role: 'system',
name: 'agent',
},
{
type: 'prompt',
role: 'system',
Expand Down Expand Up @@ -202,6 +203,11 @@ describe('@explain agent', () => {
role: 'system',
name: 'agent',
},
{
type: 'prompt',
role: 'system',
name: 'agent',
},
{
type: 'prompt',
role: 'system',
Expand Down
Loading