Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
118 changes: 33 additions & 85 deletions __tests__/unit/context.test.ts
Original file line number Diff line number Diff line change
@@ -1,111 +1,59 @@
import { describe, test, expect } from "bun:test";
import { buildConversationContext } from "../../src/utils/context";
import {
createMockRuntime,
createMockMessage,
createMockState,
} from "../helpers/mockRuntime";
import { describe, test, expect } from 'bun:test';
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor - Code Style: Good job removing the unused createMockRuntime import. The test cleanup is thorough and maintains good coverage.

import { buildConversationContext } from '../../src/utils/context';
import { createMockMessage, createMockState } from '../helpers/mockRuntime';
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused Import: createMockRuntime is no longer imported but the test file doesn't need it anymore since the runtime parameter was removed. Good cleanup!


describe("buildConversationContext", () => {
test("returns message text when no recent messages", () => {
const runtime = createMockRuntime();
describe('buildConversationContext', () => {
test('returns message text when no recent messages in values', () => {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test Coverage Gap: The old tests validated important behaviors that are now gone:

  1. Message limiting (5 messages max) - removed in the refactor but not explicitly verified to be unnecessary
  2. Role labeling (User vs Assistant) - now delegated to provider, but should we test the provider contract?

Consider adding a test that verifies behavior when state.values.recentMessages is set to an unexpected type (number, object, etc.) to ensure graceful degradation.

const message = createMockMessage({
content: { text: "Activate my workflow" },
content: { text: 'Activate my workflow' },
});
const state = createMockState();

const result = buildConversationContext(runtime, message, state);
expect(result).toBe("Activate my workflow");
const result = buildConversationContext(message, state);
expect(result).toBe('Activate my workflow');
});

test("returns empty string when no text and no recent messages", () => {
const runtime = createMockRuntime();
const message = createMockMessage({ content: { text: "" } });
test('returns empty string when no text and no recent messages', () => {
const message = createMockMessage({ content: { text: '' } });
const state = createMockState();

const result = buildConversationContext(runtime, message, state);
expect(result).toBe("");
const result = buildConversationContext(message, state);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test Clarity: This test expects an empty string when both text and recentMessages are empty. This is correct, but consider adding a test case for when message.content.text is undefined (not just empty string) to verify the || '' fallback works correctly.

expect(result).toBe('');
});

test("handles undefined state", () => {
const runtime = createMockRuntime();
const message = createMockMessage({ content: { text: "Hello" } });
test('handles undefined state', () => {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing Edge Case Test: Consider adding a test for when message.content itself is undefined, not just when content.text is empty:

test('handles undefined message content', () => {
  const message = createMockMessage({ content: undefined });
  const state = createMockState();

  const result = buildConversationContext(message, undefined);
  expect(result).toBe('');
});

This would ensure the type safety issue mentioned in context.ts:6 is covered.

const message = createMockMessage({ content: { text: 'Hello' } });

const result = buildConversationContext(runtime, message, undefined);
expect(result).toBe("Hello");
const result = buildConversationContext(message, undefined);
expect(result).toBe('Hello');
});

test("includes recent messages in context", () => {
const runtime = createMockRuntime({ agentId: "agent-001" });
const message = createMockMessage({ content: { text: "Activate it" } });
test('appends current request to recentMessages', () => {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test Coverage: The tests now validate the new string-based format, which is good. However, consider adding tests for:

  1. Malformed input: What happens if recentMessages is not a string?
  2. Very long messages: Test behavior with extremely long recentMessages strings
  3. Special characters: Test with messages containing newlines, control characters, or other special chars
  4. Empty strings vs undefined: Test the difference between recentMessages: '' and recentMessages: undefined

These edge cases are important for robustness in production.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test Coverage Gap: Consider adding a test for when state.values.recentMessages is an empty string (not undefined):

test('handles empty string recentMessages', () => {
  const message = createMockMessage({ content: { text: 'Test' } });
  const state = createMockState({
    values: { recentMessages: '' },
  });

  const result = buildConversationContext(message, state);
  expect(result).toBe('Test');
});

This would clarify the expected behavior for the falsy check on line 8 of context.ts.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test Coverage - Missing Case: Consider adding a test for when state.values.recentMessages exists but is not a string type. This would verify the type guard on line 5 of context.ts works correctly.

Example:

test('handles non-string recentMessages gracefully', () => {
  const message = createMockMessage({ content: { text: 'Hello' } });
  const state = createMockState({
    values: { recentMessages: { invalid: 'object' } },
  });
  
  const result = buildConversationContext(message, state);
  expect(result).toBe('Hello'); // Should fall back to just the message
});

const message = createMockMessage({ content: { text: 'Activate it' } });
const state = createMockState({
data: {
recentMessages: [
{
entityId: "user-001",
content: { text: "Show me my workflows" },
},
{
entityId: "agent-001",
content: { text: "Here are your workflows: Stripe, Gmail" },
},
],
values: {
recentMessages:
'User: Show me my workflows\nAssistant: Here are your workflows: Stripe, Gmail',
},
});

const result = buildConversationContext(runtime, message, state);
expect(result).toContain("User: Show me my workflows");
expect(result).toContain("Assistant: Here are your workflows");
expect(result).toContain("Current request: Activate it");
const result = buildConversationContext(message, state);
expect(result).toContain('User: Show me my workflows');
expect(result).toContain('Assistant: Here are your workflows');
expect(result).toContain('Current request: Activate it');
});

test("limits to last 5 messages", () => {
const runtime = createMockRuntime({ agentId: "agent-001" });
const message = createMockMessage({ content: { text: "Current" } });

const messages = Array.from({ length: 10 }, (_, i) => ({
entityId: "user-001",
content: { text: `Message ${i}` },
}));
test('preserves recentMessages formatting from provider', () => {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing Test Coverage: While this test verifies that preformatted messages are preserved, there's no test coverage for:

  1. Non-string values in state.values.recentMessages - What happens if it's a number, object, or array? The as string assertion on line 4 of context.ts will silently coerce these.
  2. Edge cases - Empty string, whitespace-only strings, very long strings, strings with special characters/unicode

Consider adding tests like:

test('handles non-string recentMessages gracefully', () => {
  const message = createMockMessage({ content: { text: 'Test' } });
  const state = createMockState({
    values: { recentMessages: 123 }, // number instead of string
  });
  
  const result = buildConversationContext(message, state);
  expect(result).toBe('Test'); // Should fall back to message text
});

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good test coverage: This test verifies that the function preserves whatever formatting the provider supplies, which is exactly the right behavior. This demonstrates the simplified approach works correctly with different provider implementations.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good Test: This test validates that pre-formatted messages from providers are preserved as-is, which is important for the cloud compatibility fix. Nice addition! 👍

However, the test name "preserves recentMessages formatting from provider" could be more specific about what formatting is being preserved (timestamps, custom prefixes, etc.).

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent Test: This test is critical for ensuring the fix works correctly. It verifies that:

  1. Provider-formatted messages are preserved as-is (no re-parsing or reformatting)
  2. The "Current request" is properly appended
  3. Different formatting styles from different providers are supported

This is a key requirement for cross-provider compatibility. Great addition!

const message = createMockMessage({ content: { text: 'Do something' } });
const preformattedMessages = `[2024-01-01 10:00] Alice: Hello
[2024-01-01 10:01] Bot: Hi there!
[2024-01-01 10:02] Alice: Help me`;

const state = createMockState({
data: { recentMessages: messages },
});

const result = buildConversationContext(runtime, message, state);
// Should only contain the last 5 messages (5-9)
expect(result).not.toContain("Message 4");
expect(result).toContain("Message 5");
expect(result).toContain("Message 9");
});

test("labels agent messages as Assistant", () => {
const runtime = createMockRuntime({ agentId: "agent-001" });
const message = createMockMessage({ content: { text: "Next" } });
const state = createMockState({
data: {
recentMessages: [
{ entityId: "agent-001", content: { text: "Bot response" } },
],
},
});

const result = buildConversationContext(runtime, message, state);
expect(result).toContain("Assistant: Bot response");
});

test("labels non-agent messages as User", () => {
const runtime = createMockRuntime({ agentId: "agent-001" });
const message = createMockMessage({ content: { text: "Next" } });
const state = createMockState({
data: {
recentMessages: [
{ entityId: "someone-else", content: { text: "User msg" } },
],
},
values: { recentMessages: preformattedMessages },
});

const result = buildConversationContext(runtime, message, state);
expect(result).toContain("User: User msg");
const result = buildConversationContext(message, state);
expect(result).toBe(`${preformattedMessages}\n\nCurrent request: Do something`);
});
});
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing test coverage: Consider adding tests for edge cases:

  1. Non-string recentMessages: What happens if a provider mistakenly provides an array or object?
test('handles non-string recentMessages gracefully', () => {
  const message = createMockMessage({ content: { text: 'Test' } });
  const state = createMockState({
    values: { recentMessages: ['msg1', 'msg2'] }, // Wrong type
  });
  const result = buildConversationContext(message, state);
  expect(result).toBe('Test'); // Should fallback safely
});
  1. Undefined message.content: Verify it doesn't throw
test('handles undefined message content', () => {
  const message = createMockMessage({ content: undefined });
  const state = createMockState();
  expect(() => buildConversationContext(message, state)).not.toThrow();
});

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing Test Coverage: The tests don't cover important edge cases:

  1. What happens if state.values.recentMessages is not a string (e.g., array, number, object)?
  2. What happens if message.content.text is falsy but recentMessages exists?
  3. What happens if recentMessages is a very long string (potential token/context issues)?

Consider adding tests like:

test('handles non-string recentMessages gracefully', () => {
  const message = createMockMessage({ content: { text: 'Request' } });
  const state = createMockState({
    values: { recentMessages: ['msg1', 'msg2'] }, // Array instead of string
  });
  const result = buildConversationContext(message, state);
  expect(result).toBe('Request'); // Should fallback safely
});

test('handles empty current message with recentMessages', () => {
  const message = createMockMessage({ content: { text: '' } });
  const state = createMockState({
    values: { recentMessages: 'User: Previous message' },
  });
  const result = buildConversationContext(message, state);
  // Verify expected behavior for this edge case
});

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@elizaos/plugin-n8n-workflow",
"version": "1.0.10",
"version": "1.0.11",
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Version bump is appropriate: The change from 1.0.10 to 1.0.11 is a patch version bump, which is correct since this is a bug fix. However, note that removing the runtime parameter from buildConversationContext is technically a breaking change for any external code that imports this function.

If this utility is only used internally within the plugin (not exported in the public API), then patch version is fine. Otherwise, consider if this should be a minor version bump (1.1.0).

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Version Bump: ✅ Appropriate patch version bump (1.0.10 → 1.0.11) for a bug fix. This follows semantic versioning correctly.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Version Bump: Patch version bump (1.0.10 → 1.0.11) is appropriate for a bug fix. Good adherence to semantic versioning.

"description": "ElizaOS plugin for generating and managing n8n workflows from natural language",
"type": "module",
"main": "dist/index.js",
Expand Down
2 changes: 1 addition & 1 deletion src/actions/activateWorkflow.ts
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,7 @@ export const activateWorkflowAction: Action = {
return { success: false };
}

const context = buildConversationContext(runtime, message, state);
const context = buildConversationContext(message, state);
const matchResult = await matchWorkflow(runtime, context, workflows);

if (!matchResult.matchedWorkflowId || matchResult.confidence === 'none') {
Expand Down
2 changes: 1 addition & 1 deletion src/actions/createWorkflow.ts
Original file line number Diff line number Diff line change
Expand Up @@ -269,7 +269,7 @@ export const createWorkflowAction: Action = {
const userText = (content.text ?? '').trim();
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consistent refactoring: Good - all 5 action files (createWorkflow, activateWorkflow, deactivateWorkflow, deleteWorkflow, getExecutions) have been updated consistently to remove the runtime parameter from buildConversationContext.

const userId = message.entityId;
const cacheKey = `workflow_draft:${userId}`;
const generationContext = buildConversationContext(runtime, message, state);
const generationContext = buildConversationContext(message, state);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consistent Update: ✅ Correctly updated to use the new 2-parameter signature. This is the critical path where conversation context is used for workflow generation, so the fix here ensures the LLM gets proper context on both cloud and bootstrap providers.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct Usage: The runtime parameter has been correctly removed from the buildConversationContext call. This is the critical fix that enables cloud provider compatibility.

Verification: All 6 actions (createWorkflow, activateWorkflow, deactivateWorkflow, deleteWorkflow, getExecutions) have been updated consistently. Good job!

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good Practice: This is the most critical usage of buildConversationContext since it's used for workflow generation. The fix ensures that the LLM receives the actual conversation context (user's request) instead of a confirmation message like "yes" or "ok".

The PR description correctly identifies this as fixing workflows being generated from confirmation messages on cloud - this change directly addresses that issue.


try {
let existingDraft = await runtime.getCache<WorkflowDraft>(cacheKey);
Expand Down
2 changes: 1 addition & 1 deletion src/actions/deactivateWorkflow.ts
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ export const deactivateWorkflowAction: Action = {
return { success: false };
}

const context = buildConversationContext(runtime, message, state);
const context = buildConversationContext(message, state);
const matchResult = await matchWorkflow(runtime, context, workflows);

if (!matchResult.matchedWorkflowId || matchResult.confidence === 'none') {
Expand Down
2 changes: 1 addition & 1 deletion src/actions/deleteWorkflow.ts
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ export const deleteWorkflowAction: Action = {
return { success: false };
}

const context = buildConversationContext(runtime, message, state);
const context = buildConversationContext(message, state);
const matchResult = await matchWorkflow(runtime, context, workflows);

if (!matchResult.matchedWorkflowId || matchResult.confidence === 'none') {
Expand Down
2 changes: 1 addition & 1 deletion src/actions/getExecutions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ export const getExecutionsAction: Action = {
return { success: false };
}

const context = buildConversationContext(runtime, message, state);
const context = buildConversationContext(message, state);
const matchResult = await matchWorkflow(runtime, context, workflows);

if (!matchResult.matchedWorkflowId || matchResult.confidence === 'none') {
Expand Down
21 changes: 7 additions & 14 deletions src/utils/context.ts
Original file line number Diff line number Diff line change
@@ -1,22 +1,15 @@
import { type IAgentRuntime, type Memory, type State, type UUID } from '@elizaos/core';
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused Import: IAgentRuntime and UUID are no longer used after removing the runtime parameter. Consider cleaning up:

import { type Memory, type State } from '@elizaos/core';


export function buildConversationContext(
runtime: IAgentRuntime,
message: Memory,
state: State | undefined
): string {
const recentMessages = (state?.data?.recentMessages as Memory[]) || [];
export function buildConversationContext(message: Memory, state: State | undefined): string {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Breaking Change Alert: Removing the runtime parameter is a breaking change that affects the function signature. Ensure all callers have been updated.

Code Quality: The change from processing individual messages to receiving pre-formatted strings is good for simplification, but consider:

  1. Adding JSDoc to document the expected format of recentMessages
  2. Adding validation to ensure recentMessages is a string if present

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good refactoring: Removing the unused runtime parameter simplifies the API and reduces coupling. The function signature is now cleaner and more focused.

const raw = state?.values?.recentMessages;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Edge Case - Type Safety: While typeof raw === 'string' is a good runtime check, consider what happens if state.values.recentMessages exists but is not a string (e.g., it's an object or array due to a provider bug or version mismatch).

Current behavior: Falls back to empty string, which is safe but silent. Consider logging a warning if the value exists but has an unexpected type:

const raw = state?.values?.recentMessages;
if (raw !== undefined && typeof raw !== 'string') {
  logger.warn('state.values.recentMessages has unexpected type:', typeof raw);
}
const recentMessages = typeof raw === 'string' ? raw : '';

This would help catch provider contract violations during development.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TypeScript Best Practice: The type check here is good, but consider using a type guard for better type safety throughout the function:

const raw = state?.values?.recentMessages;
const recentMessages = typeof raw === 'string' ? raw : '';

This is already implemented correctly. Good defensive coding! ✅

However, if state.values.recentMessages could legitimately be other types (number, object, etc.), you might want to add logging when it's not a string to help debug configuration issues in different providers.

const recentMessages = typeof raw === 'string' ? raw : '';
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Type Safety Issue: This type check is too narrow. If state.values.recentMessages is not a string (e.g., an array, object, or number), it will be coerced to an empty string, potentially hiding data issues.

Consider explicit validation:

Suggested change
const recentMessages = typeof raw === 'string' ? raw : '';
const raw = state?.values?.recentMessages;
if (typeof raw !== 'string') {
return message.content.text || '';
}
const recentMessages = raw;

This would make unexpected types fail fast during development rather than silently converting to empty string.

const currentText = message.content?.text ?? '';
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Type Safety Issue: message.content could be undefined according to the Memory type from @elizaos/core. This would cause message.content?.text to be evaluated but message.content.text is accessed without the optional chaining on line 6.

Suggested change
const currentText = message.content?.text ?? '';
const currentText = (message.content?.text ?? '') as string;

This ensures type safety even if message.content is undefined.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Type Safety Concern: The use of message.content?.text with optional chaining is good, but message.content could potentially be undefined in the type system. Consider adding a guard or making the type contract more explicit.

However, looking at the usage in the codebase, all callers cast to Content type first (e.g., message.content as Content in createWorkflow.ts:268), so this is defensive programming which is good practice.


if (recentMessages.length === 0) {
return message.content.text || '';
if (!recentMessages) {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potential Logic Bug: This condition treats an empty string "" as falsy and returns only currentText. This means if the provider returns an empty string for recentMessages (instead of undefined), the context won't include "Current request:" prefix.

Consider making this check more explicit:

Suggested change
if (!recentMessages) {
if (recentMessages === '' || recentMessages === undefined) {

Or alternatively, if empty string should be treated the same as missing:

Suggested change
if (!recentMessages) {
if (!recentMessages || recentMessages.trim() === '') {

This makes the intention clearer and prevents potential bugs with different providers.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logic Question: Should an empty string be treated the same as undefined?

Currently, if (!recentMessages) will be true for both undefined and '' (empty string). This means if a provider sets state.values.recentMessages = '', you'd return just the current text without the "Current request:" prefix.

Is this intended behavior? If so, it's fine. If not, consider:

if (!recentMessages || recentMessages.length === 0) {

or be explicit about what you're checking.

return currentText;
}

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Security Consideration: The pre-formatted recentMessages string is now inserted directly into the context without any sanitization. If this context is used in prompts to LLMs, ensure that:

  1. The provider that creates recentMessages properly sanitizes input
  2. There's no risk of prompt injection through user-controlled message content
  3. Message length is bounded to prevent context overflow

Recommendation: Add length validation:

if (recentMessages.length &gt; 10000) { // reasonable limit
  return `${recentMessages.slice(-10000)}\n\nCurrent request: ${message.content.text || ''}`;
}

const context = recentMessages
.slice(-5)
.map((m) => `${m.entityId === runtime.agentId ? 'Assistant' : 'User'}: ${m.content.text}`)
.join('\n');

return `Recent conversation:\n${context}\n\nCurrent request: ${message.content.text || ''}`;
return `${recentMessages}\n\nCurrent request: ${currentText}`;
}

export async function getUserTagName(runtime: IAgentRuntime, userId: string): Promise<string> {
Expand Down