Skip to content

refactor(plugin-local-ai): migrate build system to Bun and modularize architecture #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
odilitime wants to merge 1 commit into
base: 1.x
Choose a base branch
from
Open

refactor(plugin-local-ai): migrate build system to Bun and modularize architecture #13

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 101 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# Changelog

All notable changes to `@elizaos/plugin-local-ai` are documented in this file.
Format based on [Keep a Changelog](https://keepachangelog.com/). Newest entries first.

## [Unreleased]

### Added

- **Startup configuration banner** (`src/banner.ts`)
- Displays compact table showing models directory, cache directory, loaded model filenames, and embedding dimensions
- Shows status indicators: `(default)` for default values, `(set)` for user-configured values
- **WHY**: Local AI involves multiple file paths and configurations. The banner makes it immediately obvious which models are loaded and where they're stored, critical for debugging "model not found" errors.

- **Modular architecture** (`src/models/`, `src/utils/`, `src/constants.ts`, `src/manager.ts`)
- Extracted `LocalAIManager` from `index.ts` to dedicated `manager.ts` (core logic)
- Created `constants.ts` for shared defaults (`wordsToPunish`, default model filenames)
- Created `utils/config.ts` with helper functions: `getSetting`, `getSmallModel`, `getLargeModel`, `getEmbeddingModel`
- Created `models/` directory with separate files: `text.ts`, `embedding.ts`, `image.ts`, `audio.ts`, `tokenizer.ts`
- **WHY**: Original `index.ts` was 2000+ lines mixing model handlers, initialization, config resolution, and manager logic. Smaller focused modules are easier to test, debug, and maintain. Follows pattern from `@elizaos/plugin-openai`.

- **Runtime-aware configuration resolution** (`src/utils/config.ts`, model handlers)
- `getSetting(runtime, key, defaultValue)` checks `runtime.getSetting()` first, then falls back to `process.env`
- `LocalAIManager.initializeEnvironment(runtime?)` accepts optional `IAgentRuntime` parameter
- All model handlers pass `runtime` to manager initialization
- **WHY**: Plugin must respect both character-level settings (via `runtime.getSetting`) and environment variables. Original code only checked `process.env`, completely ignoring character configuration.

- **Consistent logging prefixes** (all modules)
- Added `[LocalAI]` prefix to all logger calls
- Removed emoji from log messages
- **WHY**: Makes logs greppable (`grep '\[LocalAI\]' logs.txt`) and clearly identifies which plugin generated each message. Emoji breaks certain terminal configurations and log parsers.

### Changed

- **Build system: tsup → Bun.build** (`build.ts`, `package.json`)
- Replaced `tsup.config.ts` with custom `build.ts` using `@elizaos/build-utils`
- Uses monorepo's shared `createBuildRunner` for consistent configuration
- Generates Node ESM, browser bundle, and CJS outputs
- Declaration files via `tsc --emitDeclarationOnly`
- Moved `tsup` from `dependencies` to `devDependencies`
- **WHY**: Standardizing build configuration across all plugins reduces maintenance burden. `tsup` had issues with native dependency bundling; Bun's builder handles them correctly and is significantly faster.

- **Test framework: vitest → bun:test** (`__tests__/*.test.ts`, `__tests__/test-utils.ts`)
- Converted all test files from `vitest` to Bun's native test runner
- Replaced `vi.fn()` with `mock()`, `describe`/`expect`/`test` now from `bun:test`
- Added `mock.module()` calls at top of each test file to mock native dependencies BEFORE plugin imports them
- Mocked dependencies: `node-llama-cpp`, `@huggingface/transformers`, `nodejs-whisper`, `node:child_process`
- **WHY**: `node-llama-cpp` has incompatible import patterns with vitest's module mocking, causing `SyntaxError: Missing 'default' export in module 'node:fs'`. The issue was that static imports of native dependencies happened before test setup, and vitest couldn't intercept them. Bun's `mock.module()` hoists mocks before imports execute, preventing load-time crashes. Additionally, Bun's test runner is 3-5x faster.

### Technical Details

#### Module-level mocking pattern (critical for native dependencies)

The plugin uses heavy native dependencies (`node-llama-cpp` for GGUF models, `@huggingface/transformers` for vision/TTS, `nodejs-whisper` for transcription) that load WASM/native bindings at import time. In test environments, these bindings fail because models aren't present.

**Problem**: Static imports execute before test setup code:
```typescript
// ❌ This fails — node-llama-cpp loads native code before mock() runs
import { localAiPlugin } from '../src/index';
mock.module('node-llama-cpp', () => ({ ... }));
```

**Solution**: Hoist mocks using `mock.module()` at the TOP of test files:
```typescript
import { mock, describe, expect, test } from 'bun:test';

// ✅ Mock BEFORE any plugin imports
mock.module('node-llama-cpp', () => ({
getLlama: mock(async () => ({ /* mock implementation */ })),
LlamaChatSession: class {},
// ... other exports
}));

// NOW safe to import plugin (it sees the mocked module)
import { localAiPlugin } from '../src/index';
```

Bun's `mock.module()` is hoisted like `jest.mock()` — it executes before any imports in the file, regardless of source order.

#### Configuration resolution flow

1. Model handlers call manager: `await localAIManager.initializeEnvironment(runtime)`
2. Manager calls config helpers: `const smallModel = getSetting(runtime, 'LOCAL_SMALL_MODEL')`
3. Config helper checks runtime first: `runtime.getSetting(key)` (character settings)
4. Falls back to environment: `process.env[key]`
5. Returns default if neither exist: `defaultValue`

This ensures character-level configuration overrides environment variables, which override plugin defaults.

## [1.6.6-alpha.5] - 2026-01-XX
Copy link

@coderabbitai coderabbitai bot Feb 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Placeholder date 2026-01-XX should be filled in.

The version entry has an incomplete date. Replace with the actual release date before merging.

🤖 Prompt for AI Agents 
In `@CHANGELOG.md` at line 90, Replace the placeholder date in the changelog
header "## [1.6.6-alpha.5] - 2026-01-XX" with the actual release date (e.g.,
"2026-01-14") so the entry reads "## [1.6.6-alpha.5] - YYYY-MM-DD"; ensure the
format matches other entries in CHANGELOG.md.


### Changed

- **Logger API compatibility**: Updated all logging to use string-only format (no object arguments)
- **Model handler signatures**: Aligned with `@elizaos/core` union types (`string | ParamsType | null`)
- **GenerateText refactor**: Separated `modelType` from `GenerateTextParams` (passed as explicit parameter)
- **node-llama-cpp upgrade**: Updated from v3.10.0 to v3.15.1 for better GGUF support

### Fixed

- All handlers now properly handle `null` and `string` parameter types in addition to typed objects
143 changes: 143 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,8 +99,151 @@ const audioStream = await runtime.useModel(ModelType.TEXT_TO_SPEECH, 'Text to co
const transcription = await runtime.useModel(ModelType.TRANSCRIPTION, audioBuffer);
```

## Architecture

The plugin uses a modular architecture with clear separation of concerns:

```
src/
index.ts Plugin entry — registers model handlers
manager.ts LocalAIManager singleton — coordinates all model operations
banner.ts Startup configuration display
constants.ts Shared constants and defaults
models/
text.ts TEXT_SMALL, TEXT_LARGE handlers
embedding.ts TEXT_EMBEDDING handler
image.ts IMAGE_DESCRIPTION handler
audio.ts TEXT_TO_SPEECH, TRANSCRIPTION handlers
Copy link

Copilot AI Feb 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The architecture diagram is missing object.ts from the models/ directory listing. The file exists (handles OBJECT_SMALL and OBJECT_LARGE model types) but is not documented in the structure overview.

Suggested change
audio.ts TEXT_TO_SPEECH, TRANSCRIPTION handlers
audio.ts TEXT_TO_SPEECH, TRANSCRIPTION handlers
object.ts OBJECT_SMALL, OBJECT_LARGE handlers

Copilot uses AI. Check for mistakes.
tokenizer.ts TEXT_TOKENIZER_ENCODE/DECODE handlers
utils/
config.ts Configuration resolution (runtime.getSetting)
environment.ts Environment setup and model downloading
```

**Why this structure?**

- **Separation of concerns**: Model handlers are isolated from initialization logic
- **Testability**: Each module can be tested independently with mocked dependencies
- **Maintainability**: Configuration logic centralized in `utils/config.ts`, reducing duplication
- **Consistency**: Follows the pattern established in `@elizaos/plugin-openai` for easier cross-plugin maintenance

### Startup Banner

On initialization, the plugin displays its configuration:

```
+----------------------------------------------------------------------+
| Local AI Plugin |
+----------------------------+-----------------------------------------+
| MODELS_DIR | /root/.eliza/models (default) |
| CACHE_DIR | /root/.eliza/cache (default) |
| Small model | DeepHermes-3-Llama-3-3B-... (set) |
| Large model | DeepHermes-3-Llama-3-8B-... (set) |
| Embedding model | bge-small-en-v1.5.Q4_K_M... (set) |
| Embedding dimensions | 384 (default) |
+----------------------------------------------------------------------+
```

**Why a banner?** Local AI models involve multiple file paths and configurations. The banner makes it immediately obvious which models are loaded, where they're stored, and whether defaults are being used. This is critical for debugging "model not found" errors.

### Build System

The plugin uses Bun's native builder instead of `tsup`:

- **`build.ts`**: Custom build script using `@elizaos/build-utils`
- **Multi-format output**: Node ESM, browser bundle, and CJS for maximum compatibility
- **Declaration files**: TypeScript declarations generated via `tsc --emitDeclarationOnly`

**Why Bun.build?** Faster builds, better tree-shaking, and consistent configuration across the monorepo. The `build-utils` package provides shared configuration so all plugins build identically.

## Testing

The plugin uses **Bun's native test runner** (`bun:test`) instead of vitest or jest:

```bash
bun test # Run all tests
bun test --watch # Watch mode
bun test specific.test.ts # Run specific file
```

**Why Bun test?** Native dependencies like `node-llama-cpp` have import compatibility issues with vitest. Bun's test runner handles native modules correctly and is significantly faster.

**Test structure**:
- `__tests__/test-utils.ts`: Mock factories for `IAgentRuntime` and heavy dependencies
- `__tests__/initialization.test.ts`: Plugin initialization and config loading
- `__tests__/text-gen.test.ts`: Text generation handlers
- `__tests__/image-desc.test.ts`: Image description handler
- `__tests__/text-transcribe.test.ts`: Audio transcription handler
- `__tests__/tts.test.ts`: Text-to-speech handler

**Critical pattern**: Heavy native dependencies (`node-llama-cpp`, `@huggingface/transformers`) are mocked at the module level using `mock.module()` **before** the plugin imports them. This prevents load-time crashes from WASM/native bindings in test environments.

Example:
```typescript
import { mock, describe, expect, test } from 'bun:test';

// Mock BEFORE importing plugin
mock.module('node-llama-cpp', () => ({
getLlama: mock(async () => ({ /* mock llama */ })),
// ... other mocks
}));

// NOW safe to import
import { localAiPlugin } from '../src/index';
```

## Recent Updates

### February 2026 - Modernization & Build System Update

**Plugin Structure Refactor:**

1. **Modularization**:
- Extracted `LocalAIManager` from `index.ts` to dedicated `manager.ts`
- Created `constants.ts` for shared defaults (`wordsToPunish`, model filenames)
- Created `utils/config.ts` for configuration resolution helpers
- Created `models/` directory with separate files per model type
- Created `banner.ts` for startup configuration display
- **WHY**: The original `index.ts` was 2000+ lines and mixing concerns. Smaller, focused modules are easier to test, debug, and maintain.

2. **Runtime-aware configuration**:
- All config helpers now accept `runtime?: IAgentRuntime` parameter
- Use `runtime.getSetting(key)` when available, falling back to `process.env`
- `LocalAIManager.initializeEnvironment(runtime?)` accepts optional runtime
- Model handlers pass `runtime` to manager initialization
- **WHY**: The plugin needs to respect both character settings (via `runtime.getSetting`) and environment variables. The original code only checked `process.env`, ignoring character-level configuration.

3. **Build system modernization**:
- Replaced `tsup` with custom `build.ts` using `Bun.build`
- Uses monorepo's shared `@elizaos/build-utils` for consistency
- Generates Node ESM, browser bundle, and CJS outputs
- **WHY**: Standardizing build configuration across all plugins. `tsup` had issues with native dependency bundling; Bun's builder handles them correctly.

4. **Logging consistency**:
- Added `[LocalAI]` prefix to all logger calls
- Removed emoji from log messages (emoji breaks certain terminal configs)
- **WHY**: Makes logs greppable and clearly identifies which plugin generated each message.

**Test Suite Overhaul:**

1. **vitest → bun:test migration**:
- Converted all test files from `vitest` to Bun's native test runner
- Replaced `vi.fn()` with `mock()`
- Replaced `vi.mocked()` with direct `mock()` usage
- Added `mock.module()` calls for native dependencies
- **WHY**: `node-llama-cpp` has incompatible import patterns with vitest's module mocking system, causing `SyntaxError: Missing 'default' export in module 'node:fs'`. Bun's test runner handles native ESM imports correctly.

2. **Module-level mocking pattern**:
- `mock.module()` must appear at the TOP of each test file
- Mocks are hoisted before any imports execute
- Prevents load-time crashes from WASM/native bindings
- **WHY**: Static imports happen before test setup code runs. If `node-llama-cpp` loads its native bindings during import, it crashes the test process. Hoisted `mock.module()` intercepts the import before any native code executes.

**Configuration:**
- No breaking changes to user-facing configuration
- All existing environment variables work identically
- Character-level settings now properly respected

### v1.6.6-alpha.5 (January 2026)

**Core API Compatibility Updates:**
Expand Down
Loading