Conversation
…pdate build system Split monolithic src/index.ts into dedicated model handlers (text, audio, image, embedding, object, tokenizer) under src/models/. Add manager, constants, config utilities, and banner. Migrate from tsup to bun build. Update tests for new module structure and @elizaos/core API compatibility. Co-authored-by: Cursor <[email protected]>
WalkthroughThis PR refactors the Changes
Sequence Diagram(s)sequenceDiagram
actor User
participant Runtime as IAgentRuntime
participant Config as config.ts
participant Manager as LocalAIManager
participant Handler as Model Handler<br/>(e.g., text.ts)
participant Model as Local Model<br/>(Llama/Florence)
User->>Handler: Call handleTextSmall(runtime, params)
Handler->>Config: getSetting(runtime, "LOCAL_SMALL_MODEL")
Config->>Runtime: runtime.getSetting()
Config-->>Handler: Model name or default
Handler->>Manager: manager.generateText(type, prompt)
Manager->>Manager: Lazy init model if needed
Manager->>Model: Create context & session
Model->>Model: Generate response
Model-->>Manager: Text output
Manager-->>Handler: Generated text
Handler-->>User: Result string
Estimated Code Review Effort🎯 4 (Complex) | ⏱️ ~65 minutes The diff spans multiple distinct areas with heterogeneous changes: a complete build system migration (tsup→Bun), test framework overhaul affecting 6+ files with varying mock patterns, substantial source refactoring (1400+ line reduction in index.ts paired with 900+ lines in manager.ts), and creation of 7+ new modular handler files. While individual changes follow consistent patterns, the breadth and variety of affected domains (build, tests, architecture) require careful cross-cutting review. Poem
🚥 Pre-merge checks | ✅ 3 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
🧪 Generate unit tests (beta)
⚔️ Resolve merge conflicts (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| this.visionInitializingPromise = (async () => { | ||
| try { | ||
| // Initialize vision model directly | ||
| // Use existing initialization code from the file | ||
| // ... | ||
| this.visionInitialized = true; | ||
| logger.info('[LocalAI] Vision model initialized successfully'); | ||
| } catch (error) { | ||
| logger.error( | ||
| `[LocalAI] Failed to initialize vision model: ${error instanceof Error ? error.message : String(error)}` | ||
| ); | ||
| this.visionInitializingPromise = null; | ||
| throw error; | ||
| } | ||
| })(); | ||
| } |
There was a problem hiding this comment.
Empty lazyInitVision implementation
The lazyInitVision method body is a placeholder with only comments (// Use existing initialization code from the file / // ...). It immediately sets this.visionInitialized = true without actually performing any vision model initialization (e.g., no call to this.visionManager.initialize() or equivalent). This means describeImage() will call this.visionManager.processImage() on a VisionManager instance that may never have been properly initialized, likely causing runtime failures when image description is used.
Compare with lazyInitTranscription() (line 809) and lazyInitTTS() (line 859), which have actual initialization logic.
| punishTokensFilter: () => | ||
| this.smallModel ? this.smallModel.tokenize(WORDS_TO_PUNISH.join(' ')) : [], |
There was a problem hiding this comment.
punishTokensFilter always uses smallModel tokenizer
When generateText is called with ModelType.TEXT_LARGE, this line still uses this.smallModel.tokenize(...) instead of the medium model's tokenizer. If only the medium model has been initialized (or if the small model's tokenizer is incompatible with the medium model), the punishment tokens will be incorrect or the call will return an empty array.
| punishTokensFilter: () => | |
| this.smallModel ? this.smallModel.tokenize(WORDS_TO_PUNISH.join(' ')) : [], | |
| punishTokensFilter: () => { | |
| const model = modelType === ModelType.TEXT_LARGE ? this.mediumModel : this.smallModel; | |
| return model ? model.tokenize(WORDS_TO_PUNISH.join(' ')) : []; | |
| }, |
Additional Comments (1)
After extracting the |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
| try { | ||
| const manager = localAIManager.getTokenizerManager(); | ||
| const config = localAIManager.getActiveModelConfig(); | ||
| return await manager.encode(params.prompt, config); |
There was a problem hiding this comment.
Tokenizer and image handlers crash without environment initialization
High Severity
The tokenizer handlers (handleTokenizerEncode, handleTokenizerDecode) and image handler (handleImageDescription) never call localAIManager.initializeEnvironment(). The managers they depend on — tokenizerManager and visionManager — are only created in _postValidateInit(), which is called from initializeEnvironment(). The old init() called initializeEnvironment() eagerly, but the new init() only prints a banner. If any of these handlers is invoked before a text/embedding/object handler initializes the environment, getTokenizerManager() returns undefined and visionManager.processImage() throws a TypeError.
Additional Locations (2)
| `[LocalAI] Error in TRANSCRIPTION handler: ${error instanceof Error ? error.message : String(error)}` | ||
| ); | ||
| throw error; | ||
| } |
There was a problem hiding this comment.
Audio handlers omit runtime from environment initialization
Medium Severity
handleTranscription and handleTextToSpeech accept runtime but never pass it to localAIManager.initializeEnvironment(runtime). Their internal lazy init methods (lazyInitTranscription, lazyInitTTS) call initializeEnvironment() without runtime. Since initializeEnvironment is guarded by a one-shot environmentInitialized flag, if either audio handler triggers initialization first, runtime-specific settings (character-level model overrides) are permanently ignored for all handlers — directly contradicting the PR's "runtime-aware configuration" goal.
Additional Locations (1)
There was a problem hiding this comment.
Pull request overview
This PR modernizes the @elizaos/plugin-local-ai codebase through significant architectural refactoring and tooling updates. The changes extract a 2000+ line monolithic index.ts into a modular structure with dedicated manager, model handlers, configuration utilities, and a startup banner. The build system migrates from tsup to Bun's native builder, and tests migrate from vitest to bun:test to better handle native dependency mocking.
Changes:
- Extracted
LocalAIManagerinto dedicatedmanager.tswith lazy initialization patterns - Created modular
models/directory with separate handlers for each model type (text, embedding, image, audio, tokenizer, object) - Added runtime-aware configuration resolution via
utils/config.tshelpers that checkruntime.getSetting()before falling back to environment variables - Replaced
tsupwith custombuild.tsusing@elizaos/build-utilsfor standardized builds - Migrated tests from vitest to
bun:testwith module-level mocking for native dependencies - Added startup configuration banner displaying all plugin settings
Reviewed changes
Copilot reviewed 23 out of 23 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
tsup.config.ts |
Removed - replaced with custom build.ts script |
src/utils/config.ts |
New configuration helper utilities for runtime-aware settings resolution |
src/models/tokenizer.ts |
New tokenizer encode/decode handlers extracted from index.ts |
src/models/text.ts |
New TEXT_SMALL/TEXT_LARGE handlers extracted from index.ts |
src/models/object.ts |
New OBJECT_SMALL/OBJECT_LARGE handlers with JSON extraction logic |
src/models/index.ts |
Barrel export for all model handlers |
src/models/image.ts |
New IMAGE_DESCRIPTION handler extracted from index.ts |
src/models/embedding.ts |
New TEXT_EMBEDDING handler extracted from index.ts |
src/models/audio.ts |
New TRANSCRIPTION/TEXT_TO_SPEECH handlers extracted from index.ts |
src/manager.ts |
Extracted LocalAIManager singleton with all model initialization logic |
src/index.ts |
Simplified plugin entry point, now just registers handlers and displays banner |
src/constants.ts |
New shared constants file (default models, WORDS_TO_PUNISH) |
src/banner.ts |
New startup configuration display with ANSI art |
package.json |
Updated build scripts, moved tsup to devDependencies |
build.ts |
New custom build script using @elizaos/build-utils |
__tests__/tts.test.ts |
Migrated from vitest to bun:test with module-level mocking |
__tests__/text-transcribe.test.ts |
Migrated from vitest to bun:test with module-level mocking |
__tests__/text-gen.test.ts |
Migrated from vitest to bun:test with module-level mocking |
__tests__/test-utils.ts |
Updated mock utilities for bun:test framework |
__tests__/initialization.test.ts |
Migrated from vitest to bun:test with module-level mocking |
__tests__/image-desc.test.ts |
Migrated from vitest to bun:test with module-level mocking |
README.md |
Added extensive architecture documentation and testing guide |
CHANGELOG.md |
New comprehensive changelog documenting all changes |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| logger.error( | ||
| `[LocalAI] Error in TEXT_EMBEDDING handler: ${error instanceof Error ? error.message : String(error)}` | ||
| ); | ||
| return new Array(384).fill(0); |
There was a problem hiding this comment.
Hardcoded embedding dimension (384) doesn't respect the LOCAL_EMBEDDING_DIMENSIONS configuration. The fallback zero vector should use getEmbeddingDimensions(runtime) to get the configured dimension size instead of the hardcoded value.
| */ | ||
| export function getEmbeddingDimensions(runtime: IAgentRuntime): number { | ||
| const dims = getSetting(runtime, 'LOCAL_EMBEDDING_DIMENSIONS', String(DEFAULT_EMBEDDING_DIMENSIONS)); | ||
| return parseInt(dims!, 10); |
There was a problem hiding this comment.
Non-null assertion operator (!) is used on dims which could be undefined if getSetting returns undefined. While unlikely given the default value, this could cause a runtime error. Consider adding explicit undefined handling or updating the return type of getSetting when a default is provided.
| private _setupModelsDir(): void { | ||
| // Set up models directory consistently, similar to cacheDir | ||
| const modelsDirEnv = this.config?.MODELS_DIR?.trim() || process.env.MODELS_DIR?.trim(); | ||
| if (modelsDirEnv) { | ||
| this.modelsDir = path.resolve(modelsDirEnv); | ||
| logger.info('[LocalAI] Using models directory from MODELS_DIR environment variable:', this.modelsDir); | ||
| } else { | ||
| this.modelsDir = path.join(os.homedir(), '.eliza', 'models'); | ||
| logger.info( | ||
| '[LocalAI] MODELS_DIR environment variable not set, using default models directory:', | ||
| this.modelsDir | ||
| ); | ||
| } | ||
|
|
||
| // Ensure models directory exists | ||
| if (!fs.existsSync(this.modelsDir)) { | ||
| fs.mkdirSync(this.modelsDir, { recursive: true }); | ||
| logger.debug('Ensured models directory exists (created):', this.modelsDir); | ||
| } else { | ||
| logger.debug('Models directory already exists:', this.modelsDir); | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Sets up the cache directory, reading from config or environment variables, | ||
| * and ensures the directory exists. | ||
| */ | ||
| private _setupCacheDir(): void { | ||
| // Set up cache directory | ||
| const cacheDirEnv = this.config?.CACHE_DIR?.trim() || process.env.CACHE_DIR?.trim(); | ||
| if (cacheDirEnv) { | ||
| this.cacheDir = path.resolve(cacheDirEnv); | ||
| logger.info('[LocalAI] Using cache directory from CACHE_DIR environment variable:', this.cacheDir); | ||
| } else { | ||
| const cacheDir = path.join(os.homedir(), '.eliza', 'cache'); | ||
| // Ensure cache directory exists | ||
| if (!fs.existsSync(cacheDir)) { | ||
| fs.mkdirSync(cacheDir, { recursive: true }); | ||
| logger.debug('Ensuring cache directory exists (created):', cacheDir); | ||
| } | ||
| this.cacheDir = cacheDir; | ||
| logger.info( | ||
| '[LocalAI] CACHE_DIR environment variable not set, using default cache directory:', | ||
| this.cacheDir | ||
| ); | ||
| } | ||
| // Ensure cache directory exists if specified via env var but not yet created | ||
| if (!fs.existsSync(this.cacheDir)) { | ||
| fs.mkdirSync(this.cacheDir, { recursive: true }); | ||
| logger.debug('Ensured cache directory exists (created):', this.cacheDir); | ||
| } else { | ||
| logger.debug('Cache directory already exists:', this.cacheDir); | ||
| } | ||
| } |
There was a problem hiding this comment.
The _setupModelsDir() and _setupCacheDir() methods don't use runtime settings. While initializeEnvironment() overrides the config for model filenames (lines 212-218), it doesn't override MODELS_DIR or CACHE_DIR. These methods are called from _postValidateInit() which happens AFTER config override, but they still only check this.config and process.env, never calling getSetting(runtime, 'MODELS_DIR') or getSetting(runtime, 'CACHE_DIR'). This means character-level directory configuration is ignored. Either pass runtime to these methods or store runtime in the class after first initialization.
| repeatPenalty: { | ||
| punishTokensFilter: () => | ||
| this.smallModel ? this.smallModel.tokenize(WORDS_TO_PUNISH.join(' ')) : [], | ||
| penalty: 1.2, | ||
| frequencyPenalty: 0.7, | ||
| presencePenalty: 0.7, | ||
| }, | ||
| }); |
There was a problem hiding this comment.
The punishTokensFilter always uses this.smallModel to tokenize WORDS_TO_PUNISH, even when generating with ModelType.TEXT_LARGE. This means when the large model is active, it will tokenize the punishment words using the small model's vocabulary, which may not match the large model's tokens. This should use the active model: this.mediumModel || this.smallModel or access the model through this.activeModelConfig.
| let text: string; | ||
| if (params === null || params === undefined) { | ||
| logger.debug('[LocalAI] Null or empty text input for embedding, returning zero vector'); | ||
| return new Array(384).fill(0); |
There was a problem hiding this comment.
Hardcoded embedding dimension (384) doesn't respect the LOCAL_EMBEDDING_DIMENSIONS configuration. When returning zero vectors on null/empty input or errors, the code should use getEmbeddingDimensions(runtime) to get the configured dimension size instead of the hardcoded value. This will cause dimension mismatches if users configure a different embedding model with different dimensions.
| // Handle null/undefined/empty text | ||
| if (!text) { | ||
| logger.debug('[LocalAI] Null or empty text input for embedding, returning zero vector'); | ||
| return new Array(384).fill(0); |
There was a problem hiding this comment.
Hardcoded embedding dimension (384) doesn't respect the LOCAL_EMBEDDING_DIMENSIONS configuration. When returning zero vectors for empty text, use getEmbeddingDimensions(runtime) to get the configured dimension size.
| export function getSmallModel(runtime: IAgentRuntime): string { | ||
| return getSetting(runtime, 'LOCAL_SMALL_MODEL', DEFAULT_SMALL_MODEL)!; | ||
| } | ||
|
|
||
| /** | ||
| * Helper function to get the large model filename with fallbacks | ||
| * | ||
| * @param runtime - The runtime context | ||
| * @returns The configured large model filename | ||
| */ | ||
| export function getLargeModel(runtime: IAgentRuntime): string { | ||
| return getSetting(runtime, 'LOCAL_LARGE_MODEL', DEFAULT_LARGE_MODEL)!; | ||
| } | ||
|
|
||
| /** | ||
| * Helper function to get the embedding model filename with fallbacks | ||
| * | ||
| * @param runtime - The runtime context | ||
| * @returns The configured embedding model filename | ||
| */ | ||
| export function getEmbeddingModel(runtime: IAgentRuntime): string { | ||
| return getSetting(runtime, 'LOCAL_EMBEDDING_MODEL', DEFAULT_EMBEDDING_MODEL)!; |
There was a problem hiding this comment.
Non-null assertion operator (!) is used unsafely. While defaultValue is provided, getSetting can still return undefined if both runtime.getSetting() and process.env[key] return undefined/null AND no defaultValue is passed (which can't happen here due to the constant). However, this pattern relies on the function signature allowing string | undefined return type. For clarity and type safety, the return type of these helper functions should be string (non-nullable) since they always provide defaults.
| import fs from 'node:fs'; | ||
| import os from 'node:os'; | ||
| import path from 'node:path'; | ||
| import { Readable } from 'node:stream'; | ||
| import type { | ||
| GenerateTextParams, | ||
| ModelTypeName, | ||
| TextEmbeddingParams, | ||
| ObjectGenerationParams, | ||
| TokenizeTextParams, | ||
| DetokenizeTextParams, | ||
| ImageDescriptionParams, | ||
| TranscriptionParams, | ||
| TextToSpeechParams, | ||
| } from '@elizaos/core'; | ||
| import { type IAgentRuntime, ModelType, type Plugin, logger } from '@elizaos/core'; | ||
| import { | ||
| type Llama, | ||
| LlamaChatSession, | ||
| type LlamaContext, | ||
| type LlamaContextSequence, | ||
| LlamaEmbeddingContext, | ||
| type LlamaModel, | ||
| getLlama, | ||
| } from 'node-llama-cpp'; | ||
| import { validateConfig, type Config } from './environment'; | ||
| import { MODEL_SPECS, type ModelSpec, type EmbeddingModelSpec } from './types'; | ||
| import { DownloadManager } from './utils/downloadManager'; | ||
| import { getPlatformManager } from './utils/platform'; | ||
| import { TokenizerManager } from './utils/tokenizerManager'; | ||
| import { TranscribeManager } from './utils/transcribeManager'; | ||
| import { TTSManager } from './utils/ttsManager'; | ||
| import { VisionManager } from './utils/visionManager'; | ||
| import { basename } from 'path'; |
There was a problem hiding this comment.
Unused imports should be removed to improve code clarity. The following imports on lines 26-59 are not used in the simplified index.ts after the refactor: fs, os, path (used in old code), Readable (from 'node:stream'), various types (GenerateTextParams, TextEmbeddingParams, ObjectGenerationParams, etc.), Llama, LlamaChatSession, LlamaContext, LlamaContextSequence, LlamaEmbeddingContext, LlamaModel, getLlama (from 'node-llama-cpp'), validateConfig, Config, MODEL_SPECS, ModelSpec, EmbeddingModelSpec, DownloadManager, getPlatformManager, TokenizerManager, TranscribeManager, TTSManager, VisionManager, and basename. These are now only used in manager.ts and the individual model handler files.
| text.ts TEXT_SMALL, TEXT_LARGE handlers | ||
| embedding.ts TEXT_EMBEDDING handler | ||
| image.ts IMAGE_DESCRIPTION handler | ||
| audio.ts TEXT_TO_SPEECH, TRANSCRIPTION handlers |
There was a problem hiding this comment.
The architecture diagram is missing object.ts from the models/ directory listing. The file exists (handles OBJECT_SMALL and OBJECT_LARGE model types) but is not documented in the structure overview.
| audio.ts TEXT_TO_SPEECH, TRANSCRIPTION handlers | |
| audio.ts TEXT_TO_SPEECH, TRANSCRIPTION handlers | |
| object.ts OBJECT_SMALL, OBJECT_LARGE handlers |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 14
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
__tests__/image-desc.test.ts (1)
158-175:⚠️ Potential issue | 🟡 MinorTest expectation is intentionally loose due to URL behavior.
The comment on line 173 acknowledges that
picsum.photosredirects "invalid" paths, so the error message check is relaxed to justerror.message.toBeDefined(). This means this test may not actually test the "invalid URL" path — it could succeed silently if picsum returns a valid redirect. If the intent is to test a failed fetch, use a guaranteed-invalid URL (e.g., a non-routable IP or a mocked fetch that rejects).src/index.ts (2)
26-59: 🛠️ Refactor suggestion | 🟠 MajorSignificant number of stale imports left over from the refactor.
After extracting the
LocalAIManagertomanager.tsand delegating model handling to./models/*, many of these imports are no longer used in this file —fs,os,path,Readable, allnode-llama-cppsymbols,validateConfig,MODEL_SPECS,DownloadManager,getPlatformManager,TokenizerManager,TranscribeManager,TTSManager,VisionManager, andbasename. Removing them would improve clarity and reduce unnecessary module loading at startup.
83-90:⚠️ Potential issue | 🔴 CriticalMissing lazy initialization in audio, image, and tokenizer handlers.
The plugin's init() method correctly delegates environment initialization to handlers, but the coverage is incomplete. The text, object, and embedding handlers properly call
initializeEnvironment(runtime)before use. However, the following handlers skip initialization and call manager methods directly:
- Audio handlers (
handleTranscription,handleTextToSpeech): No initialization call beforetranscribeAudio()orgenerateSpeech()- Image handler (
handleImageDescription): No initialization call beforedescribeImage()- Tokenizer handlers (
handleTokenizerEncode,handleTokenizerDecode): No initialization call before accessing config viagetActiveModelConfig()These handlers must call
await localAIManager.initializeEnvironment(runtime)before invoking manager methods to ensure environment variables, model paths, and per-agent runtime context are properly set up.
🤖 Fix all issues with AI agents
In `@__tests__/test-utils.ts`:
- Around line 62-70: The pending-mock-rejections block in
__tests__/test-utils.ts is dead/unnecessary because it inspects past mock
results instead of the current call and can conflate prior test state; remove
that entire conditional that reads downloadModelMock.mock.calls and
mock.results, and rely on the existing calls to downloadModelMock inside
useModel's per-model branches (the branches that call downloadModelMock around
the lines after the check) so that any injected mock rejection will naturally
propagate from the actual invocation of downloadModelMock instead of being
checked retrospectively.
In `@CHANGELOG.md`:
- Line 90: Replace the placeholder date in the changelog header "##
[1.6.6-alpha.5] - 2026-01-XX" with the actual release date (e.g., "2026-01-14")
so the entry reads "## [1.6.6-alpha.5] - YYYY-MM-DD"; ensure the format matches
other entries in CHANGELOG.md.
In `@src/banner.ts`:
- Around line 73-92: Import the default constants from src/constants.ts and use
them instead of hardcoded strings in the settings array: update the file-level
imports and replace each defaultValue in the settings const (the
SettingDisplay[] with entries for LOCAL_SMALL_MODEL, LOCAL_LARGE_MODEL,
LOCAL_EMBEDDING_MODEL, MODELS_DIR, CACHE_DIR, LOCAL_EMBEDDING_DIMENSIONS) to
reference the corresponding constant from constants.ts so the banner always
reflects the single source of truth.
In `@src/manager.ts`:
- Around line 782-804: lazyInitVision currently short-circuits without actually
initializing the vision subsystem; replace the placeholder with real
initialization logic that awaits the VisionManager's setup (e.g., call and await
the appropriate init method on this.visionManager such as initialize(), init(),
or loadModel()/initializeModel() depending on the VisionManager API), ensure any
resources are created before setting this.visionInitialized = true, keep the
existing visionInitializingPromise concurrency guard, and on failure reset
visionInitializingPromise to null and rethrow the error so subsequent calls can
retry; also make describeImage continue to call this.lazyInitVision() before
invoking this.visionManager.processImage(dataUrl).
- Around line 569-573: The current code sends a hardcoded system prompt via
this.chatSession.prompt(systemMessage, { maxTokens: 1, ... }) on every generate
call, which wastes context and latency; remove the per-request system prompt and
instead set the system prompt once when initializing the ChatSession (or store
it as session-level metadata) so generateText does not call
this.chatSession.prompt each time; search for the temporary QUICK TEST FIX
comments and update the initialization path that creates ChatSession (or the
constructor for the ChatSession class) to accept and apply the systemMessage
once rather than invoking prompt before every generation.
- Around line 338-383: Refactor to remove duplicated logic by extracting the
shared embedding initialization steps into a single implementation and have the
other method delegate: consolidate the download + llama instance creation +
model load + createEmbeddingContext flow (calls to this.initializeEnvironment(),
this.downloadModel(ModelType.TEXT_EMBEDDING), getLlama(), this.llama,
this.llama.loadModel({... modelPath: this.embeddingModelPath ...}), and creating
this.embeddingContext) into one method (keep the current initializeEmbedding or
create a private method like initEmbeddingCore) and have lazyInitEmbedding
simply call that core method (or have initializeEmbedding call
lazyInitEmbedding) while preserving existing guards (checks for this.llama and
this.embeddingModel) and the logging/error behavior. Ensure only one code path
handles model directory creation and error logging and remove the duplicated
block from the other method.
- Around line 575-586: The punishTokensFilter currently always uses
this.smallModel.tokenize which misapplies penalties when ModelType.TEXT_LARGE
(uses this.mediumModel) or when smallModel isn't loaded; update
punishTokensFilter to select the tokenizer based on the active model (e.g., use
this.mediumModel.tokenize when modelType === ModelType.TEXT_LARGE, otherwise
this.smallModel.tokenize) or derive the tokenizer from the active chat
session/model (e.g., this.chatSession or session.model) so the filter tokenizes
with the same model used for generation and returns [] only if that chosen model
is unavailable; adjust the repeatPenalty block to reference the correct model
(punishTokensFilter, smallModel, mediumModel, ModelType.TEXT_LARGE,
repeatPenalty) accordingly.
- Around line 575-586: The code currently hardcodes generation options when
calling this.chatSession.prompt (maxTokens: 8192, temperature: 0.7, and
repeatPenalty.frequencyPenalty/presencePenalty = 0.7) and ignores values passed
in via params (GenerateTextParams); update the call in the block that invokes
this.chatSession.prompt so it merges caller-provided params with sensible
defaults: use params.maxTokens and params.temperature if present (fall back to
existing defaults otherwise), and use params.frequencyPenalty and
params.presencePenalty to populate the repeatPenalty object (keeping the
punishTokensFilter and penalty logic referencing this.smallModel and
WORDS_TO_PUNISH); ensure the merged options are passed into chatSession.prompt
so callers can control generation behavior.
- Around line 737-777: lazyInitMediumModel currently calls lazyInitSmallModel to
obtain this.llama, which causes an unnecessary download/load of the small model;
replace that call with a direct llama-only initialization so the small model is
not downloaded when only the medium model is needed. Implement or reuse a method
(e.g., ensureLlamaInitialized or a new private initLlamaOnly) that sets
this.llama without loading any model files, call that instead of
lazyInitSmallModel inside lazyInitMediumModel, and ensure
downloadModel(ModelType.TEXT_LARGE) runs before loading the medium model using
this.mediumModelPath; keep error handling/logging as-is and clear
this.mediumModelInitializingPromise on failure.
In `@src/models/audio.ts`:
- Around line 18-51: handleTranscription (and similarly handleTextToSpeech)
never calls initializeEnvironment(runtime) before using localAIManager, which
can leave LocalAI uninitialized; add a call to initializeEnvironment(runtime) at
the top of handleTranscription (and handleTextToSpeech) before any
fetch/transcribe/generate calls so the runtime is set up, then proceed to fetch
the audio/params and call localAIManager.transcribeAudio /
localAIManager.generateSpeech as before; ensure to import/resolve
initializeEnvironment if needed and keep error handling unchanged.
In `@src/models/image.ts`:
- Around line 18-41: The handler handleImageDescription is missing the required
environment initialization; add await
localAIManager.initializeEnvironment(runtime) at the start of
handleImageDescription (before any ops like fetching) so the runtime is used and
paths/configs are set, and either update lazyInitVision to call
initializeEnvironment(runtime) internally or keep the explicit
initializeEnvironment call in handleImageDescription to ensure proper
initialization before calling localAIManager.describeImage(buffer, mimeType).
In `@src/models/object.ts`:
- Around line 60-68: The current regex-based fixes applied to extractedJsonText
when computing fixedJson (the two problematic replaces:
.replace(/"([^"]*?)[^a-zA-Z0-9\s\.,;:\-_\(\)"'\[\]{}]([^"]*?)"/g, ...) and
.replace(/(\s*)(\w+)(\s*):/g, ...)) can remove valid punctuation inside string
values and corrupt quoted text; replace this approach by using a robust JSON
repair library (e.g., jsonrepair) to parse and repair the broken JSON or, if you
must keep regexes, restrict them to operate only on the JSON structure (outside
string literals) — i.e., stop touching characters inside quoted strings, and
only apply the property-name and trailing-comma fixes to non-string contexts;
update the code that produces fixedJson (and references to extractedJsonText) to
use the chosen library or to include string-aware parsing so string contents are
not altered.
In `@src/models/tokenizer.ts`:
- Around line 18-31: Both tokenizer handlers (handleTokenizerEncode and
handleTokenizerDecode) omit the required environment initialization so the
tokenizerManager created in _postValidateInit() may be uninitialized; add an
awaited call to await localAIManager.initializeEnvironment(runtime) at the start
of each handler (before calling localAIManager.getTokenizerManager()) and then
use the manager as currently done; also ensure the runtime parameter is used
only for that call so it is no longer unused.
In `@src/utils/config.ts`:
- Around line 101-104: The getEmbeddingDimensions function currently uses
parseInt on the value returned by getSetting and can return NaN for non-numeric
input; update getEmbeddingDimensions to validate the parsed value (using
Number.isFinite or Number.isNaN) and fall back to DEFAULT_EMBEDDING_DIMENSIONS
when parsing fails or yields a non-positive integer. Specifically, parse the
dims from getSetting(runtime, 'LOCAL_EMBEDDING_DIMENSIONS',
String(DEFAULT_EMBEDDING_DIMENSIONS)), check the result from parseInt(dims, 10)
is a valid finite integer (and optionally > 0), and return the default constant
DEFAULT_EMBEDDING_DIMENSIONS when validation fails so downstream code never
receives NaN.
🧹 Nitpick comments (14)
src/models/text.ts (1)
14-48: Consider a shared helper to reduce duplication.
handleTextSmallandhandleTextLargeare identical except for theModelTypeand log label. A small factory would reduce this to one place to maintain, though with only two handlers the current form is perfectly readable.♻️ Optional: extract shared logic
+function handleText(modelType: ModelType, label: string) { + return async (runtime: IAgentRuntime, params: GenerateTextParams): Promise<string> => { + try { + await localAIManager.initializeEnvironment(runtime); + return await localAIManager.generateText(params, modelType); + } catch (error) { + logger.error( + `[LocalAI] Error in ${label} handler: ${error instanceof Error ? error.message : String(error)}` + ); + throw error; + } + }; +} + +export const handleTextSmall = handleText(ModelType.TEXT_SMALL, 'TEXT_SMALL'); +export const handleTextLarge = handleText(ModelType.TEXT_LARGE, 'TEXT_LARGE');README.md (1)
106-121: Add language identifiers to fenced code blocks.Lines 106 and 134 use fenced code blocks without a language specifier, flagged by markdownlint (MD040). Use
textorplaintextfor these blocks.Fix for line 106
-``` +```text src/Fix for line 134
-``` +```text +----------------------------------------------------------------------+src/utils/config.ts (1)
24-35:getSettingreturns empty strings from runtime as valid values.If
runtime.getSetting(key)returns"", the check on line 31 passes ("" !== undefined && "" !== null), andString("")is returned. Downstream callers likegetModelsDirtreat any truthy return as a user-provided value, so an empty string would be resolved viapath.resolve("")(i.e., cwd). This is likely benign but worth being aware of — ifruntime.getSettingcan return empty strings for "unset" keys, you may want to add an empty-string check.src/models/embedding.ts (1)
27-48: Consider extracting the magic number384into a named constant.The embedding dimension
384appears in three places (lines 27, 37, 48). If the BGE model dimension ever changes, all three would need updating.♻️ Suggested refactor
+const EMBEDDING_DIMENSION = 384; + export async function handleTextEmbedding( runtime: IAgentRuntime, params: TextEmbeddingParams | string | null ): Promise<number[]> { try { let text: string; if (params === null || params === undefined) { logger.debug('[LocalAI] Null or empty text input for embedding, returning zero vector'); - return new Array(384).fill(0); + return new Array(EMBEDDING_DIMENSION).fill(0); } else if (typeof params === 'string') { text = params; } else { text = params.text; } if (!text) { logger.debug('[LocalAI] Null or empty text input for embedding, returning zero vector'); - return new Array(384).fill(0); + return new Array(EMBEDDING_DIMENSION).fill(0); } await localAIManager.initializeEnvironment(runtime); return await localAIManager.generateEmbedding(text); } catch (error) { logger.error( `[LocalAI] Error in TEXT_EMBEDDING handler: ${error instanceof Error ? error.message : String(error)}` ); - return new Array(384).fill(0); + return new Array(EMBEDDING_DIMENSION).fill(0); } }src/models/object.ts (1)
161-191: Double error logging on failures.When
generateObjectByModelTypethrows (lines 149–153), the wrappinghandleObjectSmall/handleObjectLargecatch blocks (lines 167–171, 185–189) log the same error again before re-throwing. This produces duplicate[LocalAI]error entries for a single failure.Option: Remove redundant catch in thin wrappers
export async function handleObjectSmall( runtime: IAgentRuntime, params: ObjectGenerationParams ): Promise<Record<string, unknown>> { - try { - return await generateObjectByModelType(runtime, ModelType.TEXT_SMALL, params); - } catch (error) { - logger.error( - `[LocalAI] Error in OBJECT_SMALL handler: ${error instanceof Error ? error.message : String(error)}` - ); - throw error; - } + return await generateObjectByModelType(runtime, ModelType.TEXT_SMALL, params); }__tests__/image-desc.test.ts (2)
118-156: Tests make real HTTP requests, which may cause flakiness in CI.The
mockRuntime.useModel(ModelType.IMAGE_DESCRIPTION, ...)path intest-utils.ts(lines 153–159) performs a livefetch(imageUrl)againsthttps://picsum.photos. This makes these tests dependent on network connectivity and external service availability, leading to non-deterministic failures.Consider mocking
fetch(viamock.moduleorglobalThis.fetch) so unit tests remain hermetic.
200-223: Mock restoration is fragile ifgetMockImplementation()returnsundefined.If
getMockImplementation()returnsundefined(e.g., if Bun treats the initial function passed tomock()differently frommockImplementation()), thefinallyblock silently skips restoration. SincemockImplementationOnceauto-falls-back after one call, this is low-risk in practice, but for robustness consider unconditional restoration:Proposed fix
} finally { // Restore the original implementation - if (originalImpl) { - mockFromPretrained.mockImplementation(originalImpl); - } + mockFromPretrained.mockImplementation(async () => { + logger.info('Creating mock Florence2ForConditionalGeneration model'); + return { + generate: async () => new Int32Array([1, 2, 3, 4, 5]), + _merge_input_ids_with_image_features: mock(), + _prepare_inputs_embeds: mock(), + forward: mock(), + main_input_name: 'pixel_values', + }; + }); }__tests__/test-utils.ts (3)
149-169: Mock runtime makes a realfetchcall for IMAGE_DESCRIPTION.Line 153 performs a live HTTP fetch against the provided
imageUrl. This turns every IMAGE_DESCRIPTION unit test into an integration test with external network dependency. Consider stubbingfetchor returning a canned response directly to keep unit tests deterministic and fast.
18-23: Hardcoded monorepo path assumption may break in different environments.
WORKSPACE_ROOTassumes__dirnameis exactly 3 levels below the monorepo root, andAGENT_DIRassumespackages/project-starterexists. If the repo layout changes or tests run in isolation (e.g., a standalone checkout of this plugin), this will fail with a confusing error. Consider deriving these paths from a package root marker (e.g., walking up to find apackage.jsonwith a knownname) or making them configurable via environment variables.
33-323: The 290-line inline mock runtime is difficult to maintain.
createMockRuntimeis a single function returning a ~290-line object literal. TheuseModelmethod alone is ~180 lines with deeply nested branching. Consider extracting theuseModelhandler map into a separateconst modelHandlers: Record<ModelTypeName, ...>or splitting per-model-type helpers into their own functions to improve readability and make individual handlers easier to test/update.src/manager.ts (1)
29-29: Duplicatepathimport.
basenameis imported from'path'on line 29, butpathis already imported from'node:path'on line 8. Usepath.basenameinstead to avoid the redundant import.Proposed fix
-import { basename } from 'path';Then replace
basename(...)withpath.basename(...)on lines 228–230.__tests__/text-transcribe.test.ts (1)
196-201:error.codeis set to a number via double-cast, which doesn't matchExecExceptiontyping.
ExecException.codeisstring | undefinedin Node.js typings, but line 198 assigns1 as unknown as string. While functionally harmless here (the test asserts onerror.message), this masks a type mismatch. Use a string code instead:Proposed fix
- error.code = 1 as unknown as string; + error.code = 'ERR_CONVERSION';__tests__/tts.test.ts (1)
99-149: Same mock-restoration fragility as inimage-desc.test.ts.Both the "model initialization failure" (line 103) and "audio generation failure" (line 129) tests rely on
getMockImplementation()returning a truthy value. If it returnsundefined, thefinallyblock silently skips restoration. SincemockImplementationOnceauto-falls-back after consumption this is low-risk, but the pattern is repeated across test files and could be made more robust by usingmockRestore()or unconditionally re-assigning the known good implementation.src/index.ts (1)
101-112: The double cast masks a type mismatch withPlugin['models'].All handlers follow a
(runtime: IAgentRuntime, params: T) => Promise<U>signature, but each has different param and return types (e.g.,handleTextSmallreturnsPromise<string>, whilehandleObjectSmallreturnsPromise<Record<string, unknown>>). Rather than masking this with a double cast, align the handler type signatures with whatPlugin['models']expects, or implement a typed adapter that mapsModelTypekeys to their corresponding handler signatures. This prevents signature drift when thePlugininterface evolves.
| // Check if there are any pending mock rejections | ||
| const mockCalls = downloadModelMock.mock.calls; | ||
| if ( | ||
| mockCalls.length > 0 && | ||
| downloadModelMock.mock.results[mockCalls.length - 1].type === 'throw' | ||
| ) { | ||
| // Rethrow the error from the mock | ||
| throw downloadModelMock.mock.results[mockCalls.length - 1].value; | ||
| } |
There was a problem hiding this comment.
The "pending mock rejections" check inspects past results, not future behavior — likely dead code.
This block reads the last completed call's result from downloadModelMock.mock.results. Since downloadModelMock defaults to async () => undefined and is never called before this check within the same useModel invocation, mockCalls.length will be 0 on the first call, making this block unreachable. On subsequent calls it checks whether a previous invocation threw — which conflates prior test state with the current one.
If the goal is to allow tests to inject download failures, consider having the individual model-type branches call downloadModelMock first (they already do on lines 74, 81, etc.) and let the mock's own rejection propagate naturally, removing this block entirely.
🤖 Prompt for AI Agents
In `@__tests__/test-utils.ts` around lines 62 - 70, The pending-mock-rejections
block in __tests__/test-utils.ts is dead/unnecessary because it inspects past
mock results instead of the current call and can conflate prior test state;
remove that entire conditional that reads downloadModelMock.mock.calls and
mock.results, and rely on the existing calls to downloadModelMock inside
useModel's per-model branches (the branches that call downloadModelMock around
the lines after the check) so that any injected mock rejection will naturally
propagate from the actual invocation of downloadModelMock instead of being
checked retrospectively.
|
|
||
| This ensures character-level configuration overrides environment variables, which override plugin defaults. | ||
|
|
||
| ## [1.6.6-alpha.5] - 2026-01-XX |
There was a problem hiding this comment.
Placeholder date 2026-01-XX should be filled in.
The version entry has an incomplete date. Replace with the actual release date before merging.
🤖 Prompt for AI Agents
In `@CHANGELOG.md` at line 90, Replace the placeholder date in the changelog
header "## [1.6.6-alpha.5] - 2026-01-XX" with the actual release date (e.g.,
"2026-01-14") so the entry reads "## [1.6.6-alpha.5] - YYYY-MM-DD"; ensure the
format matches other entries in CHANGELOG.md.
| const settings: SettingDisplay[] = [ | ||
| { | ||
| name: 'LOCAL_SMALL_MODEL', | ||
| value: smallModel, | ||
| defaultValue: 'DeepHermes-3-Llama-3-3B-Preview-q4.gguf', | ||
| }, | ||
| { | ||
| name: 'LOCAL_LARGE_MODEL', | ||
| value: largeModel, | ||
| defaultValue: 'DeepHermes-3-Llama-3-8B-q4.gguf', | ||
| }, | ||
| { | ||
| name: 'LOCAL_EMBEDDING_MODEL', | ||
| value: embeddingModel, | ||
| defaultValue: 'bge-small-en-v1.5.Q4_K_M.gguf', | ||
| }, | ||
| { name: 'MODELS_DIR', value: modelsDir, defaultValue: '~/.eliza/models' }, | ||
| { name: 'CACHE_DIR', value: cacheDir, defaultValue: '~/.eliza/cache' }, | ||
| { name: 'LOCAL_EMBEDDING_DIMENSIONS', value: embeddingDims, defaultValue: '384' }, | ||
| ]; |
There was a problem hiding this comment.
🛠️ Refactor suggestion | 🟠 Major
Default values are hardcoded instead of imported from constants.ts.
The defaultValue strings on lines 77, 82, 87, 89, 90, 91 duplicate the constants defined in src/constants.ts. If defaults change in one place, the banner will display stale values.
♻️ Import and use constants
import { getSetting } from './utils/config';
+import {
+ DEFAULT_SMALL_MODEL,
+ DEFAULT_LARGE_MODEL,
+ DEFAULT_EMBEDDING_MODEL,
+ DEFAULT_EMBEDDING_DIMENSIONS,
+} from './constants';Then reference them in the settings array:
- defaultValue: 'DeepHermes-3-Llama-3-3B-Preview-q4.gguf',
+ defaultValue: DEFAULT_SMALL_MODEL,- defaultValue: 'DeepHermes-3-Llama-3-8B-q4.gguf',
+ defaultValue: DEFAULT_LARGE_MODEL,- defaultValue: 'bge-small-en-v1.5.Q4_K_M.gguf',
+ defaultValue: DEFAULT_EMBEDDING_MODEL,- { name: 'LOCAL_EMBEDDING_DIMENSIONS', value: embeddingDims, defaultValue: '384' },
+ { name: 'LOCAL_EMBEDDING_DIMENSIONS', value: embeddingDims, defaultValue: String(DEFAULT_EMBEDDING_DIMENSIONS) },🤖 Prompt for AI Agents
In `@src/banner.ts` around lines 73 - 92, Import the default constants from
src/constants.ts and use them instead of hardcoded strings in the settings
array: update the file-level imports and replace each defaultValue in the
settings const (the SettingDisplay[] with entries for LOCAL_SMALL_MODEL,
LOCAL_LARGE_MODEL, LOCAL_EMBEDDING_MODEL, MODELS_DIR, CACHE_DIR,
LOCAL_EMBEDDING_DIMENSIONS) to reference the corresponding constant from
constants.ts so the banner always reflects the single source of truth.
| public async initializeEmbedding(): Promise<void> { | ||
| try { | ||
| await this.initializeEnvironment(); // Ensure environment/paths are ready | ||
| logger.info('[LocalAI] Initializing embedding model...'); | ||
| logger.info('[LocalAI] Models directory:', this.modelsDir); | ||
|
|
||
| // Ensure models directory exists | ||
| if (!fs.existsSync(this.modelsDir)) { | ||
| logger.warn('[LocalAI] Models directory does not exist, creating it:', this.modelsDir); | ||
| fs.mkdirSync(this.modelsDir, { recursive: true }); | ||
| } | ||
|
|
||
| // Download the embedding model using the common downloadModel function | ||
| // This will now use the correct embeddingModelPath | ||
| await this.downloadModel(ModelType.TEXT_EMBEDDING); | ||
|
|
||
| // Initialize the llama instance if not already done | ||
| if (!this.llama) { | ||
| this.llama = await getLlama(); | ||
| } | ||
|
|
||
| // Load the embedding model | ||
| if (!this.embeddingModel) { | ||
| logger.info('[LocalAI] Loading embedding model:', this.embeddingModelPath); // Use the correct path | ||
|
|
||
| this.embeddingModel = await this.llama.loadModel({ | ||
| modelPath: this.embeddingModelPath, // Use the correct path | ||
| gpuLayers: 0, // Embedding models are typically small enough to run on CPU | ||
| vocabOnly: false, | ||
| }); | ||
|
|
||
| // Create context for embeddings | ||
| this.embeddingContext = await this.embeddingModel.createEmbeddingContext({ | ||
| contextSize: this.embeddingModelConfig.contextSize, | ||
| batchSize: 512, | ||
| }); | ||
|
|
||
| logger.success('[LocalAI] Embedding model initialized successfully'); | ||
| } | ||
| } catch (error) { | ||
| logger.error( | ||
| `[LocalAI] Embedding initialization failed: ${error instanceof Error ? error.message : String(error)}. modelsDir=${this.modelsDir}, embeddingModelPath=${this.embeddingModelPath}${error instanceof Error && error.stack ? `\n${error.stack}` : ''}` | ||
| ); | ||
| throw error; | ||
| } | ||
| } |
There was a problem hiding this comment.
🛠️ Refactor suggestion | 🟠 Major
initializeEmbedding() and lazyInitEmbedding() contain nearly identical logic.
Both methods download the embedding model, get a Llama instance, load the model, and create an embedding context with the same parameters. This duplication will lead to drift. Consider having initializeEmbedding delegate to lazyInitEmbedding, or vice versa, so the core logic lives in one place.
Also applies to: 447-490
🤖 Prompt for AI Agents
In `@src/manager.ts` around lines 338 - 383, Refactor to remove duplicated logic
by extracting the shared embedding initialization steps into a single
implementation and have the other method delegate: consolidate the download +
llama instance creation + model load + createEmbeddingContext flow (calls to
this.initializeEnvironment(), this.downloadModel(ModelType.TEXT_EMBEDDING),
getLlama(), this.llama, this.llama.loadModel({... modelPath:
this.embeddingModelPath ...}), and creating this.embeddingContext) into one
method (keep the current initializeEmbedding or create a private method like
initEmbeddingCore) and have lazyInitEmbedding simply call that core method (or
have initializeEmbedding call lazyInitEmbedding) while preserving existing
guards (checks for this.llama and this.embeddingModel) and the logging/error
behavior. Ensure only one code path handles model directory creation and error
logging and remove the duplicated block from the other method.
| const systemMessage = 'You are a helpful AI assistant. Respond to the current request only.'; | ||
| await this.chatSession.prompt(systemMessage, { | ||
| maxTokens: 1, // Minimal tokens for system message | ||
| temperature: 0.0, | ||
| }); |
There was a problem hiding this comment.
System prompt consumes a chat turn with hardcoded maxTokens: 1 before every generation.
Every call to generateText sends a system message with maxTokens: 1 (line 571), which uses context window space and adds latency for every request. The // QUICK TEST FIX comments (lines 547, 550, 568) suggest these are temporary workarounds that should be revisited before merging.
🤖 Prompt for AI Agents
In `@src/manager.ts` around lines 569 - 573, The current code sends a hardcoded
system prompt via this.chatSession.prompt(systemMessage, { maxTokens: 1, ... })
on every generate call, which wastes context and latency; remove the per-request
system prompt and instead set the system prompt once when initializing the
ChatSession (or store it as session-level metadata) so generateText does not
call this.chatSession.prompt each time; search for the temporary QUICK TEST FIX
comments and update the initialization path that creates ChatSession (or the
constructor for the ChatSession class) to accept and apply the systemMessage
once rather than invoking prompt before every generation.
| export async function handleTranscription( | ||
| runtime: IAgentRuntime, | ||
| params: TranscriptionParams | Buffer | string | ||
| ): Promise<string> { | ||
| try { | ||
| let audioBuffer: Buffer; | ||
|
|
||
| if (Buffer.isBuffer(params)) { | ||
| audioBuffer = params; | ||
| } else if (typeof params === 'string') { | ||
| // If it's a URL, fetch it | ||
| const response = await fetch(params); | ||
| if (!response.ok) { | ||
| throw new Error(`Failed to fetch audio: ${response.statusText}`); | ||
| } | ||
| audioBuffer = Buffer.from(await response.arrayBuffer()); | ||
| } else { | ||
| // It's TranscriptionParams with audioUrl | ||
| const response = await fetch(params.audioUrl); | ||
| if (!response.ok) { | ||
| throw new Error(`Failed to fetch audio: ${response.statusText}`); | ||
| } | ||
| audioBuffer = Buffer.from(await response.arrayBuffer()); | ||
| } | ||
|
|
||
| logger.info(`[LocalAI] Processing audio transcription: bufferSize=${audioBuffer.length}`); | ||
|
|
||
| return await localAIManager.transcribeAudio(audioBuffer); | ||
| } catch (error) { | ||
| logger.error( | ||
| `[LocalAI] Error in TRANSCRIPTION handler: ${error instanceof Error ? error.message : String(error)}` | ||
| ); | ||
| throw error; | ||
| } |
There was a problem hiding this comment.
Missing initializeEnvironment(runtime) — same pattern as image.ts and tokenizer.ts.
Both handleTranscription and handleTextToSpeech declare runtime but never use it, and skip the initializeEnvironment call. If transcribeAudio or generateSpeech depend on initialization state, these will fail. See the comments on image.ts and tokenizer.ts for the same concern.
Proposed fix for handleTranscription
export async function handleTranscription(
runtime: IAgentRuntime,
params: TranscriptionParams | Buffer | string
): Promise<string> {
try {
let audioBuffer: Buffer;
// ... input normalization ...
logger.info(`[LocalAI] Processing audio transcription: bufferSize=${audioBuffer.length}`);
+ await localAIManager.initializeEnvironment(runtime);
return await localAIManager.transcribeAudio(audioBuffer);Proposed fix for handleTextToSpeech
export async function handleTextToSpeech(
runtime: IAgentRuntime,
params: TextToSpeechParams | string
): Promise<Buffer> {
try {
const text = typeof params === 'string' ? params : params.text;
+ await localAIManager.initializeEnvironment(runtime);
const speechStream = await localAIManager.generateSpeech(text);📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| export async function handleTranscription( | |
| runtime: IAgentRuntime, | |
| params: TranscriptionParams | Buffer | string | |
| ): Promise<string> { | |
| try { | |
| let audioBuffer: Buffer; | |
| if (Buffer.isBuffer(params)) { | |
| audioBuffer = params; | |
| } else if (typeof params === 'string') { | |
| // If it's a URL, fetch it | |
| const response = await fetch(params); | |
| if (!response.ok) { | |
| throw new Error(`Failed to fetch audio: ${response.statusText}`); | |
| } | |
| audioBuffer = Buffer.from(await response.arrayBuffer()); | |
| } else { | |
| // It's TranscriptionParams with audioUrl | |
| const response = await fetch(params.audioUrl); | |
| if (!response.ok) { | |
| throw new Error(`Failed to fetch audio: ${response.statusText}`); | |
| } | |
| audioBuffer = Buffer.from(await response.arrayBuffer()); | |
| } | |
| logger.info(`[LocalAI] Processing audio transcription: bufferSize=${audioBuffer.length}`); | |
| return await localAIManager.transcribeAudio(audioBuffer); | |
| } catch (error) { | |
| logger.error( | |
| `[LocalAI] Error in TRANSCRIPTION handler: ${error instanceof Error ? error.message : String(error)}` | |
| ); | |
| throw error; | |
| } | |
| export async function handleTranscription( | |
| runtime: IAgentRuntime, | |
| params: TranscriptionParams | Buffer | string | |
| ): Promise<string> { | |
| try { | |
| let audioBuffer: Buffer; | |
| if (Buffer.isBuffer(params)) { | |
| audioBuffer = params; | |
| } else if (typeof params === 'string') { | |
| // If it's a URL, fetch it | |
| const response = await fetch(params); | |
| if (!response.ok) { | |
| throw new Error(`Failed to fetch audio: ${response.statusText}`); | |
| } | |
| audioBuffer = Buffer.from(await response.arrayBuffer()); | |
| } else { | |
| // It's TranscriptionParams with audioUrl | |
| const response = await fetch(params.audioUrl); | |
| if (!response.ok) { | |
| throw new Error(`Failed to fetch audio: ${response.statusText}`); | |
| } | |
| audioBuffer = Buffer.from(await response.arrayBuffer()); | |
| } | |
| logger.info(`[LocalAI] Processing audio transcription: bufferSize=${audioBuffer.length}`); | |
| await localAIManager.initializeEnvironment(runtime); | |
| return await localAIManager.transcribeAudio(audioBuffer); | |
| } catch (error) { | |
| logger.error( | |
| `[LocalAI] Error in TRANSCRIPTION handler: ${error instanceof Error ? error.message : String(error)}` | |
| ); | |
| throw error; | |
| } |
🤖 Prompt for AI Agents
In `@src/models/audio.ts` around lines 18 - 51, handleTranscription (and similarly
handleTextToSpeech) never calls initializeEnvironment(runtime) before using
localAIManager, which can leave LocalAI uninitialized; add a call to
initializeEnvironment(runtime) at the top of handleTranscription (and
handleTextToSpeech) before any fetch/transcribe/generate calls so the runtime is
set up, then proceed to fetch the audio/params and call
localAIManager.transcribeAudio / localAIManager.generateSpeech as before; ensure
to import/resolve initializeEnvironment if needed and keep error handling
unchanged.
| export async function handleImageDescription( | ||
| runtime: IAgentRuntime, | ||
| params: ImageDescriptionParams | string | ||
| ): Promise<{ title: string; description: string }> { | ||
| try { | ||
| const imageUrl = typeof params === 'string' ? params : params.imageUrl; | ||
| logger.info(`[LocalAI] Processing image from URL: ${imageUrl}`); | ||
|
|
||
| // Fetch the image from URL | ||
| const response = await fetch(imageUrl); | ||
| if (!response.ok) { | ||
| throw new Error(`Failed to fetch image: ${response.statusText}`); | ||
| } | ||
|
|
||
| const buffer = Buffer.from(await response.arrayBuffer()); | ||
| const mimeType = response.headers.get('content-type') || 'image/jpeg'; | ||
|
|
||
| return await localAIManager.describeImage(buffer, mimeType); | ||
| } catch (error) { | ||
| logger.error( | ||
| `[LocalAI] Error in IMAGE_DESCRIPTION handler: ${error instanceof Error ? error.message : String(error)}` | ||
| ); | ||
| throw error; | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, let's find the embedding.ts and object.ts files
find . -name "embedding.ts" -o -name "object.ts" | head -20Repository: elizaos-plugins/plugin-local-ai
Length of output: 123
🏁 Script executed:
# Search for initializeEnvironment calls in the codebase to see which handlers use it
rg "initializeEnvironment" -A 2 -B 2Repository: elizaos-plugins/plugin-local-ai
Length of output: 6549
🏁 Script executed:
# Look at the image.ts file to understand its context
cat -n src/models/image.tsRepository: elizaos-plugins/plugin-local-ai
Length of output: 1852
🏁 Script executed:
# Find the localAIManager implementation to see describeImage
find . -name "*.ts" -o -name "*.js" | xargs grep -l "describeImage" | head -10Repository: elizaos-plugins/plugin-local-ai
Length of output: 113
🏁 Script executed:
# Look at the describeImage implementation in manager.ts
rg "describeImage" -A 10 -B 2 src/manager.tsRepository: elizaos-plugins/plugin-local-ai
Length of output: 497
🏁 Script executed:
# Check if describeImage accesses any state that would be initialized by initializeEnvironment
rg "describeImage\|environmentInitialized\|this\.ctx\|this\.models" src/manager.ts | head -40Repository: elizaos-plugins/plugin-local-ai
Length of output: 57
🏁 Script executed:
# Check the lazyInitVision implementation to see if it calls initializeEnvironment
rg "lazyInitVision" -A 15 src/manager.tsRepository: elizaos-plugins/plugin-local-ai
Length of output: 1247
🏁 Script executed:
# Compare the pattern - check all handlers to see which ones call initializeEnvironment
rg "export async function.*RuntimeTextGeneratorParams|export async function.*ImageDescriptionParams|export async function.*EmbeddingParams" -A 5 src/models/Repository: elizaos-plugins/plugin-local-ai
Length of output: 57
🏁 Script executed:
# Get the complete lazyInitVision implementation
rg "private async lazyInitVision" -A 30 src/manager.tsRepository: elizaos-plugins/plugin-local-ai
Length of output: 1054
🏁 Script executed:
# Check other lazy init methods to understand the pattern
rg "private async lazyInit" -A 8 src/manager.ts | head -50Repository: elizaos-plugins/plugin-local-ai
Length of output: 1948
Missing initializeEnvironment(runtime) call — required by all other handlers.
All other model handlers (handleTextSmall, handleTextLarge, handleEmbedding, generateObjectByModelType) explicitly call await localAIManager.initializeEnvironment(runtime) before delegating to the manager. This handler skips that step. Additionally, lazyInitVision does not call initializeEnvironment() internally like other lazy init methods (lazyInitEmbedding, lazyInitSmallModel, lazyInitMediumModel, lazyInitTranscription) do. This prevents proper initialization of environment paths and configuration resolution. The unused runtime parameter reinforces this oversight.
Add the initialization call:
Fix
export async function handleImageDescription(
runtime: IAgentRuntime,
params: ImageDescriptionParams | string
): Promise<{ title: string; description: string }> {
try {
const imageUrl = typeof params === 'string' ? params : params.imageUrl;
logger.info(`[LocalAI] Processing image from URL: ${imageUrl}`);
+ await localAIManager.initializeEnvironment(runtime);
+
// Fetch the image from URL
const response = await fetch(imageUrl);📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| export async function handleImageDescription( | |
| runtime: IAgentRuntime, | |
| params: ImageDescriptionParams | string | |
| ): Promise<{ title: string; description: string }> { | |
| try { | |
| const imageUrl = typeof params === 'string' ? params : params.imageUrl; | |
| logger.info(`[LocalAI] Processing image from URL: ${imageUrl}`); | |
| // Fetch the image from URL | |
| const response = await fetch(imageUrl); | |
| if (!response.ok) { | |
| throw new Error(`Failed to fetch image: ${response.statusText}`); | |
| } | |
| const buffer = Buffer.from(await response.arrayBuffer()); | |
| const mimeType = response.headers.get('content-type') || 'image/jpeg'; | |
| return await localAIManager.describeImage(buffer, mimeType); | |
| } catch (error) { | |
| logger.error( | |
| `[LocalAI] Error in IMAGE_DESCRIPTION handler: ${error instanceof Error ? error.message : String(error)}` | |
| ); | |
| throw error; | |
| } | |
| export async function handleImageDescription( | |
| runtime: IAgentRuntime, | |
| params: ImageDescriptionParams | string | |
| ): Promise<{ title: string; description: string }> { | |
| try { | |
| const imageUrl = typeof params === 'string' ? params : params.imageUrl; | |
| logger.info(`[LocalAI] Processing image from URL: ${imageUrl}`); | |
| await localAIManager.initializeEnvironment(runtime); | |
| // Fetch the image from URL | |
| const response = await fetch(imageUrl); | |
| if (!response.ok) { | |
| throw new Error(`Failed to fetch image: ${response.statusText}`); | |
| } | |
| const buffer = Buffer.from(await response.arrayBuffer()); | |
| const mimeType = response.headers.get('content-type') || 'image/jpeg'; | |
| return await localAIManager.describeImage(buffer, mimeType); | |
| } catch (error) { | |
| logger.error( | |
| `[LocalAI] Error in IMAGE_DESCRIPTION handler: ${error instanceof Error ? error.message : String(error)}` | |
| ); | |
| throw error; | |
| } | |
| } |
🤖 Prompt for AI Agents
In `@src/models/image.ts` around lines 18 - 41, The handler handleImageDescription
is missing the required environment initialization; add await
localAIManager.initializeEnvironment(runtime) at the start of
handleImageDescription (before any ops like fetching) so the runtime is used and
paths/configs are set, and either update lazyInitVision to call
initializeEnvironment(runtime) internally or keep the explicit
initializeEnvironment call in handleImageDescription to ensure proper
initialization before calling localAIManager.describeImage(buffer, mimeType).
| // Replace any unescaped newlines in string values | ||
| const fixedJson = extractedJsonText | ||
| .replace(/:\s*"([^"]*)(?:\n)([^"]*)"/g, ': "$1\\n$2"') | ||
| // Remove any non-JSON text that might have gotten mixed into string values | ||
| .replace(/"([^"]*?)[^a-zA-Z0-9\s\.,;:\-_\(\)"'\[\]{}]([^"]*?)"/g, '"$1$2"') | ||
| // Fix missing quotes around property names | ||
| .replace(/(\s*)(\w+)(\s*):/g, '$1"$2"$3:') | ||
| // Fix trailing commas in arrays and objects | ||
| .replace(/,(\s*[\]}])/g, '$1'); |
There was a problem hiding this comment.
JSON repair regexes can corrupt valid string content.
The repair on line 64 silently strips characters like @, #, $, /, =, +, ?, etc. from within string values. The repair on line 66 ((\s*)(\w+)(\s*):) will match words followed by colons inside string values (e.g., "note: important" → ""note": important"), further corrupting the output.
These repairs run on already-broken JSON, so they're best-effort, but they can make the situation worse. Consider using a more robust JSON repair library (e.g., jsonrepair) or at minimum narrowing the regex scope.
npm jsonrepair library
🤖 Prompt for AI Agents
In `@src/models/object.ts` around lines 60 - 68, The current regex-based fixes
applied to extractedJsonText when computing fixedJson (the two problematic
replaces: .replace(/"([^"]*?)[^a-zA-Z0-9\s\.,;:\-_\(\)"'\[\]{}]([^"]*?)"/g, ...)
and .replace(/(\s*)(\w+)(\s*):/g, ...)) can remove valid punctuation inside
string values and corrupt quoted text; replace this approach by using a robust
JSON repair library (e.g., jsonrepair) to parse and repair the broken JSON or,
if you must keep regexes, restrict them to operate only on the JSON structure
(outside string literals) — i.e., stop touching characters inside quoted
strings, and only apply the property-name and trailing-comma fixes to non-string
contexts; update the code that produces fixedJson (and references to
extractedJsonText) to use the chosen library or to include string-aware parsing
so string contents are not altered.
| export async function handleTokenizerEncode( | ||
| runtime: IAgentRuntime, | ||
| params: TokenizeTextParams | ||
| ): Promise<number[]> { | ||
| try { | ||
| const manager = localAIManager.getTokenizerManager(); | ||
| const config = localAIManager.getActiveModelConfig(); | ||
| return await manager.encode(params.prompt, config); | ||
| } catch (error) { | ||
| logger.error( | ||
| `[LocalAI] Error in TEXT_TOKENIZER_ENCODE handler: ${error instanceof Error ? error.message : String(error)}` | ||
| ); | ||
| throw error; | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
cd /tmp && find . -name "tokenizer.ts" -o -name "image.ts" -o -name "embedding.ts" -o -name "object.ts" 2>/dev/null | head -20Repository: elizaos-plugins/plugin-local-ai
Length of output: 57
🏁 Script executed:
# Check which handlers call initializeEnvironment
rg -n "initializeEnvironment" --type=ts src/Repository: elizaos-plugins/plugin-local-ai
Length of output: 1340
🏁 Script executed:
# Read the tokenizer.ts file to verify the handlers
cat -n src/models/tokenizer.tsRepository: elizaos-plugins/plugin-local-ai
Length of output: 2224
🏁 Script executed:
# Check image.ts, embedding.ts, object.ts for comparison
rg -B3 -A8 "export async function handle" src/models/image.ts src/models/embedding.ts src/models/object.tsRepository: elizaos-plugins/plugin-local-ai
Length of output: 3118
🏁 Script executed:
# Check if image.ts handler calls initializeEnvironment
cat -n src/models/image.tsRepository: elizaos-plugins/plugin-local-ai
Length of output: 1852
🏁 Script executed:
# Double-check which files have initializeEnvironment calls in handlers
rg -B5 "await localAIManager.initializeEnvironment" src/models/ --type=tsRepository: elizaos-plugins/plugin-local-ai
Length of output: 1540
🏁 Script executed:
# Check the manager to understand what initializeEnvironment does
sed -n '195,250p' src/manager.tsRepository: elizaos-plugins/plugin-local-ai
Length of output: 2539
🏁 Script executed:
# Check what getTokenizerManager() does and if it requires initialization
rg -B2 -A10 "getTokenizerManager" src/manager.tsRepository: elizaos-plugins/plugin-local-ai
Length of output: 431
🏁 Script executed:
# Check what getActiveModelConfig() does
rg -B2 -A10 "getActiveModelConfig" src/manager.tsRepository: elizaos-plugins/plugin-local-ai
Length of output: 423
🏁 Script executed:
# Check when tokenizerManager and activeModelConfig are initialized in manager.ts
rg -B5 -A5 "this.tokenizerManager|this.activeModelConfig" src/manager.ts | head -60Repository: elizaos-plugins/plugin-local-ai
Length of output: 2205
🏁 Script executed:
# Check the constructor and initialization order
sed -n '1,200p' src/manager.ts | grep -A30 "constructor"Repository: elizaos-plugins/plugin-local-ai
Length of output: 1333
Missing initializeEnvironment(runtime) call in tokenizer handlers.
Both handleTokenizerEncode and handleTokenizerDecode skip the await localAIManager.initializeEnvironment(runtime) call that other handlers (text, object, embedding) perform. The tokenizerManager is only initialized within _postValidateInit(), which is called during initializeEnvironment(). Without this call, the manager may not be properly initialized, causing runtime failures.
The runtime parameter is declared but unused in both handlers.
Apply to both tokenizer handlers
export async function handleTokenizerEncode(
runtime: IAgentRuntime,
params: TokenizeTextParams
): Promise<number[]> {
try {
+ await localAIManager.initializeEnvironment(runtime);
const manager = localAIManager.getTokenizerManager();
const config = localAIManager.getActiveModelConfig(); export async function handleTokenizerDecode(
runtime: IAgentRuntime,
params: DetokenizeTextParams
): Promise<string> {
try {
+ await localAIManager.initializeEnvironment(runtime);
const manager = localAIManager.getTokenizerManager();
const config = localAIManager.getActiveModelConfig();🤖 Prompt for AI Agents
In `@src/models/tokenizer.ts` around lines 18 - 31, Both tokenizer handlers
(handleTokenizerEncode and handleTokenizerDecode) omit the required environment
initialization so the tokenizerManager created in _postValidateInit() may be
uninitialized; add an awaited call to await
localAIManager.initializeEnvironment(runtime) at the start of each handler
(before calling localAIManager.getTokenizerManager()) and then use the manager
as currently done; also ensure the runtime parameter is used only for that call
so it is no longer unused.
| export function getEmbeddingDimensions(runtime: IAgentRuntime): number { | ||
| const dims = getSetting(runtime, 'LOCAL_EMBEDDING_DIMENSIONS', String(DEFAULT_EMBEDDING_DIMENSIONS)); | ||
| return parseInt(dims!, 10); | ||
| } |
There was a problem hiding this comment.
parseInt can return NaN for non-numeric input — no validation.
If LOCAL_EMBEDDING_DIMENSIONS is set to a non-numeric string (e.g., "abc"), parseInt returns NaN, which will silently propagate as the embedding dimension. Consider adding a fallback:
🛡️ Proposed fix
export function getEmbeddingDimensions(runtime: IAgentRuntime): number {
const dims = getSetting(runtime, 'LOCAL_EMBEDDING_DIMENSIONS', String(DEFAULT_EMBEDDING_DIMENSIONS));
- return parseInt(dims!, 10);
+ const parsed = parseInt(dims!, 10);
+ return Number.isNaN(parsed) ? DEFAULT_EMBEDDING_DIMENSIONS : parsed;
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| export function getEmbeddingDimensions(runtime: IAgentRuntime): number { | |
| const dims = getSetting(runtime, 'LOCAL_EMBEDDING_DIMENSIONS', String(DEFAULT_EMBEDDING_DIMENSIONS)); | |
| return parseInt(dims!, 10); | |
| } | |
| export function getEmbeddingDimensions(runtime: IAgentRuntime): number { | |
| const dims = getSetting(runtime, 'LOCAL_EMBEDDING_DIMENSIONS', String(DEFAULT_EMBEDDING_DIMENSIONS)); | |
| const parsed = parseInt(dims!, 10); | |
| return Number.isNaN(parsed) ? DEFAULT_EMBEDDING_DIMENSIONS : parsed; | |
| } |
🤖 Prompt for AI Agents
In `@src/utils/config.ts` around lines 101 - 104, The getEmbeddingDimensions
function currently uses parseInt on the value returned by getSetting and can
return NaN for non-numeric input; update getEmbeddingDimensions to validate the
parsed value (using Number.isFinite or Number.isNaN) and fall back to
DEFAULT_EMBEDDING_DIMENSIONS when parsing fails or yields a non-positive
integer. Specifically, parse the dims from getSetting(runtime,
'LOCAL_EMBEDDING_DIMENSIONS', String(DEFAULT_EMBEDDING_DIMENSIONS)), check the
result from parseInt(dims, 10) is a valid finite integer (and optionally > 0),
and return the default constant DEFAULT_EMBEDDING_DIMENSIONS when validation
fails so downstream code never receives NaN.
odilitime
changed the title
1 participant
Note
Medium Risk
Large refactor across initialization/config resolution plus build/test runner migration; runtime settings precedence and lazy model init paths could regress startup or model loading if assumptions changed.
Overview
Major refactor: Splits the former monolithic
src/index.tsinto a modular structure (src/manager.ts,src/models/*,src/utils/config.ts,src/constants.ts) and updatesindex.tsto mainly register handlers and delegate to the sharedlocalAIManager.Behavior/config changes: Adds a startup configuration banner (
src/banner.ts) and makes configuration resolution runtime-aware (prefersruntime.getSetting()overprocess.env), wiringruntimethrough model handlers andLocalAIManager.initializeEnvironment(runtime?). Logging is normalized with a consistent[LocalAI]prefix and emoji removed.Tooling updates: Replaces
tsupwith a Bun-based build runner (build.ts, updatedpackage.jsonscripts;tsupmoved to dev deps) and migrates tests fromvitesttobun:test, including module-levelmock.module()for native deps to avoid import-time crashes.Written by Cursor Bugbot for commit b91f9c4. This will update automatically on new commits. Configure here.
Summary by CodeRabbit
New Features
Changed
Documentation
Greptile Overview
Greptile Summary
This PR refactors the monolithic
src/index.ts(~2000 lines) into a modular architecture with separate files for the manager singleton (src/manager.ts), model handlers (src/models/), constants, config utilities, and a startup banner. It also migrates the build system from tsup to Bun.build and the test framework from vitest to bun:test.LocalAIManagerclass extracted tosrc/manager.ts; model handlers split intosrc/models/{text,object,embedding,image,audio,tokenizer}.ts; constants and config helpers extracted to dedicated filesIAgentRuntimetoinitializeEnvironment(), enabling character-level settings viaruntime.getSetting()in addition toprocess.envtsup.config.tswithbuild.tsusing monorepo's sharedcreateBuildRunner;tsupmoved from dependencies to devDependenciesmock.module()hoisting to handle native dependency mockinglazyInitVisioninsrc/manager.ts— the method body is a placeholder that setsvisionInitialized = truewithout performing any actual initialization, which will cause runtime failures for image descriptionpunishTokensFilteralways usessmallModelinsrc/manager.ts— when generating text withTEXT_LARGE, the token punishment filter still referencesthis.smallModelinstead of the active modelsrc/index.tsretains ~35 lines of unused imports from the pre-refactor monolithic fileConfidence Score: 3/5
lazyInitVisionplaceholder is a functional regression — image description will fail at runtime. ThepunishTokensFilterbug usingsmallModelwhen the large model is active is a secondary concern. The unused imports in index.ts are cosmetic but indicate incomplete cleanup. Score of 3 reflects that most of the PR is safe, but the vision initialization gap needs to be addressed before merging.src/manager.tsrequires attention for the emptylazyInitVisionimplementation (lines 786-801) and thepunishTokensFiltermodel mismatch (line 580-581).src/index.tshas unused imports that should be cleaned up.Important Files Changed
lazyInitVisionplaceholder (lines 786-801) that will cause runtime failures for image description. Also has a bug wherepunishTokensFilteralways referencessmallModeleven when using the large model.generateObjectByModelTypehelper and robust JSON parsing/repair utilities.createBuildRunner. Replaces tsup.config.ts with Bun-based build.Flowchart
flowchart TD A["index.ts<br/>(Plugin Registration)"] -->|"registers handlers"| B["models/text.ts<br/>TEXT_SMALL, TEXT_LARGE"] A -->|"registers handlers"| C["models/object.ts<br/>OBJECT_SMALL, OBJECT_LARGE"] A -->|"registers handlers"| D["models/embedding.ts<br/>TEXT_EMBEDDING"] A -->|"registers handlers"| E["models/image.ts<br/>IMAGE_DESCRIPTION"] A -->|"registers handlers"| F["models/audio.ts<br/>TRANSCRIPTION, TTS"] A -->|"registers handlers"| G["models/tokenizer.ts<br/>ENCODE, DECODE"] B -->|"initializeEnvironment(runtime)"| H["manager.ts<br/>LocalAIManager Singleton"] C -->|"initializeEnvironment(runtime)"| H D -->|"initializeEnvironment(runtime)"| H E -->|"describeImage()"| H F -->|"transcribeAudio() / generateSpeech()"| H G -->|"getTokenizerManager()"| H H -->|"reads config"| I["utils/config.ts<br/>getSetting(runtime, key)"] H -->|"uses defaults"| J["constants.ts<br/>Model names, WORDS_TO_PUNISH"] H -->|"validates"| K["environment.ts<br/>validateConfig()"] I -->|"1. runtime.getSetting()"| L["IAgentRuntime"] I -->|"2. process.env fallback"| M["Environment"] A -->|"init()"| N["banner.ts<br/>printLocalAiBanner()"] N -->|"reads config"| ILast reviewed commit: b91f9c4