Local AI Plugin

This plugin provides local AI model capabilities through the ElizaOS platform, supporting text generation, image analysis, speech synthesis, and audio transcription.

Note: This plugin uses node-llama-cpp v3.15.1 for improved performance and compatibility with newer GGUF model formats.

Usage

Add the plugin to your character configuration:

"plugins": ["@elizaos-plugins/plugin-local-ai"]

Configuration

The plugin is configured using environment variables (typically set in a .env file or via your deployment settings):

Or in .env file:

# Optional: Specify a custom directory for models (GGUF files)
# MODELS_DIR=/path/to/your/models

# Optional: Specify a custom directory for caching other components (tokenizers, etc.)
# CACHE_DIR=/path/to/your/cache

# Optional: Specify filenames for the text generation and embedding models within the models directory
# LOCAL_SMALL_MODEL=my-custom-small-model.gguf
# LOCAL_LARGE_MODEL=my-custom-large-model.gguf
# LOCAL_EMBEDDING_MODEL=my-custom-embedding-model.gguf

# Optional: Fallback dimension size for embeddings if generation fails. Defaults to the model's default (e.g., 384).
# LOCAL_EMBEDDING_DIMENSIONS=384

Configuration Options

• MODELS_DIR (Optional): Specifies a custom directory for storing model files (GGUF format). If not set, defaults to ~/.eliza/models.
• CACHE_DIR (Optional): Specifies a custom directory for caching other components like tokenizers. If not set, defaults to ~/.eliza/cache.
• LOCAL_SMALL_MODEL (Optional): Specifies the filename for the small text generation model (e.g., DeepHermes-3-Llama-3-3B-Preview-q4.gguf) located in the models directory.
• LOCAL_LARGE_MODEL (Optional): Specifies the filename for the large text generation model (e.g., DeepHermes-3-Llama-3-8B-q4.gguf) located in the models directory.
• LOCAL_EMBEDDING_MODEL (Optional): Specifies the filename for the text embedding model (e.g., bge-small-en-v1.5.Q4_K_M.gguf) located in the models directory.
• LOCAL_EMBEDDING_DIMENSIONS (Optional): Defines the expected dimension size for text embeddings. This is primarily used as a fallback dimension if the embedding model fails to generate an embedding. If not set, it defaults to the embedding model's native dimension size (e.g., 384 for bge-small-en-v1.5.Q4_K_M.gguf).

Features

The plugin provides these model classes:

• TEXT_SMALL: Fast, efficient text generation using smaller models
• TEXT_LARGE: More capable text generation using larger models
• TEXT_EMBEDDING: Generates text embeddings locally.
• IMAGE_DESCRIPTION: Local image analysis using Florence-2 vision model
• TEXT_TO_SPEECH: Local text-to-speech synthesis
• TRANSCRIPTION: Local audio transcription using Whisper

Text Generation

// Using small model
const smallResponse = await runtime.useModel(ModelType.TEXT_SMALL, {
  prompt: 'Generate a short response',
  stopSequences: [],
});

// Using large model
const largeResponse = await runtime.useModel(ModelType.TEXT_LARGE, {
  prompt: 'Generate a detailed response',
  stopSequences: [],
});

Text Embedding

const embedding = await runtime.useModel(ModelType.TEXT_EMBEDDING, {
  text: 'Text to get embedding for',
});

Image Analysis

const { title, description } = await runtime.useModel(
  ModelType.IMAGE_DESCRIPTION,
  'https://example.com/image.jpg'
);

Text-to-Speech

const audioStream = await runtime.useModel(ModelType.TEXT_TO_SPEECH, 'Text to convert to speech');

Audio Transcription

const transcription = await runtime.useModel(ModelType.TRANSCRIPTION, audioBuffer);

Recent Updates

v1.6.6-alpha.5 (January 2026)

Core API Compatibility Updates:

1. Logger API Changes: Updated all logging calls to use string-only format

   • WHY: The @elizaos/core logger API changed to accept only string arguments for consistency and performance
   • WHAT: Converted object logging to template literals with serialized content

2. Model Handler Signatures: Aligned with @elizaos/core type definitions

   • WHY: Core model parameter types were refactored to use union types for better flexibility
   • WHAT: Updated handlers to accept string | ParamsType | null where applicable
   • AFFECTED: TEXT_TOKENIZER_ENCODE, TEXT_TOKENIZER_DECODE, TEXT_EMBEDDING, IMAGE_DESCRIPTION, TRANSCRIPTION, TEXT_TO_SPEECH

3. GenerateText Refactor: Separated modelType from GenerateTextParams

   • WHY: Core API changed to pass model type as explicit parameter rather than embedded in params
   • WHAT: generateText(params, modelType) instead of generateText(params) with params.modelType

4. node-llama-cpp Upgrade: Updated from v3.10.0 to v3.15.1

   • WHY: Newer version provides better GGUF format support, performance improvements, and bug fixes
   • WHAT: Updated dependency in package.json

Breaking Changes:

• None for end users. All changes are internal API alignments with @elizaos/core.

Migration Notes:

• If you're extending this plugin, note that generateText now receives modelType as a separate parameter
• Model handlers must handle null and string parameter types in addition to typed objects