Skip to content

Vector Search and Embeddings

Jump to bottom
overthelex edited this page May 17, 2026 · 2 revisions

Vector Search & Embeddings

Vector Search and Embeddings represent the core semantic intelligence layer of the SecondLayer platform. This system enables the application to move beyond simple keyword matching to understand the legal context and meaning of user queries, court decisions, and legislative texts. By converting text into high-dimensional numerical vectors (embeddings), the system can perform similarity searches that find relevant legal precedents and procedural norms even when the specific terminology varies.

The infrastructure supports multiple embedding providers -- VoyageAI (primary) and AWS Bedrock (fallback) -- for vector generation, and uses Qdrant as the high-performance vector database for storage and retrieval. This system is integrated across multiple services, including legal document analysis, semantic legislation search, and secure document vault.

Sources: CLAUDE.md, mcp_backend/src/services/embedding-service.ts

Architecture and Components

The Vector Search architecture follows a pipeline where raw legal text is processed, embedded, and indexed for future retrieval. The embedding provider is configurable at runtime through the EMBEDDING_PROVIDER environment variable, supporting a dual-provider strategy.

graph TD
    Text[Raw Legal Text] --> Sectionizer[Semantic Sectionizer]
    Sectionizer --> Chunker[Text Chunker]
    Chunker --> Embedder[Embedding Service]
    Embedder -->|provider=voyage| Voyage[VoyageAI API voyage-3.5]
    Embedder -->|provider=bedrock| Bedrock[AWS Bedrock Titan Embed v2]
    Voyage --> Vector[1024-dim Vectors]
    Bedrock --> Vector
    Vector --> Qdrant[(Qdrant Vector DB)]
    Query[User Query] --> Embedder
    Qdrant -- Cosine Similarity --> Results[Ranked Results]
Loading

This diagram shows the data flow from raw legal text through the embedding process into the Qdrant database, and the subsequent retrieval flow for user queries. Both VoyageAI and Bedrock produce 1024-dimensional vectors.

Sources: mcp_backend/src/services/embedding-service.ts, mcp_backend/src/utils/voyage-client.ts, packages/shared/src/utils/bedrock-client.ts

Core Services

  • EmbeddingService (mcp_backend/src/services/embedding-service.ts): The primary interface for generating embeddings. It supports two providers -- VoyageAI (voyage-3.5 model, default) and AWS Bedrock (amazon.titan-embed-text-v2:0). Provider selection is controlled by the EMBEDDING_PROVIDER environment variable.
  • VoyageAIClient (mcp_backend/src/utils/voyage-client.ts): Dedicated client for the VoyageAI embedding API. Supports batch embedding (up to 50 texts per batch), round-robin API key rotation for rate-limit distribution, and automatic retry with exponential backoff on 429 responses.
  • Qdrant: A dedicated vector database that stores embeddings and associated metadata, allowing for efficient similarity searches using cosine distance. Two primary collections are maintained: legal_sections for court decisions and legislation, and vault_documents for private attorney documents.
  • LegislationService (mcp_backend/src/services/legislation-service.ts): Orchestrates the indexing of legislative articles. It breaks down laws into chunks, generates embeddings for each, and stores them in Qdrant for semantic legislation search.

Sources: mcp_backend/src/services/embedding-service.ts, mcp_backend/src/utils/voyage-client.ts, mcp_backend/src/services/legislation-service.ts

Implementation Details

Embedding Generation

The system has migrated from OpenAI's text-embedding-ada-002 (1536 dimensions) to a dual-provider architecture:

Provider Model Dimensions Notes
VoyageAI (default) voyage-3.5 1024 Strong multilingual support including Ukrainian; configurable via VOYAGEAI_EMBEDDING_MODEL
AWS Bedrock (fallback) amazon.titan-embed-text-v2:0 1024 Configurable via BEDROCK_EMBEDDING_MODEL; max 8000 characters per input

The active provider is selected with the EMBEDDING_PROVIDER environment variable (default: voyage). Both providers output vectors normalized to unit length with cosine distance.

The VoyageAI client distributes requests across multiple API keys (round-robin) and retries rate-limited requests with exponential backoff (up to 3 attempts). Batch size is capped at 50 texts per request.

Sources: mcp_backend/src/services/embedding-service.ts, mcp_backend/src/utils/voyage-client.ts, packages/shared/src/utils/bedrock-client.ts

Chunking Strategy

Text is split into chunks using a token-based approach:

  • Max chunk size: 512 tokens (approximated as 4 characters per token)
  • Overlap: 50 words carried over between consecutive chunks to preserve context at boundaries
  • Empty or whitespace-only texts are filtered out before embedding

Sources: mcp_backend/src/services/embedding-service.ts

Vector Indexing Workflow

Legislative documents are indexed through a specific workflow to ensure granular retrieval (e.g., specific articles or points within a law).

Step Action Description
1 Parsing Extracts articles (stattti) or points (punkty) from the Rada API response.
2 Chunking Splits large articles into overlapping segments using the adapter's createArticleChunks method.
3 Embedding Calls EmbeddingService.generateEmbedding() for each chunk.
4 Vector ID Generates a deterministic UUID from an MD5 hash of leg_{radaId}_art_{articleNumber}_chunk_{chunkIndex}.
5 Qdrant Upsert Stores the vector in Qdrant with metadata (rada_id, article_number, section_number, chapter_number, chunk_index, text, context).
6 PostgreSQL Sync Writes chunk data to the legislation_chunks table with an upsert (ON CONFLICT) for idempotency.

Sources: mcp_backend/src/services/legislation-service.ts

Qdrant Collections

The system maintains two separate Qdrant collections:

Collection Purpose Payload Indexes
legal_sections Court decisions, legislation chunks, and legal analysis sections Filtered by section_type, court, chamber, dispute_category, outcome, case_number, matter_id
vault_documents Private attorney documents (E2EE vault) Indexed on doc_id, section_type, matter_id, user_id

Both collections use 1024-dimensional vectors with cosine distance. On initialization, the service checks whether an existing collection's dimension matches the expected 1024 and recreates it if there is a mismatch.

Sources: mcp_backend/src/services/embedding-service.ts

Semantic Similarity Search

The search system supports rich metadata filtering. When a user queries the system, their query is embedded and compared against stored vectors. Qdrant's filter system enables narrowing results by:

  • Section type -- e.g., legislation, court decision heading, reasoning
  • Date range -- filter by document date
  • Court and chamber -- target specific courts or multiple chambers (OR logic)
  • Dispute category and outcome -- narrow by case type
  • Precedent status and deviation flag -- find landmark decisions or deviations
  • Matter ID -- scope search to a specific legal matter

Results are ranked by cosine similarity score and returned with full metadata payloads.

Sources: mcp_backend/src/services/embedding-service.ts

Data Structures and Configuration

Configuration Options

The vector search components are configured via environment variables to manage database connections, provider selection, and model choices.

Variable Type Description Default
QDRANT_URL String URL for the Qdrant vector database http://localhost:6333
QDRANT_API_KEY String Optional API key for Qdrant authentication -
EMBEDDING_PROVIDER String Active embedding provider (voyage or bedrock) voyage
VOYAGEAI_API_KEY String Primary VoyageAI API key -
VOYAGEAI_API_KEY_2 String Secondary VoyageAI API key (round-robin) -
VOYAGEAI_EMBEDDING_MODEL String VoyageAI model name voyage-3.5
BEDROCK_EMBEDDING_MODEL String AWS Bedrock embedding model ID amazon.titan-embed-text-v2:0

Sources: mcp_backend/src/services/embedding-service.ts, packages/shared/src/utils/bedrock-client.ts

Qdrant Metadata Payloads

When storing vectors in Qdrant, the system includes structured metadata payloads for filtering and source identification.

Legal sections collection (legal_sections):

  • doc_id -- Unique document identifier
  • section_type -- Type of legal section (e.g., reasoning, operative part)
  • text -- The raw text associated with the vector
  • date -- Document date
  • court -- Court name
  • case_number -- Case reference number
  • chamber -- Court chamber
  • dispute_category -- Category of the dispute
  • outcome -- Case outcome
  • deviation_flag -- Whether the decision deviates from established practice
  • precedent_status -- Status as a precedent
  • law_articles -- Array of referenced law articles
  • matter_id -- Optional link to a specific legal matter

Vault collection (vault_documents):

  • doc_id -- Document identifier
  • section_type -- Section type
  • text -- Document text
  • date -- Upload date
  • matter_id -- Associated legal matter
  • user_id -- Owner of the document

Sources: mcp_backend/src/services/embedding-service.ts

Infrastructure

Qdrant runs as a Docker container in both local and production environments:

  • Production: qdrant/qdrant:v1.17.0 with persistent volume (qdrant_prod_data)
  • Local: qdrant/qdrant:latest
  • Ports: 6333 (HTTP API), 6334 (gRPC) for the backend service

Sources: deployment/docker-compose.prod.yml, deployment/docker-compose.local.yml

Cost Tracking

Every embedding call is tracked for cost accounting. The EmbeddingService accepts a token usage callback that reports the number of tokens consumed, the model used, and the task type. This feeds into the platform's per-request cost tracking system.

Pricing (per 1M tokens):

Model Input Cost
voyage-3.5 $0.06
voyage-3.5-lite $0.02
voyage-law-2 $0.12
amazon.titan-embed-text-v2:0 $0.02

Sources: packages/shared/src/utils/model-selector.ts, packages/shared/src/services/base-cost-tracker.ts

Conclusion

Vector search and embeddings are fundamental to the SecondLayer platform's ability to provide semantic legal intelligence. The system has evolved from a single-provider OpenAI setup to a dual-provider architecture (VoyageAI + AWS Bedrock) with 1024-dimensional vectors, providing both performance and resilience. By leveraging Qdrant for vector storage with rich metadata filtering, the platform enables sophisticated features like semantic legislation search, court decision analysis, and private document retrieval -- all based on conceptual similarity rather than exact keyword matches.

Sources: mcp_backend/src/services/embedding-service.ts, mcp_backend/src/utils/voyage-client.ts, packages/shared/src/utils/bedrock-client.ts

Relevant source files

The following files were used as context for generating this wiki page:

SecondLayer

Getting Started

Architecture

Backend

Services

Business Logic

Frontend & Mobile

Operations

Clone this wiki locally