diff --git a/.opencode/agents/code-reviewer.md b/.opencode/agents/code-reviewer.md new file mode 100644 index 0000000..95958d5 --- /dev/null +++ b/.opencode/agents/code-reviewer.md @@ -0,0 +1,88 @@ +--- +description: Production-grade code reviewer - architecture, quality, security checks +mode: subagent +model: zai-coding-plan/glm-5-turbo +permission: + edit: deny + bash: allow + write: deny +steps: 20 +--- + +You are a strict production code reviewer for the Laravel RAG package. Your job is to catch every issue before it reaches production. + +## Review Process + +### Step 1: Gather context +```bash +git diff HEAD # all changes +git diff --staged # staged changes only +git status # current state +``` + +### Step 2: Check against checklist + +#### Architecture (Critical) +- No provider-specific code in `src/Services/` namespace +- All external calls go through interfaces in `src/Contracts/` +- Constructors depend on interfaces, not concrete implementations +- IngestionPipeline and QueryPipeline are fully provider-agnostic + +#### Code Quality (Critical) +- `declare(strict_types=1);` at the top of every PHP file +- All methods have explicit parameter types +- All methods have explicit return types +- Constructor dependencies use `readonly` +- No `mixed` type without justification + +#### Performance (Major) +- `embedBatch()` used instead of looping `embed()` +- `storeMany()` used instead of looping `store()` +- No N+1 queries +- No single-item operations inside ingestion loops + +#### Safety & Reliability (Major) +- `JSON_THROW_ON_ERROR` used when decoding JSON +- Domain errors throw custom exceptions +- Unique constraint violations handled in `storeMany()` +- All log entries include `trace_id` and `pipeline_stage` +- Operations are idempotent (safe to retry) + +#### PHPStan Compatibility (Major) +- Array shapes annotated with `@return array{...}` +- No dynamic property access +- Null safety handled with `?` operator where applicable +- Generic types used in collections + +#### Testing (Minor) +- `declare(strict_types=1)` in test files +- Test names clearly describe behavior +- AAA pattern (Arrange / Act / Assert) + +## Output Format + +``` +## Review Summary +- Critical: X issues +- Major: Y issues +- Minor: Z issues + +## Critical Issues +### [File:Line] Issue title +**Problem**: Detailed description +**Fix**: +```php +// corrected code +``` + +## Major Issues +... + +## Minor Issues +... + +## ✅ Looks good +- Things done correctly +``` + +If no issues found: report `✅ Code passes all production checks`. diff --git a/.opencode/agents/debugger.md b/.opencode/agents/debugger.md new file mode 100644 index 0000000..a3d3627 --- /dev/null +++ b/.opencode/agents/debugger.md @@ -0,0 +1,89 @@ +--- +description: Systematic debugger for RAG pipeline issues +mode: subagent +model: zai-coding-plan/glm-5-turbo +permission: + edit: deny + bash: allow + write: deny +steps: 30 +--- + +You are a debugging specialist for the Laravel RAG pipeline. You diagnose problems systematically — never guess. + +## Debugging Methodology + +### 1. Reproduce +Confirm the issue exists with specific data. + +### 2. Isolate +Identify the failing component: +- IngestionPipeline: DataSource → Chunker → EmbeddingDriver → VectorStore +- QueryPipeline: Retriever → PromptBuilder → LlmDriver + +### 3. Inspect +```bash +# Recent logs +tail -100 storage/logs/laravel.log | grep -E "trace_id|ERROR|error" + +# Database state +php artisan tinker --execute="echo DB::table('rag_chunks')->count();" + +# Config state +php artisan tinker --execute="print_r(config('rag'));" +``` + +### 4. Hypothesize & Verify +Form a hypothesis → test it immediately with a bash command. + +### 5. Report +Root cause + specific fix + how to prevent recurrence. + +## Common Issues + +### Ingestion Pipeline +| Symptom | Root Cause | Check | +|---------|------------|-------| +| Empty embeddings (zeros) | Wrong API key / rate limit | Check env, check logs | +| Zero chunks stored | VectorStore connection failure | DB connection, pgvector extension | +| Duplicate entry error | Unique constraint triggered | Check `insertOrIgnore` logic | +| Slow ingestion | Single-item operations in loop | Verify `embedBatch()` usage | + +### Query Pipeline +| Symptom | Root Cause | Check | +|---------|------------|-------| +| No results returned | Similarity threshold too strict | Lower threshold, check topK | +| Irrelevant chunks | Embedding dimension mismatch | Compare dimensions | +| LLM timeout | Prompt too long | Check `maxTokens` in PromptBuilder | +| Missing context | topK too low | Increase topK, check filters | + +### PHPStan Errors +| Error | Fix | +|-------|-----| +| Missing return type | Add explicit return type | +| Mixed type | Narrow with assert or conditional check | +| Property not found | Add `@property` PHPDoc | +| Array shape mismatch | Update `@return` annotation | + +## Output Format + +``` +## Debug Report: [Issue Description] + +### Reproduced +[Yes/No] - [reproduction steps] + +### Root Cause +[Failing component]: [concise explanation] + +### Evidence +[Log output or query result proving the issue] + +### Fix +```php +// specific code fix +``` + +### Prevention +[How to prevent this from recurring] +``` diff --git a/.opencode/agents/laravel-expert.md b/.opencode/agents/laravel-expert.md new file mode 100644 index 0000000..1c38f3e --- /dev/null +++ b/.opencode/agents/laravel-expert.md @@ -0,0 +1,43 @@ +--- +description: Laravel package development expert - primary agent for all implementation tasks +mode: primary +model: zai-coding-plan/glm-5-turbo +temperature: 0.2 +steps: 60 +permission: + edit: allow + bash: allow + write: allow +--- + +You are a Laravel package development expert specializing in clean architecture, service providers, and production-grade PHP code. + +## Expertise +- Service Provider patterns and package bootstrapping +- Facade creation and Laravel container bindings +- Dependency injection with constructor injection +- Artisan command development +- Package configuration structure +- Testing with Orchestra Testbench and Pest + +## Core Rules (from AGENTS.md) +- Every PHP file MUST have `declare(strict_types=1);` +- All methods MUST have explicit parameter and return types +- Constructor dependencies MUST use `readonly` +- Use `embedBatch()` and `storeMany()` — never single-item loops +- No provider-specific code in `src/Services/` +- PHPStan level 6 — zero tolerance for errors + +## When to Use Me +- Implementing any Laravel-specific feature +- Building pipelines, services, drivers +- Creating Artisan commands +- Setting up service providers and facades +- Fixing PHPStan errors +- Refactoring code for type safety + +## Guidelines +- Always provide complete implementations — never truncate code +- Explain root cause before proposing a fix +- Run `composer analyse` after every implementation +- Follow Conventional Commits for commit messages diff --git a/.opencode/agents/pipeline-validator.md b/.opencode/agents/pipeline-validator.md new file mode 100644 index 0000000..b61e935 --- /dev/null +++ b/.opencode/agents/pipeline-validator.md @@ -0,0 +1,62 @@ +--- +description: Validates RAG pipeline correctness +mode: subagent +model: zai-coding-plan/glm-5-turbo +permission: + edit: deny + bash: allow + write: deny +--- + +You are a pipeline validation specialist. Focus on data flow correctness and integration testing. + +## Expertise +- End-to-end pipeline testing +- Data flow validation +- Integration testing with real providers +- Debugging pipeline failures + +## When to Use Me +Invoke me for: +- Validating ingestion pipeline +- Validating query pipeline +- Debugging pipeline failures +- Checking data integrity + +## Validation Approach + +### Ingestion Pipeline Validation +1. Load a test document +2. Chunk the text +3. Generate embeddings (verify non-zero vectors) +4. Store in vector database (verify persistence) +5. Query back to verify storage + +### Query Pipeline Validation +1. Store known documents +2. Run test queries +3. Verify relevant chunks are retrieved +4. Check relevance scores +5. Validate prompt construction + +## Common Issues to Check +- Empty embeddings (all zeros) +- Chunks too large/small +- Metadata not stored correctly +- Poor retrieval accuracy +- Context window overflow +- Missing trace IDs in logs +- PHPStan errors + +## Validation Checklist +- [ ] Document loads correctly +- [ ] Chunks are reasonable size (500-1000 tokens) +- [ ] Embeddings are generated (not all zeros) +- [ ] Vectors stored in database +- [ ] Retrieved chunks are relevant +- [ ] Prompt fits in context window +- [ ] LLM generates response +- [ ] All files use declare(strict_types=1) +- [ ] All methods have explicit types +- [ ] Batch operations used (embedBatch, storeMany) +- [ ] Trace IDs in logs diff --git a/.opencode/agents/rag-architect.md b/.opencode/agents/rag-architect.md new file mode 100644 index 0000000..33ea391 --- /dev/null +++ b/.opencode/agents/rag-architect.md @@ -0,0 +1,99 @@ +--- +description: RAG system architecture specialist +mode: subagent +model: zai-coding-plan/glm-5-turbo +permission: + edit: ask + bash: ask + write: ask +--- + +You are a RAG (Retrieval-Augmented Generation) architecture expert focused on provider abstraction and modular design. + +## Expertise +- RAG pipeline architecture (ingestion + query) +- Vector database design +- Embedding generation strategies +- Document chunking algorithms +- Provider abstraction patterns + +## Critical Principles (NON-NEGOTIABLE) + +### 1. Provider Abstraction +- ALL provider interactions go through interfaces +- NO provider-specific logic in core domain +- Drivers are swappable via configuration + +**Example**: +```php +// ✅ CORRECT - Abstract interface +interface EmbeddingDriver { + public function embed(string $text): array; +} + +// ❌ WRONG - Direct provider call +class Chunker { + public function embed(string $text): array { + return OpenAI::embeddings()->create($text); // NO! + } +} +``` + +### 2. Modular Design +- Each component has a clear interface +- Dependencies explicit via constructor injection +- No hidden coupling + +### 3. Replaceable Components +- Any driver can be swapped without changing core logic +- Configuration drives provider selection + +## When to Use Me +Invoke me for: +- Designing RAG pipeline architecture +- Defining interfaces for components +- Implementing driver abstractions +- Reviewing code for provider lock-in +- Planning ingestion/query pipelines + +## MVP Architecture + +### Core Interfaces +``` +- DataSource // Load data from generic sources (not files-specific) +- Chunker // Split text into chunks +- EmbeddingDriver // Generate embeddings (provider abstraction) +- VectorStore // Store/search vectors (provider abstraction) +- Retriever // Retrieve relevant chunks (abstraction over VectorStore) +- PromptBuilder // Build prompts for LLM +- LlmDriver // Generate responses (provider abstraction) +``` + +### Pipeline Stages + +**Ingestion**: +``` +DataSource → Chunker → EmbeddingDriver → VectorStore +``` + +**Query**: +``` +Query → Retriever (handles embedding internally) → PromptBuilder → LlmDriver +``` + +**Key**: Retriever abstraction allows for: +- Reranking strategies +- Hybrid search (semantic + keyword) +- Metadata filtering +- Caching layers + +## Red Flags to Watch For +- Provider-specific code in Services namespace +- Hard-coded API calls (should be in Drivers) +- Components that depend on concrete classes (depend on interfaces) +- Configuration in code instead of config files +- Direct VectorStore::search() calls (use Retriever abstraction) +- Missing batch operations (use embedBatch, storeMany) +- Missing strict_types declarations +- Missing type hints on methods +- Loosely-typed arrays diff --git a/.opencode/agents/test-writer.md b/.opencode/agents/test-writer.md new file mode 100644 index 0000000..c44541d --- /dev/null +++ b/.opencode/agents/test-writer.md @@ -0,0 +1,92 @@ +--- +description: Pest test writing specialist +mode: subagent +model: zai-coding-plan/glm-5-turbo +permission: + edit: allow + write: allow + bash: allow +--- + +You are a Pest testing specialist for Laravel packages. + +## Focus +- Writing clear, maintainable Pest tests +- Test doubles (mocks, spies, fakes) +- Laravel integration testing +- Coverage analysis + +## When to Use Me +Invoke me for: +- Writing new tests +- Improving test coverage +- Creating test doubles +- Debugging test failures + +## Testing Strategy + +### Early Phase (Phases 1-3) +- Simple integration tests with real providers +- Focus on data flow correctness +- Test happy paths first +- Use real OpenAI/pgvector for validation + +### Late Phase (Phase 4) +- Comprehensive unit tests +- Mock providers for unit tests +- Edge case coverage +- Error handling tests + +## Guidelines +- Follow AAA pattern (Arrange, Act, Assert) +- Use descriptive test names +- Test one thing per test +- Use appropriate Pest assertions +- Mock external dependencies in late phases +- Use declare(strict_types=1) in all test files + +## Test Structure Template + +```php +toBe('expected'); +}); +``` + +## Integration Test Example (Early Phase) + +```php +ingest(__DIR__.'/fixtures/sample.txt'); + + // Assert + expect($result['stored'])->toBeGreaterThan(0); + expect($result['errors'])->toBe(0); +}); +``` diff --git a/.opencode/commands/analyse.md b/.opencode/commands/analyse.md new file mode 100644 index 0000000..ef4783a --- /dev/null +++ b/.opencode/commands/analyse.md @@ -0,0 +1,25 @@ +--- +description: Run PHPStan static analysis and fix all errors +agent: laravel-expert +template: Run PHPStan and fix all errors +--- + +Run PHPStan static analysis at level 6: + +!`composer analyse 2>&1` + +If errors are found: +1. Analyze each error — understand the root cause, not just the symptom +2. Fix type hints and return types +3. Add missing `declare(strict_types=1)` where needed +4. Fix invalid PHPDoc or array shape annotations +5. Re-run until output is clean + +Never use `@phpstan-ignore` — always fix the actual problem. + +Common issues: +- Missing return types on interface methods +- Loosely-typed arrays (use `array{key: type}` shapes) +- Missing `declare(strict_types=1)` at file top +- Mixed type usage without narrowing +- Invalid PHPDoc annotations diff --git a/.opencode/commands/commit.md b/.opencode/commands/commit.md new file mode 100644 index 0000000..6a67fd5 --- /dev/null +++ b/.opencode/commands/commit.md @@ -0,0 +1,27 @@ +--- +description: Suggest a conventional commit message from staged changes +agent: laravel-expert +template: Suggest a conventional commit message from staged changes +--- + +Show staged changes: + +!`git diff --staged 2>&1` + +!`git status --short 2>&1` + +Based on the changes, suggest a commit message following the Conventional Commits format: +`type(scope): description` + +Types: feat / fix / refactor / test / docs / chore / perf +Scopes: contracts / drivers / services / commands / tests / config + +Example output: +``` +feat(drivers): add OpenAI embedding driver with embedBatch support +``` + +Only suggest the message. Do not commit. The user will decide and run: +```bash +git commit -m "suggested message" +``` diff --git a/.opencode/commands/phase-1-foundation.md b/.opencode/commands/phase-1-foundation.md new file mode 100644 index 0000000..3b26bf0 --- /dev/null +++ b/.opencode/commands/phase-1-foundation.md @@ -0,0 +1,68 @@ +--- +description: Set up Phase 1 - Core abstractions and ingestion foundation +agent: laravel-expert +--- + +This command guides you through setting up Phase 1 of the RAG pipeline. + +## What We'll Build + +### 1. Define Core Interfaces +Create all interface files in `src/Contracts/`: +- DataSource.php +- Chunker.php +- EmbeddingDriver.php +- VectorStore.php +- Retriever.php +- PromptBuilder.php +- LlmDriver.php + +### 2. Implement Ingestion Components +- TextDataSource (drivers/data-source/) +- TextChunker (services/) + +### 3. Set Up Database Schema +- Create migration for rag_chunks table +- Add pgvector extension support + +### 4. Validate Each Component +- Test DataSource loads text +- Test Chunker creates reasonable chunks +- Verify schema is correct + +## Step-by-Step + +### Step 1: Create Interfaces +Use the laravel-expert agent to create all interface files with proper documentation. + +### Step 2: Implement DataSource +Create TextDataSource that: +- Checks if file exists +- Reads file content +- Returns array with id, content, and metadata + +### Step 3: Implement Chunker +Create TextChunker that: +- Splits text by character limit (configurable) +- Preserves paragraph boundaries +- Returns array with content, metadata, and index + +### Step 4: Create Database Schema +Create migration for: +- rag_chunks table (id, document_id, chunk_index, embedding, content, metadata, deleted_at, timestamps) +- pgvector extension +- Unique constraint on (document_id, chunk_index) +- Index on deleted_at + +### Step 5: Validate Each Component +Run tests to verify each component works independently. + +## Validation Checklist +- [ ] All interfaces created with proper methods +- [ ] DataSource returns proper structure (with id) +- [ ] Chunker preserves metadata and adds index +- [ ] Migration created and tested +- [ ] PHPStan passes (no errors) + +## Next Steps +After Phase 1 is complete, run `/phase-2-ingestion` to continue. diff --git a/.opencode/commands/phase-2-ingestion.md b/.opencode/commands/phase-2-ingestion.md new file mode 100644 index 0000000..3cc83de --- /dev/null +++ b/.opencode/commands/phase-2-ingestion.md @@ -0,0 +1,110 @@ +--- +description: Build Phase 2 - Complete ingestion pipeline +agent: laravel-expert +--- + +This command guides you through completing the ingestion pipeline. + +## What We'll Build + +### 1. Complete VectorStore +- Finish PgVectorStore implementation +- Implement store() and storeMany() methods +- Implement delete() method with soft delete support + +### 2. Create IngestionPipeline Service +- Orchestrate: DataSource → Chunker → EmbeddingDriver → VectorStore +- Use batch operations (embedBatch, storeMany) +- Handle errors gracefully with structured logging +- Return detailed stats + +### 3. Create Artisan Command +- `php artisan rag:ingest ` +- Validate input +- Show progress +- Report results with stats + +### 4. End-to-End Validation +- Test with real file +- Verify data persists in database +- Check logs for trace IDs +- Query back to validate storage + +## Step-by-Step + +### Step 1: Implement PgVectorStore +Complete the implementation: +- store() - Single vector storage +- storeMany() - Batch vector storage (preferred for ingestion) +- delete() - Soft delete with deleted_at timestamp + +Use insertOrIgnore() or handle unique constraint exceptions for idempotency. + +### Step 2: Create IngestionPipeline +```php + /tmp/test.txt + +# Run ingestion +php artisan rag:ingest /tmp/test.txt + +# Verify in database +php artisan tinker --execute=" +\$count = DB::table('rag_chunks')->count(); +echo 'Stored ' . \$count . ' chunks' . PHP_EOL; +" + +# Check logs for trace IDs +tail -f storage/logs/laravel.log | grep trace_id +``` + +## Validation Checklist +- [ ] PgVectorStore stores vectors correctly +- [ ] PgVectorStore supports batch operations +- [ ] IngestionPipeline orchestrates correctly +- [ ] Batch operations used (embedBatch, storeMany) +- [ ] Artisan command works +- [ ] Data persists in database +- [ ] Can query back stored data +- [ ] Logs contain trace IDs +- [ ] PHPStan passes +- [ ] Tests pass + +## Next Steps +After Phase 2 is complete, run `/phase-3-query` to build the query pipeline. diff --git a/.opencode/commands/phase-3-query.md b/.opencode/commands/phase-3-query.md new file mode 100644 index 0000000..54a1b6b --- /dev/null +++ b/.opencode/commands/phase-3-query.md @@ -0,0 +1,201 @@ +--- +description: Build Phase 3 - Query pipeline with LLM generation +agent: laravel-expert +--- + +This command guides you through building the complete query pipeline. + +## What We'll Build + +### 1. Implement Retriever +- Create SimilarityRetriever +- Accepts query string (not embeddings) +- Internally generates embedding +- Wraps VectorStore for clean abstraction + +### 2. Create PromptBuilder +- SimplePromptBuilder for MVP +- Combines query with retrieved context +- Enforces token limits via maxTokens parameter +- Ensure deterministic output + +### 3. Implement LlmDriver +- Create OpenAILlmDriver +- Calls OpenAI API with prompt +- Returns generated response +- Handles API errors + +### 4. Create QueryPipeline Service +- Orchestrate: Retriever → PromptBuilder → LlmDriver +- Note: Retriever now handles embedding internally +- Simplified orchestration + +### 5. Create Artisan Command +- `php artisan rag:query ` +- Show retrieved chunks (with scores) +- Display generated response +- Log with trace IDs + +## Step-by-Step + +### Step 1: Implement SimilarityRetriever +```php + + */ + public function retrieve(string $query, int $topK, array $filters = []): array + { + // Generate embedding internally + $queryEmbedding = $this->embedder->embed($query); + + // Query database with pgvector + // Add alias 'score' for similarity + // Note: This is DB-specific but acceptable for MVP + // TODO: Isolate in driver or store later + // ... implementation ... + } +} +``` + +### Step 2: Create SimplePromptBuilder +```php +retriever->retrieve($query, $topK); + + // Build prompt with token limits + $prompt = $this->promptBuilder->build($query, $chunks, $maxTokens); + + // Generate response + return $this->llm->generate($prompt); + } +} +``` + +### Step 5: Create Artisan Command +Create `rag:query` command that: +- Takes query as argument +- Shows retrieved chunks (with scores) +- Displays generated response +- Logs with trace IDs + +### Step 6: Test End-to-End +```bash +# First ingest some data +php artisan rag:ingest ./tests/fixtures/document.txt + +# Then query +php artisan rag:query "What is the main topic?" + +# Verify response is relevant +# Check logs for trace IDs +tail -f storage/logs/laravel.log | grep trace_id +``` + +## Validation Checklist +- [ ] SimilarityRetriever retrieves relevant chunks +- [ ] PromptBuilder constructs valid prompts +- [ ] PromptBuilder enforces token limits +- [ ] LlmDriver generates responses +- [ ] QueryPipeline orchestrates correctly +- [ ] Artisan command works +- [ ] Retrieved chunks are relevant +- [ ] Generated responses make sense +- [ ] All files use declare(strict_types=1) +- [ ] All methods have explicit types +- [ ] Logs contain trace IDs +- [ ] PHPStan passes +- [ ] Tests pass +- [ ] Batch operations used where appropriate + +## Next Steps +After Phase 3 is complete, run `/phase-4-integration` for Laravel integration. diff --git a/.opencode/commands/review.md b/.opencode/commands/review.md new file mode 100644 index 0000000..658a425 --- /dev/null +++ b/.opencode/commands/review.md @@ -0,0 +1,32 @@ +--- +description: Review code changes against production checklist +agent: code-reviewer +subtask: true +template: Review changes against production checklist +--- + +Review all current changes: + +!`git diff HEAD 2>&1` + +!`git status 2>&1` + +Check against the AGENTS.md production checklist: + +**Critical (must fix before commit):** +- `declare(strict_types=1)` on every new PHP file +- Explicit return types on every method +- No provider code in `src/Services/` +- No direct API calls in Services + +**Major (should fix):** +- Batch operations (`embedBatch`, `storeMany`) used correctly +- `JSON_THROW_ON_ERROR` when decoding JSON +- Unique constraint violations handled +- Trace IDs present in logs + +**Minor (can fix later):** +- Complete PHPDoc +- Test coverage for new code + +Conclude with: ✅ Ready to commit or ❌ Fix required with a specific list. diff --git a/.opencode/commands/test.md b/.opencode/commands/test.md new file mode 100644 index 0000000..61dba30 --- /dev/null +++ b/.opencode/commands/test.md @@ -0,0 +1,18 @@ +--- +description: Run Pest test suite and analyze failures +agent: laravel-expert +template: Run Pest test suite and analyze failures +--- + +Run the Pest test suite: + +!`composer test 2>&1` + +If there are failures: +1. Read each failure message carefully +2. Open the relevant test file and implementation +3. Identify the root cause (do not just fix the symptom) +4. Fix the implementation or the test depending on which is wrong +5. Re-run that specific test to confirm + +If all tests pass: report the number of tests passed and duration. diff --git a/.opencode/commands/validate-architecture.md b/.opencode/commands/validate-architecture.md new file mode 100644 index 0000000..26cb7e1 --- /dev/null +++ b/.opencode/commands/validate-architecture.md @@ -0,0 +1,68 @@ +--- +description: Validate architecture and code quality +agent: laravel-expert +--- + +Validate architecture and code quality for production readiness: + +## 1. Provider Abstraction Checks + +!`grep -r "OpenAI\|Pinecone\|Anthropic" src/Services/ || echo "✅ No provider code in Services"` + +!`grep -r "new OpenAI\|new Pinecone\|new Anthropic" src/Services/ || echo "✅ No hard-coded providers"` + +## 2. Code Quality Checks + +### Strict Types Check +!`find src -name "*.php" -exec grep -L "declare(strict_types=1)" {} \; | wc -l` + +# Count should match total PHP files +!`find src -name "*.php" | wc -l` + +!`echo "Files with strict_types: $(find src -name "*.php" -exec grep -L "declare(strict_types=1)" {} \; | wc -l)"` + +!`echo "Total PHP files: $(find src -name "*.php" | wc -l)"` + +### Typed Properties Check +!`grep -r "private readonly" src/ || echo "⚠️ Consider using typed readonly properties"` + +!`grep -r "protected readonly" src/ || echo "⚠️ Consider using typed readonly properties"` + +## 3. Batch Operations Check + +!`grep -r "embedBatch\|storeMany" src/ || echo "⚠️ Batch operations not implemented"` + +!`grep -r "function embedBatch" src/Contracts/EmbeddingDriver.php || echo "❌ embedBatch not in interface"` + +!`grep -r "function storeMany" src/Contracts/VectorStore.php || echo "❌ storeMany not in interface"` + +## 4. Trace ID Support + +!`grep -r "trace_id" src/ || echo "⚠️ Trace ID logging not implemented"` + +## 5. JSON_THROW_ON_ERROR + +!`grep -r "JSON_THROW_ON_ERROR" src/ || echo "⚠️ JSON_THROW_ON_ERROR not consistently used"` + +## 6. DB-Specific Logic + +!`grep -r "<=>\|pgvector" src/Services/ || echo "⚠️ DB-specific logic found (should be in Retriever)"` + +## 7. Return Type Declarations + +Check for missing return types: +!`grep -r "function \w+\(" src/Contracts/ | grep -v ": string\|: array\|: int\|: float\|: bool" || echo "⚠️ Some methods missing return types"` + +## Report + +Review all check outputs and provide: +- List of any files missing strict_types +- List of methods missing return types +- Recommendations for fixing any issues found + +If all checks pass, report: +``` +✅ All architecture checks passed +✅ All code quality checks passed +✅ Ready for production +``` diff --git a/.opencode/skills/iterative-testing/SKILL.md b/.opencode/skills/iterative-testing/SKILL.md new file mode 100644 index 0000000..9cd855c --- /dev/null +++ b/.opencode/skills/iterative-testing/SKILL.md @@ -0,0 +1,294 @@ +--- +name: iterative-testing +description: Phase-based iterative testing workflow with production standards +license: MIT +compatibility: opencode +metadata: + focus: production-readiness + testing: iterative +--- + +## What I Do + +Guide small, validated iterations: implement small part → validate → fix → proceed, with production-grade testing standards. + +## Testing Philosophy + +**Early Phases** (Phases 1-3): +- Focus on data flow correctness +- Simple integration tests with real providers +- CLI commands for manual validation +- Light testing, fast feedback + +**Late Phase** (Phase 4): +- Comprehensive unit tests +- Mock providers for isolation +- Edge case coverage +- Error handling tests + +## Workflow + +### For Each Small Part: + +1. **Implement Small Feature** + - One interface or method at a time + - Keep it focused and simple + - Use declare(strict_types=1) + +2. **Validate Immediately** + - Run phase command + - Check data flow + - Verify no provider lock-in + - Check code quality standards + +3. **Fix Issues** + - Don't proceed until working + - Run tests until green + - Ensure PHPStan passes + +4. **Repeat** + - Move to next small part + - Continue until phase complete + +## Phase Commands + +- `/phase-1-foundation` - Set up interfaces and ingestion foundation +- `/phase-2-ingestion` - Complete ingestion pipeline +- `/phase-3-query` - Complete query pipeline with LLM +- `/validate-architecture` - Check for provider lock-in and code quality + +## Code Quality Standards (NON-NEGOTIABLE) + +### Mandatory Requirements +1. **Strict Typing**: Every PHP file MUST include `declare(strict_types=1);` +2. **Explicit Types**: All methods MUST have parameter and return types +3. **Typed Properties**: Constructor dependencies MUST use `readonly` +4. **Batch Operations**: Use embedBatch() and storeMany() for efficiency +5. **Structured Logging**: All logs MUST include trace_id and pipeline_stage +6. **Duplicate Safety**: Handle unique constraint exceptions gracefully + +### Quality Checklist for Each Component + +```php +} + */ + public function load(string $source): array; +} +``` + +### Step 2: Implement +```php +} + */ + public function load(string $source): array + { + if (!file_exists($source)) { + throw new \RuntimeException("File not found: {$source}"); + } + + return [ + [ + 'id' => Str::uuid()->toString(), + 'content' => file_get_contents($source), + 'metadata' => ['source' => $source] + ] + ]; + } +} +``` + +### Step 3: Validate +```bash +# Create test file +echo "Sample text" > /tmp/test.txt + +# Test with CLI +php artisan tinker --execute=" +\$source = new \Thaolaptrinh\Rag\Drivers\DataSource\TextDataSource(); +\$data = \$source->load('/tmp/test.txt'); +print_r(\$data); +" + +# Verify output +``` + +### Step 4: Add Simple Test +```php +load(__DIR__.'/fixtures/sample.txt'); + + expect($data)->toHaveCount(1); + expect($data[0]['content'])->toBeString(); + expect($data[0]['id'])->toBeString(); +}); +``` + +### Step 5: Run Analysis +```bash +vendor/bin/pest tests/Unit/DataSourceTest.php +vendor/bin/phpstan analyse src/Drivers/DataSource/ +``` + +### Step 6: Fix and Repeat +- Fix any issues +- Don't proceed until green +- Then move to next component + +## Phase Checklists + +### Phase 1: Foundation +- [ ] All interfaces defined in Contracts namespace with explicit types +- [ ] All interfaces use declare(strict_types=1) +- [ ] TextDataSource implemented with id/content/metadata structure +- [ ] TextChunker implemented with metadata preservation and index +- [ ] PgVectorStore schema created (migration) +- [ ] Migration includes unique constraints and deleted_at +- [ ] Each component validated individually +- [ ] PHPStan passes (level 6+) + +### Phase 2: Ingestion +- [ ] PgVectorStore fully implemented with batch operations (storeMany) +- [ ] OpenAIEmbeddingDriver implements embedBatch +- [ ] IngestionPipeline service created with batch operations +- [ ] RagLogger added for structured logging +- [ ] `php artisan rag:ingest` command works +- [ ] End-to-end ingestion tested +- [ ] Data persists in database +- [ ] Logs contain trace IDs +- [ ] PHPStan passes (level 6+) +- [ ] Tests pass + +### Phase 3: Query + Generation +- [ ] SimilarityRetriever accepts query string (not embeddings) +- [ ] SimilarityRetriever handles embedding internally +- [ ] SimplePromptBuilder implemented with token limits +- [ ] PromptBuilder enforces deterministic output +- [ ] OpenAILlmDriver implemented +- [ ] QueryPipeline service created +- [ ] `php artisan rag:query` command works +- [ ] End-to-end query tested +- [ ] Logs contain trace IDs +- [ ] PHPStan passes (level 6+) +- [ ] Tests pass + +### Phase 4: Laravel Integration +- [ ] RagServiceProvider created +- [ ] Rag facade created +- [ ] config/rag.php created with all configuration +- [ ] All services bound correctly with match statements +- [ ] Configuration controls providers +- [ ] Comprehensive tests written +- [ ] Documentation complete +- [ ] PHPStan passes (level 6+) +- [ ] Tests pass + +## Validation Commands + +### Check Architecture +```bash +/validate-architecture +``` +Checks: +- No provider code in Services +- All Services depend on Contracts +- Configuration drives provider selection +- All files use strict_types +- Batch operations implemented +- Trace ID logging present + +### Run Static Analysis +```bash +/analyse +``` +Runs PHPStan at level 6 + +### Format Code +```bash +/lint +``` +Runs Laravel Pint + +## Common Issues + +### Issue: Provider Code in Services +**Symptom**: grep finds "OpenAI" in Services directory +**Fix**: Move provider logic to Drivers namespace + +### Issue: Missing Strict Types +**Symptom**: File missing `declare(strict_types=1);` +**Fix**: Add at top of every PHP file + +### Issue: Missing Return Types +**Symptom**: PHPStan errors about missing return types +**Fix**: Add explicit return types to all methods + +### Issue: Concrete Dependencies +**Symptom**: Constructor uses concrete class instead of interface +**Fix**: Change type hint to interface + +### Issue: No Batch Operations +**Symptom**: Individual store/embed calls instead of batch +**Fix**: Use embedBatch() and storeMany() + +### Issue: Missing Trace IDs +**Symptom**: Logs don't show trace_id +**Fix**: Use RagLogger for all logging + +### Issue: Duplicate Entries +**Symptom**: Same chunk stored multiple times +**Fix**: Handle unique constraint exceptions in storeMany() + +## Progress Tracking + +Use the phase checklists above to track progress. Mark each item as complete before moving to the next phase. diff --git a/.opencode/skills/rag-core/SKILL.md b/.opencode/skills/rag-core/SKILL.md new file mode 100644 index 0000000..5d2f9e8 --- /dev/null +++ b/.opencode/skills/rag-core/SKILL.md @@ -0,0 +1,304 @@ +--- +name: rag-core +description: Core RAG development workflow - interfaces, ingestion, query, generation +license: MIT +compatibility: opencode +metadata: + focus: production-readiness + code-quality: strict +--- + +## What I Do + +Guide complete RAG pipeline development with emphasis on provider abstraction, modular design, and production-grade code quality. + +## Core Architecture + +### Interfaces (Contracts Namespace) +``` +Thaolaptrinh\Rag\Contracts\ + +├── DataSource // Load data from sources (generic) +├── Chunker // Split text into chunks +├── EmbeddingDriver // Generate embeddings (provider abstraction) +├── VectorStore // Store/search vectors (provider abstraction) +├── Retriever // Retrieve relevant chunks (handles embedding internally) +├── PromptBuilder // Build prompts for LLM +├── LlmDriver // Generate responses (provider abstraction) +``` + +### Pipeline Stages + +**Ingestion**: +``` +DataSource → Chunker → EmbeddingDriver (batch) → VectorStore (batch) +``` + +**Query**: +``` +Query → Retriever (handles embedding internally) → PromptBuilder → LlmDriver +``` + +### Critical Principles (NON-NEGOTIABLE) + +#### 1. Provider Abstraction +- ALL provider interactions go through interfaces +- NO provider-specific logic in core domain +- Drivers are swappable via configuration + +**Example**: +```php +// ✅ CORRECT +interface EmbeddingDriver { + public function embed(string $text): array; + public function embedBatch(array $texts): array; // PREFERRED for ingestion +} + +// ❌ WRONG +class Chunker { + public function embed(string $text): array { + return OpenAI::embeddings()->create($text); // NO! + } +} +``` + +#### 2. Generic Data Sources +- Use `DataSource` interface (not DocumentLoader) +- Support any data source, not just files +- Keep system extensible + +```php +interface DataSource { + /** + * @return array}> + */ + public function load(string $source): array; +} + +// MVP: TextDataSource +// Future: PdfDataSource, DatabaseDataSource, ApiDataSource +``` + +#### 3. Retrieval Abstraction +- Use `Retriever` interface (not direct VectorStore::search) +- Separates storage logic from retrieval logic +- Enables reranking, hybrid search, filtering + +```php +interface Retriever { + /** + * @param string $query User query (raw text) + * @return array}> + */ + public function retrieve(string $query, int $topK, array $filters = []): array; +} + +// MVP: SimilarityRetriever (handles embedding internally) +// Future: HybridRetriever, RerankingRetriever, CachedRetriever +``` + +#### 4. Batch Operations (REQUIRED) +- Use `embedBatch()` instead of individual `embed()` calls +- Use `storeMany()` instead of individual `store()` calls +- Critical for production performance + +## Code Quality Standards (NON-NEGOTIABLE) + +### 1. Strict Typing +- **MUST** use `declare(strict_types=1);` at the top of every PHP file +- **MUST** use explicit parameter and return types on ALL methods +- **MUST** use `readonly` properties for constructor dependencies +- **MUST** avoid loosely-typed arrays in favor of specific array shapes + +### 2. PHPStan Compatibility +- All code must pass PHPStan level 6+ +- Use specific types instead of `mixed` where possible +- Avoid dynamic property access +- Use null-safe operators appropriately + +### 3. Production Best Practices +- Batch operations preferred (embedBatch, storeMany) +- Structured logging with trace IDs +- Duplicate-safe database operations +- Error handling with custom exceptions +- Idempotent ingestion via unique constraints + +### Example (CORRECT): +```php +}> + */ + public function load(string $source): array; +} +``` + +### ❌ Direct Vector Access +```php +// WRONG - Skips retrieval abstraction +class QueryPipeline { + $chunks = $this->vectorStore->search(...); +} + +// CORRECT - Retriever abstraction +class QueryPipeline { + $chunks = $this->retriever->retrieve($query, $topK); +} +``` + +### ❌ Single-Item Operations +```php +// WRONG - Inefficient +foreach ($chunks as $chunk) { + $this->vectorStore->store($embedding, $content, $metadata); +} + +// CORRECT - Batch operations +$items = []; +foreach ($chunks as $chunk) { + $items[] = ['embedding' => $embed, 'content' => $chunk['content'], 'metadata' => $chunk['metadata']]; +} +$this->vectorStore->storeMany($items); +``` + +## Namespace Structure + +``` +Thaolaptrinh\Rag\ +├── Contracts\ # All interfaces with explicit types +├── Drivers\ +│ ├── DataSource\ +│ │ └── TextDataSource.php +│ ├── Embeddings\ +│ │ └── OpenAIEmbeddingDriver.php +│ ├── VectorStores\ +│ │ └── PgVectorStore.php +│ └── Llm\ +│ └── OpenAILlmDriver.php +├── Services\ +│ ├── IngestionPipeline.php +│ ├── QueryPipeline.php +│ ├── Retrievers\ +│ │ └── SimilarityRetriever.php +│ └── Logging\ +│ └── RagLogger.php +├── Exceptions\ +│ ├── RagException.php +│ ├── EmbeddingFailedException.php +│ ├── StorageFailedException.php +│ ├── RetrievalFailedException.php +│ └── GenerationFailedException.php +├── Models\ +├── Facades\ +├── Commands\ +└── Exceptions\ +``` + +## When to Use Me + +Use for: +- Designing RAG pipeline architecture +- Implementing interfaces with explicit types +- Creating driver abstractions +- Building ingestion/query pipelines +- Reviewing code for provider lock-in +- Ensuring code quality standards + +## Validation Checklist + +Before considering a component complete: +- [ ] File has `declare(strict_types=1);` +- [ ] All methods have explicit parameter types +- [ ] All methods have explicit return types +- [ ] Constructor dependencies use `readonly` +- [ ] Batch operations are used (embedBatch, storeMany) +- [ ] JSON_THROW_ON_ERROR is used for JSON decoding +- [ ] Custom exceptions are used for errors +- [ ] Structured logging includes trace IDs +- [ ] PHPStan level 6+ compatible diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..eb3e408 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,224 @@ +# Laravel RAG - Agent Rules + +This document defines the global system rules for the Laravel RAG package development. + +## Architecture Rules + +### Pipeline-Based Design +- **Ingestion Pipeline**: Separate pipeline for data ingestion (DataSource → Chunker → EmbeddingDriver → VectorStore) +- **Query Pipeline**: Separate pipeline for queries (Query → Retriever → PromptBuilder → LlmDriver) +- **No mixing**: Never combine ingestion and query logic in the same service +- **Independence**: Each pipeline should be independently testable and deployable + +### Separation of Concerns +- **Core Domain**: Interfaces and services must remain provider-agnostic +- **Driver Layer**: All provider-specific code lives in Drivers namespace +- **Service Layer**: Business logic only, no external API calls +- **Configuration**: Provider selection via config only + +### No Tight Coupling +- Services depend on interfaces, not concrete implementations +- Use dependency injection throughout +- No static dependencies on external services +- Facades only for Laravel integration layer + +## Dependency Rules + +### No Direct External APIs in Core +- **Core Services**: MUST NOT call OpenAI, Anthropic, Pinecone APIs directly +- **All Integrations**: MUST go through driver interfaces +- **Service Boundaries**: IngestionPipeline and QueryPipeline are provider-agnostic + +### Driver Abstraction +- **EmbeddingDriver**: Interface for all embedding providers +- **VectorStore**: Interface for all vector databases +- **LlmDriver**: Interface for all LLM providers +- **DataSource**: Interface for all data sources +- **Retriever**: Interface for all retrieval strategies + +### Configuration-Driven Provider Selection +- Providers selected via `config/rag.php` +- Service provider binds implementations based on config +- No hardcoded provider references in services +- Match expressions for clean provider switching + +## Code Quality Rules + +### Strict Typing (NON-NEGOTIABLE) +- **Every PHP file**: MUST start with `declare(strict_types=1);` +- **All methods**: MUST have explicit parameter types +- **All methods**: MUST have explicit return types +- **No mixed types**: Use specific array shapes in PHPDoc +- **Readonly properties**: Constructor dependencies MUST use `readonly` + +### PHPStan Compatibility +- **Minimum level**: 6+ +- **Zero tolerance**: All code must pass PHPStan analysis +- **Run before commit**: Always run `composer analyse` before committing +- **Fix immediately**: Never commit with PHPStan errors + +### Production Best Practices +- **Batch operations**: Use `embedBatch()` and `storeMany()` (10-50x performance) +- **JSON_THROW_ON_ERROR**: Required for all JSON decoding +- **Custom exceptions**: Use domain-specific exceptions +- **Structured logging**: All logs include trace_id and pipeline_stage +- **Duplicate safety**: Handle unique constraint violations gracefully +- **Idempotent operations**: Ingestion must be safe to retry + +## Development Rules + +### Phase-Based Development +Follow the 4-phase implementation order: + +1. **Phase 1**: Core abstractions + ingestion foundation + - Define all interfaces + - Implement DataSource and Chunker + - Create database migration + - Validate with PHPStan + +2. **Phase 2**: Complete ingestion pipeline + - Implement EmbeddingDriver with embedBatch() + - Implement VectorStore with storeMany() + - Implement IngestionPipeline service + - Test end-to-end ingestion + +3. **Phase 3**: Query + generation pipeline + - Implement Retriever (accepts string query) + - Implement PromptBuilder with token limits + - Implement LlmDriver + - Implement QueryPipeline service + +4. **Phase 4**: Laravel integration + - Create service provider + - Create facade + - Create configuration + - Write documentation + +### Validation Steps +- **Never skip**: PHPStan validation after each phase +- **Never skip**: Manual testing with sample data +- **Never skip**: Architecture validation (no provider lock-in) +- **Fix before proceeding**: Do not continue to next phase with errors + +### No New Abstractions +- **Use existing interfaces**: Do not create new interfaces without justification +- **Follow patterns**: Use established patterns from existing code +- **Simple over clever**: Prefer clear, simple code +- **YAGNI principle**: Do not add features not needed for MVP + +## Constraints + +### Keep Core Generic +- **No file-specific logic**: DataSource is generic, not "DocumentLoader" +- **No database assumptions**: VectorStore interface works with any database +- **No model lock-in**: Easy to swap OpenAI for Anthropic, Cohere, local models +- **Extensibility**: New drivers implement interfaces, no core changes needed + +### Avoid Vendor Lock-In +- **Interface boundaries**: Clear separation between core and providers +- **Swappable drivers**: Change config to switch providers +- **No provider language**: Core code never mentions "OpenAI", "Pinecone", etc. +- **Standard protocols**: Use industry-standard patterns (embeddings, vectors) + +### Avoid Over-Engineering +- **MVP focus**: Text files only in Phase 1-3 +- **Simple solutions**: Basic similarity search, no reranking +- **Pragmatic choices**: pgvector for MVP (not custom vector DB) +- **Future extensibility**: Architecture supports advanced features later + +## Critical Success Factors + +### Must Have +- ✅ Provider-agnostic architecture +- ✅ Batch operations for performance +- ✅ Idempotent ingestion +- ✅ Structured logging with trace IDs +- ✅ PHPStan level 6+ compatible +- ✅ Duplicate-safe database operations + +### Must Not Have +- ❌ Provider code in core services +- ❌ Direct API calls in pipelines +- ❌ File-specific data source interface +- ❌ Single-item embed/store operations in pipelines +- ❌ Missing strict_types declarations +- ❌ Unhandled unique constraint violations + +## Development Workflow + +1. **Before coding**: Read relevant AGENTS.md sections +2. **While coding**: Follow code quality rules strictly +3. **After coding**: Run PHPStan and fix all errors +4. **Before commit**: Ensure all validation steps pass +5. **Before PR**: Review architecture rules for compliance + +## Git Conventions + +### Conventional Commits (REQUIRED) +Format: `type(scope): description` + +| Type | When to use | +|------|-------------| +| `feat` | New feature | +| `fix` | Bug fix | +| `refactor` | Code change that is not a feature or bug fix | +| `test` | Adding or updating tests | +| `docs` | Documentation changes | +| `chore` | Build process, dependencies, config | +| `perf` | Performance improvement | + +Scopes: `contracts`, `drivers`, `services`, `commands`, `tests`, `config` + +Examples: +- `feat(drivers): add OpenAI embedding driver with batch support` +- `fix(services): handle unique constraint violation in storeMany` +- `test(ingestion): add end-to-end pipeline integration test` +- `refactor(retriever): extract embedding logic to EmbeddingDriver` + +### Commit Rules +- Small, focused commits (one logical change per commit) +- Always run `composer analyse` before committing +- Always run `composer test` before pushing +- Never commit `.env`, `vendor/`, `build/` + +## OpenCode Commands Reference + +| Command | Purpose | +|---------|---------| +| `/analyse` | Run PHPStan and fix errors | +| `/lint` | Format code with Laravel Pint | +| `/test` | Run Pest test suite | +| `/test-coverage` | Run tests with coverage report | +| `/review` | Review changes before committing | +| `/commit` | Suggest a conventional commit message | +| `/refactor` | Run Rector auto-refactor (dry-run first) | +| `/debug ` | Debug a pipeline issue systematically | +| `/phase-1-foundation` | Build Phase 1 | +| `/phase-2-ingestion` | Build Phase 2 | +| `/phase-3-query` | Build Phase 3 | +| `/validate-architecture` | Validate full architecture | + +## Response Preferences + +- **Code**: Always provide full implementation, never truncate +- **Errors**: Explain root cause before proposing a fix +- **PHPStan**: Never suggest `@phpstan-ignore` — fix the actual problem +- **Brevity**: Do not over-explain things that are obvious from the code + +## When in Doubt + +- **Question**: Should I add this feature? + - **Answer**: Is it needed for MVP? If no, defer it. + +- **Question**: Should I use this provider? + - **Answer**: Is there an interface for it? If no, create interface first. + +- **Question**: Should I skip PHPStan validation? + - **Answer**: NEVER. Fix all errors before proceeding. + +- **Question**: Should I add a shortcut? + - **Answer**: NO. Batch operations are required for production performance. + +--- + +**Remember**: These rules ensure the package remains maintainable, extensible, and production-ready. Follow them strictly. diff --git a/IMPLEMENTATION_PLAN.md b/IMPLEMENTATION_PLAN.md new file mode 100644 index 0000000..4eaa48d --- /dev/null +++ b/IMPLEMENTATION_PLAN.md @@ -0,0 +1,179 @@ +# Laravel RAG Implementation Plan + +**Created:** 2026-03-26 +**Status:** Ready to Execute + +## Overview + +Build a production-ready, provider-agnostic RAG (Retrieval-Augmented Generation) core package for Laravel with emphasis on modularity, strict typing, and production-grade code quality. + +## Architecture Principles + +1. **Provider Abstraction**: All provider interactions through interfaces +2. **Generic Design**: DataSource interface (not DocumentLoader) for any data source +3. **Retrieval Abstraction**: Retriever interface (not direct VectorStore access) +4. **Batch Operations**: Use embedBatch() and storeMany() for production performance +5. **Code Quality**: strict_types, explicit types, readonly properties, PHPStan level 6+ + +## Implementation Phases + +### Phase 1: Core Abstractions + Ingestion Foundation + +**Goal**: Define all interfaces and basic data processing components + +**Tasks**: +1. Create exception classes (base RagException + 4 specific exceptions) +2. Create RagLogger for structured logging with trace IDs +3. Define all 7 core interfaces with explicit types +4. Implement TextDataSource (load text files with id/content/metadata) +5. Implement TextChunker (split text with metadata preservation) +6. Create database migration for rag_chunks table +7. Validate each component independently +8. Run PHPStan analysis + +**Verification**: +- All files have `declare(strict_types=1);` +- All methods have explicit parameter and return types +- Constructor dependencies use `readonly` +- PHPStan level 6+ passes +- Manual testing with sample data + +### Phase 2: Complete Ingestion Pipeline + +**Goal**: Build end-to-end ingestion with batch operations + +**Tasks**: +1. Implement OpenAIEmbeddingDriver with embedBatch() support +2. Implement PgVectorStore with storeMany() and duplicate handling +3. Implement IngestionPipeline service (batch ingestion) +4. Create `rag:ingest` Artisan command +5. Test end-to-end ingestion +6. Verify batch operations working +7. Verify duplicate-safe inserts +8. Verify trace ID logging + +**Verification**: +- Batch operations used (not individual embed/store calls) +- Data persists in rag_chunks table +- Logs contain trace_id and pipeline_stage +- No provider lock-in in services +- PHPStan passes + +### Phase 3: Query + Generation Pipeline + +**Goal**: Build retrieval and LLM generation + +**Tasks**: +1. Implement SimilarityRetriever (accepts query string, handles embedding internally) +2. Implement SimplePromptBuilder with token limits +3. Implement OpenAILlmDriver for response generation +4. Implement QueryPipeline service +5. Create `rag:query` Artisan command +6. Test end-to-end query flow +7. Verify retrieval accuracy +8. Verify LLM responses + +**Verification**: +- Retriever accepts string query (not embeddings) +- PromptBuilder enforces maxTokens +- QueryPipeline uses contracts only +- End-to-end query produces responses +- PHPStan passes + +### Phase 4: Laravel Integration + +**Goal**: Complete Laravel package integration + +**Tasks**: +1. Create RagServiceProvider with service bindings +2. Create Rag facade +3. Create config/rag.php with all driver configurations +4. Write comprehensive unit tests +5. Write integration tests +6. Create documentation (README, usage examples) +7. Final validation of all components + +**Verification**: +- All services bound via configuration +- Providers swappable via config +- All tests pass +- Documentation complete +- PHPStan passes +- Package installable in Laravel app + +## Code Quality Standards + +**NON-NEGOTIABLE Requirements**: +1. Every PHP file MUST start with `declare(strict_types=1);` +2. All methods MUST have explicit parameter types +3. All methods MUST have explicit return types +4. Constructor dependencies MUST use `readonly` +5. Batch operations MUST be used (embedBatch, storeMany) +6. JSON decoding MUST use `JSON_THROW_ON_ERROR` +7. All logging MUST use RagLogger with trace IDs +8. Custom exceptions MUST be used for errors + +## Directory Structure + +``` +src/ +├── Contracts/ # All interfaces +│ ├── DataSource.php +│ ├── Chunker.php +│ ├── EmbeddingDriver.php +│ ├── VectorStore.php +│ ├── Retriever.php +│ ├── PromptBuilder.php +│ └── LlmDriver.php +├── Drivers/ +│ ├── DataSource/ +│ │ └── TextDataSource.php +│ ├── Embeddings/ +│ │ └── OpenAIEmbeddingDriver.php +│ ├── VectorStores/ +│ │ └── PgVectorStore.php +│ └── Llm/ +│ └── OpenAILlmDriver.php +├── Services/ +│ ├── IngestionPipeline.php +│ ├── QueryPipeline.php +│ ├── Retrievers/ +│ │ └── SimilarityRetriever.php +│ └── Logging/ +│ └── RagLogger.php +├── Exceptions/ +│ ├── RagException.php +│ ├── EmbeddingFailedException.php +│ ├── StorageFailedException.php +│ ├── RetrievalFailedException.php +│ └── GenerationFailedException.php +├── Commands/ +│ ├── IngestCommand.php +│ └── QueryCommand.php +├── Facades/ +│ └── Rag.php +├── RagServiceProvider.php +└── helpers.php +``` + +## Pre-Build Fixes + +Before starting implementation, ensure these fixes are applied: + +1. **RagLogger syntax**: Fix array key syntax error +2. **RagLogger namespace**: Use `Thaolaptrinh\Rag\Logging` +3. **RagLogger imports**: Add `use Illuminate\Support\Str;` +4. **Retrieval query**: Ensure similarity score has alias +5. **JSON_THROW_ON_ERROR**: Use consistently in all JSON decoding + +## Success Criteria + +- [ ] All 4 phases complete +- [ ] End-to-end ingestion works +- [ ] End-to-end query works +- [ ] All tests pass +- [ ] PHPStan level 6+ passes +- [ ] No provider lock-in +- [ ] Batch operations implemented +- [ ] Structured logging with trace IDs +- [ ] Documentation complete diff --git a/README.md b/README.md index b3a0882..c798a3e 100644 --- a/README.md +++ b/README.md @@ -1,93 +1,204 @@ -# :package_description +# Laravel RAG -[![Latest Version on Packagist](https://img.shields.io/packagist/v/:vendor_slug/:package_slug.svg?style=flat-square)](https://packagist.org/packages/:vendor_slug/:package_slug) -[![GitHub Tests Action Status](https://img.shields.io/github/actions/workflow/status/:vendor_slug/:package_slug/run-tests.yml?branch=main&label=tests&style=flat-square)](https://github.com/:vendor_slug/:package_slug/actions?query=workflow%3Arun-tests+branch%3Amain) -[![GitHub Code Style Action Status](https://img.shields.io/github/actions/workflow/status/:vendor_slug/:package_slug/fix-php-code-style-issues.yml?branch=main&label=code%20style&style=flat-square)](https://github.com/:vendor_slug/:package_slug/actions?query=workflow%3A"Fix+PHP+code+style+issues"+branch%3Amain) -[![Total Downloads](https://img.shields.io/packagist/dt/:vendor_slug/:package_slug.svg?style=flat-square)](https://packagist.org/packages/:vendor_slug/:package_slug) - ---- -This repo can be used to scaffold a Laravel package. Follow these steps to get started: +[![Latest Version on Packagist](https://img.shields.io/packagist/v/thaolaptrinh/laravel-rag.svg?style=flat-square)](https://packagist.org/packages/thaolaptrinh/laravel-rag) +[![Total Downloads](https://img.shields.io/packagist/dt/thaolaptrinh/laravel-rag.svg?style=flat-square)](https://packagist.org/packages/thaolaptrinh/laravel-rag) +[![PHPStan](https://img.shields.io/badge/PHPStan-level%206-brightgreen.svg?style=flat-square)](https://phpstan.org/) -1. Press the "Use this template" button at the top of this repo to create a new repo with the contents of this skeleton. -2. Run "php ./configure.php" to run a script that will replace all placeholders throughout all the files. -3. Have fun creating your package. -4. If you need help creating a package, consider picking up our Laravel Package Training video course. ---- - -This is where your description should go. Limit it to a paragraph or two. Consider adding a small example. +A production-ready, provider-agnostic RAG (Retrieval-Augmented Generation) package for Laravel. Build AI-powered applications with modular architecture, strict typing, and enterprise-grade code quality. -## Support us +## Highlights -[](https://spatie.be/github-ad-click/:package_name) - -We invest a lot of resources into creating [best in class open source packages](https://spatie.be/open-source). You can support us by [buying one of our paid products](https://spatie.be/open-source/support-us). - -We highly appreciate you sending us a postcard from your hometown, mentioning which of our package(s) you are using. You'll find our address on [our contact page](https://spatie.be/about-us). We publish all received postcards on [our virtual postcard wall](https://spatie.be/open-source/postcards). +- **Provider Agnostic**: Easily swap between OpenAI, Anthropic, local models, and vector databases +- **Production Ready**: Batch operations, duplicate-safe inserts, structured logging with trace IDs +- **Type Safe**: Full strict typing with PHPStan level 6+ compatibility +- **Modular Design**: Clean interfaces for extensibility and testing +- **Soft Delete Support**: Preserve data integrity with soft deletes +- **Idempotent Ingestion**: Handle duplicate chunks gracefully ## Installation -You can install the package via composer: +```bash +composer require thaolaptrinh/laravel-rag +``` + +Publish the configuration: ```bash -composer require :vendor_slug/:package_slug +php artisan vendor:publish --tag="rag-config" ``` -You can publish and run the migrations with: +Publish and run migrations: ```bash -php artisan vendor:publish --tag=":package_slug-migrations" +php artisan vendor:publish --tag="rag-migrations" php artisan migrate ``` -You can publish the config file with: +## Configuration -```bash -php artisan vendor:publish --tag=":package_slug-config" +Add your OpenAI API key to `.env`: + +```env +OPENAI_API_KEY=sk-your-api-key-here ``` -This is the contents of the published config file: +The package uses sensible defaults, but you can customize via `config/rag.php`: ```php return [ + 'embedding' => [ + 'provider' => 'openai', + 'model' => 'text-embedding-3-small', + 'dimension' => 1536, + ], + 'llm' => [ + 'provider' => 'openai', + 'model' => 'gpt-4o-mini', + 'max_tokens' => 4096, + 'temperature' => 0.7, + ], + 'vector_store' => [ + 'provider' => 'pgvector', + 'table' => 'rag_chunks', + ], + // ... more options ]; ``` -Optionally, you can publish the views using +## Usage + +### Ingest Data ```bash -php artisan vendor:publish --tag=":package_slug-views" +php artisan rag:ingest /path/to/document.txt ``` -## Usage +Or programmatically: ```php -$:variable = new VendorName\Skeleton(); -echo $:variable->echoPhrase('Hello, VendorName!'); +use Thaolaptrinh\Rag\Facades\Rag; + +$result = Rag::ingest('/path/to/document.txt'); +// Returns: ['stored' => 15, 'errors' => 0, 'source' => '/path/to/document.txt'] ``` -## Testing +### Query the System ```bash -composer test +php artisan rag:query "What is the main topic of the document?" ``` -## Changelog +Or programmatically: -Please see [CHANGELOG](CHANGELOG.md) for more information on what has changed recently. +```php +use Thaolaptrinh\Rag\Facades\Rag; -## Contributing +$result = Rag::query('What is the main topic?'); +// Returns: ['answer' => '...', 'chunks' => 5, 'query' => '...'] +``` -Please see [CONTRIBUTING](CONTRIBUTING.md) for details. +## Architecture -## Security Vulnerabilities +### Core Abstractions -Please review [our security policy](../../security/policy) on how to report security vulnerabilities. +The package is built around 7 core interfaces: -## Credits +- **DataSource**: Load data from any source (files, APIs, databases) +- **Chunker**: Split content into manageable chunks +- **EmbeddingDriver**: Generate vector embeddings (OpenAI, Cohere, local) +- **VectorStore**: Store and search vectors (pgvector, Pinecone, Weaviate) +- **Retriever**: Retrieve relevant chunks (similarity, hybrid, reranking) +- **PromptBuilder**: Build prompts for LLMs with token limits +- **LlmDriver**: Generate responses (OpenAI, Anthropic, local models) -- [:author_name](https://github.com/:author_username) -- [All Contributors](../../contributors) +### Pipeline Flow + +**Ingestion**: +``` +DataSource → Chunker → EmbeddingDriver (batch) → VectorStore (batch) +``` + +**Query**: +``` +Query → Retriever → PromptBuilder → LlmDriver → Response +``` + +## Code Quality Standards + +All code follows strict production standards: + +- ✅ `declare(strict_types=1);` in every file +- ✅ Explicit parameter and return types on all methods +- ✅ `readonly` properties for constructor dependencies +- ✅ PHPStan level 6+ compatible +- ✅ Batch operations (embedBatch, storeMany) for performance +- ✅ Structured logging with trace IDs +- ✅ Duplicate-safe database operations +- ✅ Custom exceptions for error handling + +## Extending the Package + +### Custom Data Source + +```php +use Thaolaptrinh\Rag\Contracts\DataSource; + +class PdfDataSource implements DataSource +{ + public function load(string $source): array + { + // Your PDF loading logic + return [ + [ + 'id' => Str::uuid()->toString(), + 'content' => $pdfText, + 'metadata' => ['source' => $source, 'type' => 'pdf'] + ] + ]; + } +} +``` + +### Custom Vector Store + +```php +use Thaolaptrinh\Rag\Contracts\VectorStore; + +class PineconeVectorStore implements VectorStore +{ + public function storeMany(array $items): void + { + // Your Pinecone implementation + } + + // ... implement other methods +} +``` + +## Testing + +```bash +composer test +``` + +Run static analysis: + +```bash +composer analyse +``` + +## Requirements + +- PHP 8.2 or higher +- Laravel 10.x or 11.x +- PostgreSQL with pgvector extension (for vector storage) +- OpenAI API key (or compatible service) ## License The MIT License (MIT). Please see [License File](LICENSE.md) for more information. + +## Credits + +- [Thao Nguyen](https://github.com/thaolaptrinh) +- [All Contributors](../../contributors) diff --git a/composer.json b/composer.json index 5f8a049..2b41d84 100644 --- a/composer.json +++ b/composer.json @@ -1,47 +1,51 @@ { - "name": ":vendor_slug/:package_slug", - "description": ":package_description", + "name": "thaolaptrinh/laravel-rag", + "description": "Provider-agnostic RAG (Retrieval-Augmented Generation) package for Laravel", "keywords": [ - ":vendor_name", + "thaolaptrinh", "laravel", - ":package_slug" + "rag", + "retrieval-augmented-generation", + "vector-search", + "embeddings" ], - "homepage": "https://github.com/:vendor_slug/:package_slug", + "homepage": "https://github.com/thaolaptrinh/laravel-rag", "license": "MIT", "authors": [ { - "name": ":author_name", - "email": "author@domain.com", + "name": "Thao Nguyen", + "email": "dev@thaolaptrinh.com", "role": "Developer" } ], "require": { - "php": "^8.4", + "php": "^8.2", "spatie/laravel-package-tools": "^1.16", - "illuminate/contracts": "^11.0||^12.0" + "illuminate/contracts": "^10.0||^11.0" }, "require-dev": { - "laravel/pint": "^1.14", - "nunomaduro/collision": "^8.8", - "larastan/larastan": "^3.0", - "orchestra/testbench": "^10.0.0||^9.0.0", - "pestphp/pest": "^4.0", - "pestphp/pest-plugin-arch": "^4.0", - "pestphp/pest-plugin-laravel": "^4.0", - "phpstan/extension-installer": "^1.4", - "phpstan/phpstan-deprecation-rules": "^2.0", - "phpstan/phpstan-phpunit": "^2.0", + "larastan/larastan": "^2.0", + "laravel/pint": "^1.13", + "nunomaduro/collision": "^8.1", + "orchestra/testbench": "^8.0||^9.0", + "pestphp/pest": "^2.0", + "pestphp/pest-plugin-arch": "^2.0", + "pestphp/pest-plugin-laravel": "^2.0", + "phpstan/extension-installer": "^1.3", + "phpstan/phpstan-deprecation-rules": "^1.1", + "phpstan/phpstan-phpunit": "^1.3", + "rector/rector": "^1.2", "spatie/laravel-ray": "^1.35" }, "autoload": { "psr-4": { - "VendorName\\Skeleton\\": "src/", - "VendorName\\Skeleton\\Database\\Factories\\": "database/factories/" + "Thaolaptrinh\\Rag\\": "src/", + "Thaolaptrinh\\Rag\\Database\\Factories\\": "database/factories/" } }, "autoload-dev": { "psr-4": { - "VendorName\\Skeleton\\Tests\\": "tests/", + "Thaolaptrinh\\Rag\\Tests\\": "tests/", "Workbench\\App\\": "workbench/app/" } }, @@ -51,7 +55,9 @@ "analyse": "vendor/bin/phpstan analyse", "test": "vendor/bin/pest", "test-coverage": "vendor/bin/pest --coverage", - "format": "vendor/bin/pint" + "format": "vendor/bin/pint", + "refactor": "vendor/bin/rector process", + "refactor-dry": "vendor/bin/rector process --dry-run" }, "config": { "sort-packages": true, @@ -63,10 +69,10 @@ "extra": { "laravel": { "providers": [ - "VendorName\\Skeleton\\SkeletonServiceProvider" + "Thaolaptrinh\\Rag\\RagServiceProvider" ], "aliases": { - "Skeleton": "VendorName\\Skeleton\\Facades\\Skeleton" + "Rag": "Thaolaptrinh\\Rag\\Facades\\Rag" } } }, diff --git a/config/rag.php b/config/rag.php new file mode 100644 index 0000000..f8e8c48 --- /dev/null +++ b/config/rag.php @@ -0,0 +1,49 @@ + [ + 'type' => env('RAG_DATA_SOURCE', 'text'), + ], + + 'chunker' => [ + 'max_chunk_size' => (int) env('RAG_MAX_CHUNK_SIZE', 1000), + 'overlap' => (int) env('RAG_CHUNK_OVERLAP', 200), + ], + + 'embedding' => [ + 'provider' => env('RAG_EMBEDDING_PROVIDER', 'openai'), + 'api_key' => env('RAG_OPENAI_API_KEY') ?: env('OPENAI_API_KEY'), + 'model' => env('RAG_EMBEDDING_MODEL', 'text-embedding-3-small'), + 'dimension' => (int) env('RAG_EMBEDDING_DIMENSION', 1536), + 'api_url' => env('RAG_EMBEDDING_API_URL'), + ], + + 'vector_store' => [ + 'provider' => env('RAG_VECTOR_STORE', 'pgvector'), + 'table' => env('RAG_VECTOR_TABLE', 'rag_chunks'), + ], + + 'retriever' => [ + 'type' => env('RAG_RETRIEVER_TYPE', 'similarity'), + 'table' => env('RAG_VECTOR_TABLE', 'rag_chunks'), + ], + + 'prompt' => [ + 'system' => env( + 'RAG_PROMPT_SYSTEM', + 'You are a helpful assistant. Answer the question based on the provided context.' + ), + 'average_tokens_per_word' => (int) env('RAG_AVG_TOKENS_PER_WORD', 4), + ], + + 'llm' => [ + 'provider' => env('RAG_LLM_PROVIDER', 'openai'), + 'api_key' => env('RAG_OPENAI_API_KEY') ?: env('OPENAI_API_KEY'), + 'model' => env('RAG_LLM_MODEL', 'gpt-4o-mini'), + 'max_tokens' => (int) env('RAG_LLM_MAX_TOKENS', 4096), + 'temperature' => (float) env('RAG_LLM_TEMPERATURE', 0.7), + 'api_url' => env('RAG_LLM_API_URL'), + ], +]; diff --git a/database/migrations/2026_03_26_221904_create_rag_chunks_table.php b/database/migrations/2026_03_26_221904_create_rag_chunks_table.php new file mode 100644 index 0000000..5f81ac8 --- /dev/null +++ b/database/migrations/2026_03_26_221904_create_rag_chunks_table.php @@ -0,0 +1,37 @@ +uuid('id')->primary(); + $table->text('content'); + $table->json('embedding'); + $table->json('metadata')->nullable(); + $table->unsignedInteger('chunk_index')->default(0); + $table->string('source', 500)->nullable(); + $table->string('type', 100)->default('text'); + $table->timestamp('created_at')->useCurrent(); + $table->timestamp('updated_at')->useCurrent()->useCurrentOnUpdate(); + $table->timestamp('deleted_at')->nullable(); + + $table->unique(['id', 'deleted_at'], 'unique_chunk_id'); + $table->index(['source', 'deleted_at'], 'idx_source'); + $table->index(['type', 'deleted_at'], 'idx_type'); + }); + + DB::statement('CREATE INDEX rag_chunks_embedding_idx ON rag_chunks USING ivfflat (embedding vector_cosine_ops) WITH (lists = 100)'); + } + + public function down(): void + { + Schema::dropIfExists('rag_chunks'); + } +}; diff --git a/database/migrations/create_skeleton_table.php.stub b/database/migrations/create_skeleton_table.php.stub deleted file mode 100644 index 2efdce9..0000000 --- a/database/migrations/create_skeleton_table.php.stub +++ /dev/null @@ -1,19 +0,0 @@ -id(); - - // add fields - - $table->timestamps(); - }); - } -}; diff --git a/opencode.json b/opencode.json new file mode 100644 index 0000000..6eb4bcf --- /dev/null +++ b/opencode.json @@ -0,0 +1,94 @@ +{ + "$schema": "https://opencode.ai/config.json", + "model": "zai-coding-plan/glm-5-turbo", + "small_model": "zai-coding-plan/glm-5-turbo", + "default_agent": "laravel-expert", + "instructions": ["./AGENTS.md"], + "formatter": { + "pint": { + "command": ["vendor/bin/pint", "$FILE"], + "extensions": [".php"] + } + }, + "compaction": { + "auto": true + }, + "agent": { + "build": { + "model": "zai-coding-plan/glm-5-turbo" + }, + "plan": { + "model": "zai-coding-plan/glm-5-turbo" + } + }, + "command": { + "analyse": { + "description": "Run PHPStan static analysis and fix errors", + "agent": "laravel-expert", + "template": "Run PHPStan: !`composer analyse 2>&1`. If there are errors, analyze each one and fix immediately. Re-run to confirm clean. Never commit with PHPStan errors." + }, + "lint": { + "description": "Format all code with Laravel Pint", + "agent": "laravel-expert", + "template": "Format code: !`composer format 2>&1`. Then check !`git diff --stat 2>&1` to see which files changed." + }, + "test": { + "description": "Run Pest test suite", + "agent": "laravel-expert", + "template": "Run tests: !`composer test 2>&1`. If there are failures, analyze each one, identify the root cause, and propose a specific fix. Fix one test at a time." + }, + "test-coverage": { + "description": "Run tests with coverage report", + "agent": "laravel-expert", + "template": "Run coverage: !`composer test-coverage 2>&1`. Analyze the report, identify untested code paths, and suggest additional tests for critical areas (Contracts, Services, Drivers)." + }, + "review": { + "description": "Review code changes before committing", + "agent": "code-reviewer", + "subtask": true, + "template": "Review all changes: !`git diff HEAD 2>&1`. Check against AGENTS.md: (1) declare(strict_types=1) on every PHP file, (2) explicit return types, (3) no provider code in Services, (4) batch operations used, (5) PHPStan compatible, (6) trace_id in logs. List all issues by severity: Critical / Major / Minor." + }, + "commit": { + "description": "Suggest a conventional commit message", + "agent": "laravel-expert", + "template": "Show staged changes: !`git diff --staged 2>&1`. Suggest a commit message following Conventional Commits (feat/fix/refactor/test/docs/chore). Only suggest — do not commit. Format: `type(scope): description`" + }, + "refactor": { + "description": "Run Rector auto-refactor (dry-run first)", + "agent": "laravel-expert", + "template": "Run Rector dry-run to preview changes: !`composer refactor-dry 2>&1`. Review each proposed change. If all look correct, apply: !`composer refactor 2>&1`. Then validate: !`composer analyse 2>&1` and !`composer test 2>&1` to ensure no regressions." + }, + "ci": { + "description": "Run full CI quality suite: test + analyse + review", + "agent": "laravel-expert", + "template": "Run the full quality suite in order:\n1. Tests: !`composer test 2>&1`\n2. PHPStan: !`composer analyse 2>&1`\n3. Check !`git diff HEAD 2>&1` for any uncommitted quality issues.\nReport a summary: how many tests passed, any PHPStan errors, any architecture violations. The build passes only when all three are clean." + }, + "debug": { + "description": "Debug a pipeline issue systematically", + "agent": "debugger", + "subtask": true, + "template": "Debug issue: $ARGUMENTS. Follow the process: reproduce → isolate → inspect logs → hypothesize root cause → verify → report with a specific fix recommendation." + }, + "phase-1-foundation": { + "description": "Setup Phase 1 - Core abstractions and ingestion foundation", + "agent": "laravel-expert", + "template": "Set up Phase 1 of the RAG pipeline: create all core interfaces in src/Contracts/ (DataSource, Chunker, EmbeddingDriver, VectorStore, Retriever, PromptBuilder, LlmDriver), implement TextDataSource and TextChunker, create the rag_chunks migration with pgvector support. Validate with PHPStan after each step." + }, + "phase-2-ingestion": { + "description": "Build Phase 2 - Complete ingestion pipeline", + "agent": "laravel-expert", + "template": "Build Phase 2: implement PgVectorStore with storeMany() batch support, OpenAIEmbeddingDriver with embedBatch(), IngestionPipeline service with structured logging and trace_id, and the rag:ingest Artisan command. Test end-to-end with a real file." + }, + "phase-3-query": { + "description": "Build Phase 3 - Query pipeline with LLM generation", + "agent": "laravel-expert", + "template": "Build Phase 3: implement SimilarityRetriever (accepts string query, handles embedding internally), SimplePromptBuilder with token limits, OpenAILlmDriver, QueryPipeline service, and the rag:query Artisan command. Validate the full query flow end-to-end." + }, + "validate-architecture": { + "description": "Validate full architecture and code quality", + "agent": "code-reviewer", + "subtask": true, + "template": "Validate the full architecture: !`grep -r 'OpenAI\\|Pinecone\\|Anthropic' src/Services/ 2>&1 || echo OK`. !`find src -name '*.php' | xargs grep -L 'declare(strict_types=1)' 2>&1`. !`composer analyse 2>&1`. Report: provider lock-in, missing strict_types, PHPStan errors, batch operations, trace IDs." + } + } +} diff --git a/phpstan.neon.dist b/phpstan.neon.dist index 15765b3..060266e 100644 --- a/phpstan.neon.dist +++ b/phpstan.neon.dist @@ -2,7 +2,7 @@ includes: - phpstan-baseline.neon parameters: - level: 5 + level: 6 paths: - src - config diff --git a/rector.php b/rector.php new file mode 100644 index 0000000..3516997 --- /dev/null +++ b/rector.php @@ -0,0 +1,34 @@ +withPaths([ + __DIR__.'/src', + __DIR__.'/tests', + ]) + ->withSkip([ + __DIR__.'/vendor', + __DIR__.'/build', + ]) + ->withPhpSets(php82: true) + ->withSets([ + SetList::CODE_QUALITY, + SetList::DEAD_CODE, + SetList::TYPE_DECLARATION, + SetList::EARLY_RETURN, + ]) + ->withRules([ + AddVoidReturnTypeWhereNoReturnRector::class, + ReturnTypeFromReturnNewRector::class, + ReturnTypeFromStrictTypedPropertyRector::class, + TypedPropertyFromStrictConstructorRector::class, + ]); diff --git a/src/Commands/IngestCommand.php b/src/Commands/IngestCommand.php new file mode 100644 index 0000000..7344dee --- /dev/null +++ b/src/Commands/IngestCommand.php @@ -0,0 +1,48 @@ +argument('source'); + + $this->info("Starting ingestion from: {$source}"); + + try { + $result = $this->pipeline->ingest($source); + + if ($result['errors'] > 0) { + $this->error("Ingestion failed for source: {$source}"); + + return self::FAILURE; + } + + $this->info('✓ Ingestion completed successfully'); + $this->info(" Stored: {$result['stored']} chunks"); + $this->info(" Source: {$result['source']}"); + + return self::SUCCESS; + } catch (\Exception $e) { + $this->error("Ingestion error: {$e->getMessage()}"); + + return self::FAILURE; + } + } +} diff --git a/src/Commands/QueryCommand.php b/src/Commands/QueryCommand.php new file mode 100644 index 0000000..516b7c8 --- /dev/null +++ b/src/Commands/QueryCommand.php @@ -0,0 +1,54 @@ +argument('query'); + $topK = (int) $this->option('topK'); + + if ($topK < 1) { + $this->error('--topK must be at least 1'); + + return self::FAILURE; + } + + $this->info("Query: {$query}"); + $this->newLine(); + + try { + $result = $this->pipeline->query($query, $topK); + + $this->info('Answer:'); + $this->line($result['answer']); + $this->newLine(); + $this->info("Retrieved {$result['chunks']} chunks"); + + return self::SUCCESS; + } catch (\Exception $e) { + $this->error("Query failed: {$e->getMessage()}"); + + return self::FAILURE; + } + } +} diff --git a/src/Commands/SkeletonCommand.php b/src/Commands/SkeletonCommand.php deleted file mode 100644 index 3e5f628..0000000 --- a/src/Commands/SkeletonCommand.php +++ /dev/null @@ -1,19 +0,0 @@ -comment('All done'); - - return self::SUCCESS; - } -} diff --git a/src/Contracts/Chunker.php b/src/Contracts/Chunker.php new file mode 100644 index 0000000..b617e53 --- /dev/null +++ b/src/Contracts/Chunker.php @@ -0,0 +1,30 @@ +} $document + * @return array, index: int}> + */ + public function chunk(array $document): array; + + /** + * Get the maximum chunk size + * + * @return int<1, max> + */ + public function getMaxChunkSize(): int; + + /** + * Get the chunk overlap size + * + * @return int<0, max> + */ + public function getOverlap(): int; +} diff --git a/src/Contracts/DataSource.php b/src/Contracts/DataSource.php new file mode 100644 index 0000000..8951fca --- /dev/null +++ b/src/Contracts/DataSource.php @@ -0,0 +1,16 @@ +}> + */ + public function load(string $source): array; +} diff --git a/src/Contracts/EmbeddingDriver.php b/src/Contracts/EmbeddingDriver.php new file mode 100644 index 0000000..8ec3236 --- /dev/null +++ b/src/Contracts/EmbeddingDriver.php @@ -0,0 +1,31 @@ + Embedding vector + */ + public function embed(string $text): array; + + /** + * Generate embeddings for multiple texts (batch operation) + * + * @param array $texts Texts to embed + * @return array> Array of embedding vectors + */ + public function embedBatch(array $texts): array; + + /** + * Get the embedding dimension + * + * @return int<1, max> + */ + public function getDimension(): int; +} diff --git a/src/Contracts/LlmDriver.php b/src/Contracts/LlmDriver.php new file mode 100644 index 0000000..750d535 --- /dev/null +++ b/src/Contracts/LlmDriver.php @@ -0,0 +1,41 @@ + Maximum tokens + */ + public function getMaxTokens(): int; +} diff --git a/src/Contracts/PromptBuilder.php b/src/Contracts/PromptBuilder.php new file mode 100644 index 0000000..8f82013 --- /dev/null +++ b/src/Contracts/PromptBuilder.php @@ -0,0 +1,37 @@ +}> $context Retrieved chunks + * @param string $query User query + * @param int<1, max> $maxTokens Maximum tokens for the prompt + * @return string Built prompt + */ + public function build(array $context, string $query, int $maxTokens): string; + + /** + * Build a prompt with system instructions + * + * @param string $system System instructions + * @param array}> $context Retrieved chunks + * @param string $query User query + * @param int<1, max> $maxTokens Maximum tokens for the prompt + * @return string Built prompt + */ + public function buildWithSystem(string $system, array $context, string $query, int $maxTokens): string; + + /** + * Estimate token count for a string + * + * @param string $text Text to estimate + * @return int<0, max> Estimated token count + */ + public function estimateTokens(string $text): int; +} diff --git a/src/Contracts/Retriever.php b/src/Contracts/Retriever.php new file mode 100644 index 0000000..5743981 --- /dev/null +++ b/src/Contracts/Retriever.php @@ -0,0 +1,18 @@ + $topK Number of chunks to retrieve + * @param array $filters Optional filters for retrieval + * @return array}> + */ + public function retrieve(string $query, int $topK, array $filters = []): array; +} diff --git a/src/Contracts/VectorStore.php b/src/Contracts/VectorStore.php new file mode 100644 index 0000000..dc34ec7 --- /dev/null +++ b/src/Contracts/VectorStore.php @@ -0,0 +1,37 @@ +} $chunk + * @param array $embedding Embedding vector + */ + public function store(array $chunk, array $embedding): void; + + /** + * Store multiple chunks with their embeddings (batch operation) + * + * @param array}, embedding: array}> $items + */ + public function storeMany(array $items): void; + + /** + * Delete a chunk by its ID + * + * @param string $id Chunk ID + */ + public function delete(string $id): void; + + /** + * Delete multiple chunks by their IDs + * + * @param array $ids Chunk IDs + */ + public function deleteMany(array $ids): void; +} diff --git a/src/Drivers/DataSource/TextDataSource.php b/src/Drivers/DataSource/TextDataSource.php new file mode 100644 index 0000000..29a6b6f --- /dev/null +++ b/src/Drivers/DataSource/TextDataSource.php @@ -0,0 +1,46 @@ +}> + */ + public function load(string $source): array + { + if (!file_exists($source)) { + throw new \RuntimeException("File not found: {$source}"); + } + + if (!is_readable($source)) { + throw new \RuntimeException("File not readable: {$source}"); + } + + $content = file_get_contents($source); + + if ($content === false) { + throw new \RuntimeException("Failed to read file: {$source}"); + } + + return [ + [ + 'id' => Str::uuid()->toString(), + 'content' => $content, + 'metadata' => [ + 'source' => $source, + 'type' => 'text', + 'size' => strlen($content), + ], + ], + ]; + } +} diff --git a/src/Drivers/Embeddings/OpenAIEmbeddingDriver.php b/src/Drivers/Embeddings/OpenAIEmbeddingDriver.php new file mode 100644 index 0000000..1e3f410 --- /dev/null +++ b/src/Drivers/Embeddings/OpenAIEmbeddingDriver.php @@ -0,0 +1,92 @@ +apiKey = $apiKey; + $this->model = $model; + $this->dimension = $dimension; + $this->apiUrl = $apiUrl ?? 'https://api.openai.com/v1/embeddings'; + } + + public function embed(string $text): array + { + $embeddings = $this->embedBatch([$text]); + + return $embeddings[0] ?? throw new EmbeddingFailedException('No embedding returned'); + } + + public function embedBatch(array $texts): array + { + if (empty($texts)) { + return []; + } + + if ($this->apiKey === '') { + throw new EmbeddingFailedException('OpenAI API key is required'); + } + + try { + $response = Http::withToken($this->apiKey) + ->acceptJson() + ->post($this->apiUrl, [ + 'model' => $this->model, + 'input' => $texts, + 'encoding_format' => 'float', + ]); + + if (! $response->successful()) { + throw new EmbeddingFailedException( + "API request failed: {$response->status()} {$response->body()}" + ); + } + + $data = $response->json(); + + if (! is_array($data) || ! isset($data['data'])) { + throw new EmbeddingFailedException('Invalid response structure'); + } + + $embeddings = []; + foreach ($data['data'] as $item) { + if (! isset($item['embedding']) || ! is_array($item['embedding'])) { + throw new EmbeddingFailedException('Invalid embedding structure'); + } + $embeddings[] = $item['embedding']; + } + + return $embeddings; + } catch (\Exception $e) { + if ($e instanceof EmbeddingFailedException) { + throw $e; + } + throw new EmbeddingFailedException($e->getMessage(), $e->getCode(), $e); + } + } + + public function getDimension(): int + { + return $this->dimension; + } +} diff --git a/src/Drivers/Llm/OpenAILlmDriver.php b/src/Drivers/Llm/OpenAILlmDriver.php new file mode 100644 index 0000000..9774f9f --- /dev/null +++ b/src/Drivers/Llm/OpenAILlmDriver.php @@ -0,0 +1,140 @@ +apiKey = $apiKey; + $this->model = $model; + $this->maxTokens = $maxTokens; + $this->temperature = $temperature; + $this->apiUrl = $apiUrl ?? 'https://api.openai.com/v1/chat/completions'; + } + + public function generate(string $prompt): string + { + return $this->generateWithSystem('You are a helpful assistant.', $prompt); + } + + public function generateWithSystem(string $system, string $prompt): string + { + if ($this->apiKey === '') { + throw new GenerationFailedException('OpenAI API key is required'); + } + + try { + $response = Http::withToken($this->apiKey) + ->acceptJson() + ->post($this->apiUrl, [ + 'model' => $this->model, + 'messages' => [ + ['role' => 'system', 'content' => $system], + ['role' => 'user', 'content' => $prompt], + ], + 'temperature' => $this->temperature, + 'max_tokens' => $this->maxTokens, + ]); + + if (! $response->successful()) { + throw new GenerationFailedException( + "API request failed: {$response->status()} {$response->body()}" + ); + } + + $data = $response->json(); + + if (! is_array($data) || ! isset($data['choices'][0]['message']['content'])) { + throw new GenerationFailedException('Invalid response structure'); + } + + return $data['choices'][0]['message']['content']; + } catch (\Exception $e) { + if ($e instanceof GenerationFailedException) { + throw $e; + } + throw new GenerationFailedException($e->getMessage(), $e->getCode(), $e); + } + } + + public function generateStream(string $prompt, callable $callback): string + { + $fullResponse = ''; + + try { + $response = Http::withToken($this->apiKey) + ->acceptJson() + ->withOptions([ + 'stream' => true, + 'stream_callback' => function ($chunk) use ($callback, &$fullResponse) { + if (! empty($chunk)) { + $lines = explode("\n", $chunk); + foreach ($lines as $line) { + if (strpos($line, 'data: ') === 0) { + $data = substr($line, 6); + if ($data === '[DONE]') { + return; + } + $json = json_decode($data, true); + if (isset($json['choices'][0]['delta']['content'])) { + $content = $json['choices'][0]['delta']['content']; + $fullResponse .= $content; + $callback($content); + } + } + } + } + }, + ]) + ->post($this->apiUrl, [ + 'model' => $this->model, + 'messages' => [ + ['role' => 'user', 'content' => $prompt], + ], + 'temperature' => $this->temperature, + 'max_tokens' => $this->maxTokens, + 'stream' => true, + ]); + + if (! $response->successful()) { + throw new GenerationFailedException( + "API request failed: {$response->status()}" + ); + } + + return $fullResponse; + } catch (\Exception $e) { + if ($e instanceof GenerationFailedException) { + throw $e; + } + throw new GenerationFailedException($e->getMessage(), $e->getCode(), $e); + } + } + + public function getMaxTokens(): int + { + return $this->maxTokens; + } +} diff --git a/src/Drivers/VectorStores/PgVectorStore.php b/src/Drivers/VectorStores/PgVectorStore.php new file mode 100644 index 0000000..d670ed8 --- /dev/null +++ b/src/Drivers/VectorStores/PgVectorStore.php @@ -0,0 +1,123 @@ +table = $table; + } + + public function store(array $chunk, array $embedding): void + { + $this->storeMany([[ + 'chunk' => $chunk, + 'embedding' => $embedding, + ]]); + } + + public function storeMany(array $items): void + { + if (empty($items)) { + return; + } + + try { + DB::beginTransaction(); + + foreach ($items as $item) { + $chunk = $item['chunk']; + $embedding = $item['embedding']; + + $metadata = $chunk['metadata'] ?? []; + $source = $metadata['source'] ?? null; + $type = $metadata['type'] ?? 'text'; + $chunkIndex = $metadata['chunk_index'] ?? 0; + + try { + DB::table($this->table)->insert([ + 'id' => $chunk['id'], + 'content' => $chunk['content'], + 'embedding' => json_encode($embedding, JSON_THROW_ON_ERROR), + 'metadata' => json_encode($metadata, JSON_THROW_ON_ERROR), + 'chunk_index' => $chunkIndex, + 'source' => $source, + 'type' => $type, + 'created_at' => now(), + 'updated_at' => now(), + ]); + } catch (QueryException $e) { + $errorCode = $e->errorInfo[1] ?? null; + + if ($errorCode === 23505) { + continue; + } + + throw new StorageFailedException( + "Failed to store chunk {$chunk['id']}: ".$e->getMessage(), + $e->getCode(), + $e + ); + } + } + + DB::commit(); + } catch (\Exception $e) { + DB::rollBack(); + + if ($e instanceof StorageFailedException) { + throw $e; + } + + throw new StorageFailedException( + 'Batch store failed: '.$e->getMessage(), + $e->getCode(), + $e + ); + } + } + + public function delete(string $id): void + { + try { + DB::table($this->table) + ->where('id', $id) + ->update(['deleted_at' => now()]); + } catch (\Exception $e) { + throw new StorageFailedException( + "Failed to delete chunk {$id}: ".$e->getMessage(), + $e->getCode(), + $e + ); + } + } + + public function deleteMany(array $ids): void + { + if (empty($ids)) { + return; + } + + try { + DB::table($this->table) + ->whereIn('id', $ids) + ->update(['deleted_at' => now()]); + } catch (\Exception $e) { + throw new StorageFailedException( + 'Failed to delete chunks: '.$e->getMessage(), + $e->getCode(), + $e + ); + } + } +} diff --git a/src/Exceptions/EmbeddingFailedException.php b/src/Exceptions/EmbeddingFailedException.php new file mode 100644 index 0000000..b925f38 --- /dev/null +++ b/src/Exceptions/EmbeddingFailedException.php @@ -0,0 +1,13 @@ +ingestionPipeline->ingest($source); + } + + /** + * Query the RAG system + * + * @param string $query User query + * @param int<1, max> $topK Number of chunks to retrieve + * @param array $filters Optional filters for retrieval + * @return array{answer: string, chunks: int, query: string} + */ + public function query(string $query, int $topK = 5, array $filters = []): array + { + return $this->queryPipeline->query($query, $topK, $filters); + } +} diff --git a/src/RagServiceProvider.php b/src/RagServiceProvider.php new file mode 100644 index 0000000..6cbabc7 --- /dev/null +++ b/src/RagServiceProvider.php @@ -0,0 +1,167 @@ +mergeConfigFrom( + __DIR__.'/../config/rag.php', + 'rag' + ); + + $this->app->bind(DataSource::class, function () { + $config = config('rag.data_source', []); + $type = is_array($config) ? ($config['type'] ?? 'text') : (string) $config; + + return match ($type) { + 'text' => new TextDataSource, + default => throw new \InvalidArgumentException("Unsupported data source: {$type}"), + }; + }); + + $this->app->bind(Chunker::class, function () { + $config = config('rag.chunker', []); + $maxSize = $config['max_chunk_size'] ?? 1000; + $overlap = $config['overlap'] ?? 200; + + return new TextChunker($maxSize, $overlap); + }); + + $this->app->bind(EmbeddingDriver::class, function () { + $config = config('rag.embedding', []); + $provider = is_array($config) ? ($config['provider'] ?? 'openai') : 'openai'; + + return match ($provider) { + 'openai' => new OpenAIEmbeddingDriver( + apiKey: is_array($config) ? ($config['api_key'] ?? '') : '', + model: is_array($config) ? ($config['model'] ?? 'text-embedding-3-small') : 'text-embedding-3-small', + dimension: is_array($config) ? ($config['dimension'] ?? 1536) : 1536, + apiUrl: is_array($config) ? ($config['api_url'] ?? null) : null + ), + default => throw new \InvalidArgumentException("Unsupported embedding provider: {$provider}"), + }; + }); + + $this->app->bind(VectorStore::class, function () { + $config = config('rag.vector_store', []); + $provider = $config['provider'] ?? 'pgvector'; + + return match ($provider) { + 'pgvector' => new PgVectorStore( + table: $config['table'] ?? 'rag_chunks' + ), + default => throw new \InvalidArgumentException("Unsupported vector store: {$provider}"), + }; + }); + + $this->app->bind(LlmDriver::class, function () { + $config = config('rag.llm', []); + $provider = is_array($config) ? ($config['provider'] ?? 'openai') : 'openai'; + + return match ($provider) { + 'openai' => new OpenAILlmDriver( + apiKey: is_array($config) ? ($config['api_key'] ?? '') : '', + model: is_array($config) ? ($config['model'] ?? 'gpt-4o-mini') : 'gpt-4o-mini', + maxTokens: is_array($config) ? ($config['max_tokens'] ?? 4096) : 4096, + temperature: is_array($config) ? ($config['temperature'] ?? 0.7) : 0.7, + apiUrl: is_array($config) ? ($config['api_url'] ?? null) : null + ), + default => throw new \InvalidArgumentException("Unsupported LLM provider: {$provider}"), + }; + }); + + $this->app->bind(PromptBuilder::class, function () { + $config = config('rag.prompt', []); + $system = $config['system'] ?? 'You are a helpful assistant. Answer the question based on the provided context.'; + $averageTokensPerWord = $config['average_tokens_per_word'] ?? 4; + + return new SimplePromptBuilder($system, $averageTokensPerWord); + }); + + $this->app->bind(Retriever::class, function ($app) { + $config = config('rag.retriever', []); + $type = $config['type'] ?? 'similarity'; + + return match ($type) { + 'similarity' => new SimilarityRetriever( + $app->make(EmbeddingDriver::class), + $config['table'] ?? 'rag_chunks' + ), + default => throw new \InvalidArgumentException("Unsupported retriever type: {$type}"), + }; + }); + + $this->app->singleton(RagLogger::class, function () { + return new RagLogger; + }); + + $this->app->singleton(IngestionPipeline::class, function ($app) { + return new IngestionPipeline( + $app->make(DataSource::class), + $app->make(Chunker::class), + $app->make(EmbeddingDriver::class), + $app->make(VectorStore::class), + $app->make(RagLogger::class) + ); + }); + + $this->app->singleton(QueryPipeline::class, function ($app) { + return new QueryPipeline( + $app->make(Retriever::class), + $app->make(PromptBuilder::class), + $app->make(LlmDriver::class), + $app->make(RagLogger::class) + ); + }); + + $this->app->singleton('rag', function ($app) { + return new RagManager( + $app->make(IngestionPipeline::class), + $app->make(QueryPipeline::class) + ); + }); + } + + public function boot(): void + { + if ($this->app->runningInConsole()) { + $this->publishes([ + __DIR__.'/../config/rag.php' => config_path('rag.php'), + ], 'rag-config'); + + $this->publishes([ + __DIR__.'/../database/migrations/' => database_path('migrations'), + ], 'rag-migrations'); + + $this->commands([ + IngestCommand::class, + QueryCommand::class, + ]); + } + } +} diff --git a/src/Services/IngestionPipeline.php b/src/Services/IngestionPipeline.php new file mode 100644 index 0000000..b6e2207 --- /dev/null +++ b/src/Services/IngestionPipeline.php @@ -0,0 +1,99 @@ +logger->ingestionStart($source); + + try { + $documents = $this->dataSource->load($source); + + if (empty($documents)) { + $this->logger->ingestionComplete($source, 0); + + return [ + 'stored' => 0, + 'errors' => 0, + 'source' => $source, + ]; + } + + $allChunks = []; + foreach ($documents as $document) { + $chunks = $this->chunker->chunk($document); + $allChunks = array_merge($allChunks, $chunks); + } + + if (empty($allChunks)) { + $this->logger->ingestionComplete($source, 0); + + return [ + 'stored' => 0, + 'errors' => 0, + 'source' => $source, + ]; + } + + $texts = array_map(fn ($chunk) => $chunk['content'], $allChunks); + $this->logger->embeddingBatch(count($texts)); + + $embeddings = $this->embedder->embedBatch($texts); + + if (count($embeddings) !== count($allChunks)) { + throw new \RuntimeException('Embedding count mismatch'); + } + + $items = []; + foreach ($allChunks as $index => $chunk) { + $items[] = [ + 'chunk' => $chunk, + 'embedding' => $embeddings[$index], + ]; + } + + $this->logger->storeBatch(count($items)); + $this->vectorStore->storeMany($items); + + $this->logger->ingestionComplete($source, count($items)); + + return [ + 'stored' => count($items), + 'errors' => 0, + 'source' => $source, + ]; + } catch (\Exception $e) { + $this->logger->ingestionError($source, $e->getMessage()); + + return [ + 'stored' => 0, + 'errors' => 1, + 'source' => $source, + ]; + } + } +} diff --git a/src/Services/Logging/RagLogger.php b/src/Services/Logging/RagLogger.php new file mode 100644 index 0000000..a5ffdb6 --- /dev/null +++ b/src/Services/Logging/RagLogger.php @@ -0,0 +1,99 @@ +traceId = $traceId ?? (string) Str::uuid(); + } + + public function getTraceId(): string + { + return $this->traceId; + } + + public function ingestionStart(string $source): void + { + Log::info('RAG ingestion started', [ + 'trace_id' => $this->traceId, + 'pipeline_stage' => 'ingestion', + 'source' => $source, + ]); + } + + public function ingestionComplete(string $source, int $stored): void + { + Log::info('RAG ingestion completed', [ + 'trace_id' => $this->traceId, + 'pipeline_stage' => 'ingestion', + 'source' => $source, + 'chunks_stored' => $stored, + ]); + } + + public function ingestionError(string $source, string $error): void + { + Log::error('RAG ingestion failed', [ + 'trace_id' => $this->traceId, + 'pipeline_stage' => 'ingestion', + 'source' => $source, + 'error' => $error, + ]); + } + + public function queryStart(string $query): void + { + Log::info('RAG query started', [ + 'trace_id' => $this->traceId, + 'pipeline_stage' => 'query', + 'query' => $query, + ]); + } + + public function queryComplete(string $query, int $retrieved): void + { + Log::info('RAG query completed', [ + 'trace_id' => $this->traceId, + 'pipeline_stage' => 'query', + 'query' => $query, + 'chunks_retrieved' => $retrieved, + ]); + } + + public function queryError(string $query, string $error): void + { + Log::error('RAG query failed', [ + 'trace_id' => $this->traceId, + 'pipeline_stage' => 'query', + 'query' => $query, + 'error' => $error, + ]); + } + + public function embeddingBatch(int $count): void + { + Log::debug('RAG embedding batch', [ + 'trace_id' => $this->traceId, + 'pipeline_stage' => 'ingestion', + 'batch_size' => $count, + ]); + } + + public function storeBatch(int $count): void + { + Log::debug('RAG store batch', [ + 'trace_id' => $this->traceId, + 'pipeline_stage' => 'ingestion', + 'batch_size' => $count, + ]); + } +} diff --git a/src/Services/QueryPipeline.php b/src/Services/QueryPipeline.php new file mode 100644 index 0000000..59b7c5b --- /dev/null +++ b/src/Services/QueryPipeline.php @@ -0,0 +1,68 @@ + $topK Number of chunks to retrieve + * @param array $filters Optional filters for retrieval + * @return array{answer: string, chunks: int, query: string} + */ + public function query(string $query, int $topK = 5, array $filters = []): array + { + $this->logger->queryStart($query); + + try { + $chunks = $this->retriever->retrieve($query, $topK, $filters); + + if (empty($chunks)) { + $this->logger->queryComplete($query, 0); + + return [ + 'answer' => 'No relevant information found.', + 'chunks' => 0, + 'query' => $query, + ]; + } + + $maxTokens = $this->llm->getMaxTokens(); + $prompt = $this->promptBuilder->build($chunks, $query, $maxTokens); + + $answer = $this->llm->generate($prompt); + + $this->logger->queryComplete($query, count($chunks)); + + return [ + 'answer' => $answer, + 'chunks' => count($chunks), + 'query' => $query, + ]; + } catch (\Exception $e) { + $this->logger->queryError($query, $e->getMessage()); + + return [ + 'answer' => 'Query failed: '.$e->getMessage(), + 'chunks' => 0, + 'query' => $query, + ]; + } + } +} diff --git a/src/Services/Retrievers/SimilarityRetriever.php b/src/Services/Retrievers/SimilarityRetriever.php new file mode 100644 index 0000000..889080b --- /dev/null +++ b/src/Services/Retrievers/SimilarityRetriever.php @@ -0,0 +1,84 @@ +embedder = $embedder; + $this->table = $table; + } + + /** + * Retrieve relevant chunks for a query + * + * @param string $query User query (raw text, not pre-embedded) + * @param int<1, max> $topK Number of chunks to retrieve + * @param array $filters Optional filters for retrieval + * @return array}> + */ + public function retrieve(string $query, int $topK, array $filters = []): array + { + try { + $queryEmbedding = $this->embedder->embed($query); + + $results = DB::table($this->table) + ->select('content', 'metadata') + ->selectRaw( + '1 - (embedding <=> ?::vector) as score', + [$this->vectorToArrayString($queryEmbedding)] + ) + ->whereNull('deleted_at') + ->orderBy('score', 'desc') + ->limit($topK); + + foreach ($filters as $key => $value) { + $results->where($key, $value); + } + + $rows = $results->get(); + + $chunks = []; + foreach ($rows as $row) { + $metadata = json_decode($row->metadata ?? '{}', true, 512, JSON_THROW_ON_ERROR); + $chunks[] = [ + 'content' => $row->content, + 'score' => (float) $row->score, + 'metadata' => $metadata, + ]; + } + + return $chunks; + } catch (\Exception $e) { + throw new RetrievalFailedException( + 'Retrieval failed: '.$e->getMessage(), + $e->getCode(), + $e + ); + } + } + + /** + * Convert embedding vector to PostgreSQL array string + * + * @param array $embedding + */ + private function vectorToArrayString(array $embedding): string + { + return '['.implode(',', $embedding).']'; + } +} diff --git a/src/Services/SimplePromptBuilder.php b/src/Services/SimplePromptBuilder.php new file mode 100644 index 0000000..4d2e9c5 --- /dev/null +++ b/src/Services/SimplePromptBuilder.php @@ -0,0 +1,90 @@ +defaultSystem = $defaultSystem; + $this->averageTokensPerWord = $averageTokensPerWord; + } + + public function build(array $context, string $query, int $maxTokens): string + { + return $this->buildWithSystem($this->defaultSystem, $context, $query, $maxTokens); + } + + public function buildWithSystem(string $system, array $context, string $query, int $maxTokens): string + { + $systemTokens = $this->estimateTokens($system); + $queryTokens = $this->estimateTokens($query); + $availableForContext = $maxTokens - $systemTokens - $queryTokens - 100; + + if ($availableForContext <= 0) { + return $this->buildPrompt($system, '', $query); + } + + $contextText = $this->buildContextText($context, $availableForContext); + + return $this->buildPrompt($system, $contextText, $query); + } + + public function estimateTokens(string $text): int + { + $wordCount = str_word_count($text); + + return (int) ceil($wordCount / $this->averageTokensPerWord); + } + + /** + * Build context text from chunks, respecting token limit + * + * @param array}> $context + * @param int $maxTokens Maximum tokens for context + */ + private function buildContextText(array $context, int $maxTokens): string + { + $chunks = []; + $totalTokens = 0; + + foreach ($context as $item) { + $chunkTokens = $this->estimateTokens($item['content']); + + if ($totalTokens + $chunkTokens > $maxTokens) { + break; + } + + $chunks[] = "[Source: {$item['score']}] {$item['content']}"; + $totalTokens += $chunkTokens; + } + + return implode("\n\n", $chunks); + } + + /** + * Build final prompt + */ + private function buildPrompt(string $system, string $context, string $query): string + { + $prompt = "System: {$system}\n\n"; + + if (! empty($context)) { + $prompt .= "Context:\n{$context}\n\n"; + } + + $prompt .= "Question: {$query}\n\nAnswer:"; + + return $prompt; + } +} diff --git a/src/Services/TextChunker.php b/src/Services/TextChunker.php new file mode 100644 index 0000000..c2389dd --- /dev/null +++ b/src/Services/TextChunker.php @@ -0,0 +1,72 @@ +maxChunkSize = $maxChunkSize; + $this->overlap = $overlap; + + if ($this->overlap >= $this->maxChunkSize) { + throw new \InvalidArgumentException('Overlap must be less than max chunk size'); + } + } + + /** + * Split content into chunks with metadata preservation + * + * @param array{id: string, content: string, metadata: array} $document + * @return array, index: int}> + */ + public function chunk(array $document): array + { + $content = $document['content']; + $chunks = []; + $index = 0; + $position = 0; + $contentLength = strlen($content); + + while ($position < $contentLength) { + $endPosition = min($position + $this->maxChunkSize, $contentLength); + $chunkContent = substr($content, $position, $this->maxChunkSize); + + $chunks[] = [ + 'id' => Str::uuid()->toString(), + 'content' => $chunkContent, + 'metadata' => array_merge($document['metadata'], [ + 'parent_id' => $document['id'], + 'chunk_index' => $index, + 'chunk_start' => $position, + 'chunk_end' => $endPosition, + ]), + 'index' => $index, + ]; + + $position += $this->maxChunkSize - $this->overlap; + $index++; + } + + return $chunks; + } + + public function getMaxChunkSize(): int + { + return $this->maxChunkSize; + } + + public function getOverlap(): int + { + return $this->overlap; + } +} diff --git a/src/Skeleton.php b/src/Skeleton.php deleted file mode 100755 index 34c7194..0000000 --- a/src/Skeleton.php +++ /dev/null @@ -1,5 +0,0 @@ -name('skeleton') - ->hasConfigFile() - ->hasViews() - ->hasMigration('create_migration_table_name_table') - ->hasCommand(SkeletonCommand::class); - } -} diff --git a/tests/ArchTest.php b/tests/ArchTest.php index 87fb64c..6a1feb4 100644 --- a/tests/ArchTest.php +++ b/tests/ArchTest.php @@ -1,5 +1,42 @@ expect(['dd', 'dump', 'ray']) - ->each->not->toBeUsed(); +declare(strict_types=1); + +test('no debug functions used anywhere', function (): void { + expect(['dd', 'dump', 'ray', 'var_dump', 'print_r']) + ->not->toBeUsed(); +}); + +test('contracts namespace contains only interfaces', function (): void { + expect('Thaolaptrinh\Rag\Contracts') + ->toBeInterfaces(); +}); + +test('services do not depend on concrete driver implementations', function (): void { + expect('Thaolaptrinh\Rag\Services') + ->not->toUse([ + 'Thaolaptrinh\Rag\Drivers\Embeddings\OpenAIEmbeddingDriver', + 'Thaolaptrinh\Rag\Drivers\VectorStores\PgVectorStore', + 'Thaolaptrinh\Rag\Drivers\Llm\OpenAILlmDriver', + ]); +}); + +test('drivers implement their corresponding contracts', function (): void { + expect('Thaolaptrinh\Rag\Drivers\Embeddings\OpenAIEmbeddingDriver') + ->toImplement('Thaolaptrinh\Rag\Contracts\EmbeddingDriver'); + + expect('Thaolaptrinh\Rag\Drivers\VectorStores\PgVectorStore') + ->toImplement('Thaolaptrinh\Rag\Contracts\VectorStore'); + + expect('Thaolaptrinh\Rag\Drivers\Llm\OpenAILlmDriver') + ->toImplement('Thaolaptrinh\Rag\Contracts\LlmDriver'); + + expect('Thaolaptrinh\Rag\Drivers\DataSource\TextDataSource') + ->toImplement('Thaolaptrinh\Rag\Contracts\DataSource'); +}); + +test('all exceptions extend RagException', function (): void { + expect('Thaolaptrinh\Rag\Exceptions') + ->toExtend('Thaolaptrinh\Rag\Exceptions\RagException') + ->ignoring('Thaolaptrinh\Rag\Exceptions\RagException'); +}); diff --git a/tests/Integration/PipelineTest.php b/tests/Integration/PipelineTest.php new file mode 100644 index 0000000..59deccc --- /dev/null +++ b/tests/Integration/PipelineTest.php @@ -0,0 +1,139 @@ +toBeInstanceOf(TextDataSource::class) + ->and(app(Chunker::class))->toBeInstanceOf(TextChunker::class) + ->and(app(EmbeddingDriver::class))->toBeInstanceOf(OpenAIEmbeddingDriver::class) + ->and(app(VectorStore::class))->toBeInstanceOf(PgVectorStore::class) + ->and(app(PromptBuilder::class))->toBeInstanceOf(SimplePromptBuilder::class) + ->and(app(Retriever::class))->toBeInstanceOf(SimilarityRetriever::class) + ->and(app(IngestionPipeline::class))->toBeInstanceOf(IngestionPipeline::class) + ->and(app(QueryPipeline::class))->toBeInstanceOf(QueryPipeline::class); +}); + +test('rag facade resolves to RagManager', function (): void { + expect(app('rag'))->toBeInstanceOf(RagManager::class); +}); + +test('rag facade delegates ingest to IngestionPipeline', function (): void { + $result = Rag::ingest(__DIR__.'/../fixtures/sample.txt'); + + // IngestionPipeline with real OpenAI driver will fail (no real API key), + // so we expect an error return — this proves the wiring is correct + expect($result)->toHaveKeys(['stored', 'errors', 'source']) + ->and($result['source'])->toBe(__DIR__.'/../fixtures/sample.txt') + ->and($result['errors'])->toBeGreaterThanOrEqual(0); +}); + +test('rag facade delegates query to QueryPipeline', function (): void { + $result = Rag::query('What is RAG?'); + + // QueryPipeline with real OpenAI driver will fail (no real API key), + // so we expect an answer key — this proves the wiring is correct + expect($result)->toHaveKeys(['answer', 'chunks', 'query']) + ->and($result['query'])->toBe('What is RAG?'); +}); + +test('text data source loads real file', function (): void { + $source = new TextDataSource; + $documents = $source->load(__DIR__.'/../fixtures/sample.txt'); + + expect($documents)->toHaveCount(1) + ->and($documents[0])->toHaveKeys(['id', 'content', 'metadata']) + ->and($documents[0]['id'])->toBeString() + ->and($documents[0]['content'])->toBeString() + ->and($documents[0]['content'])->toContain('RAG systems') + ->and($documents[0]['metadata'])->toHaveKey('source') + ->and($documents[0]['metadata']['source'])->toBe(__DIR__.'/../fixtures/sample.txt'); +}); + +test('text chunker produces chunks from real document', function (): void { + $source = new TextDataSource; + $chunker = new TextChunker(100, 20); + $documents = $source->load(__DIR__.'/../fixtures/sample.txt'); + $chunks = $chunker->chunk($documents[0]); + + expect($chunks)->not->toBeEmpty(); + + foreach ($chunks as $chunk) { + expect($chunk)->toHaveKeys(['id', 'content', 'metadata', 'index']) + ->and($chunk['id'])->toBeString() + ->and($chunk['content'])->toBeString() + ->and($chunk['metadata'])->toHaveKey('parent_id') + ->and($chunk['metadata'])->toHaveKey('chunk_index') + ->and($chunk['index'])->toBeInt(); + } + + // Verify chunk indices are sequential + $indices = array_column($chunks, 'index'); + expect($indices)->toBe(range(0, count($chunks) - 1)); +}); + +test('end-to-end text loading and chunking preserves metadata', function (): void { + $source = new TextDataSource; + $chunker = new TextChunker(200, 50); + $documents = $source->load(__DIR__.'/../fixtures/sample.txt'); + $chunks = $chunker->chunk($documents[0]); + + foreach ($chunks as $chunk) { + expect($chunk['metadata']['source'])->toBe(__DIR__.'/../fixtures/sample.txt') + ->and($chunk['metadata']['parent_id'])->toBe($documents[0]['id']) + ->and($chunk['metadata']['type'])->toBe('text'); + } +}); + +test('prompt builder produces output with real chunk data', function (): void { + $builder = new SimplePromptBuilder; + $context = [ + ['content' => 'RAG systems combine retrieval and generation.', 'score' => 0.95, 'metadata' => []], + ['content' => 'Vector search enables semantic similarity.', 'score' => 0.87, 'metadata' => []], + ]; + + $prompt = $builder->build($context, 'What is RAG?', 4096); + + expect($prompt)->toBeString() + ->and($prompt)->toContain('RAG systems') + ->and($prompt)->toContain('What is RAG?') + ->and($prompt)->toContain('Answer:'); + + $tokenEstimate = $builder->estimateTokens($prompt); + expect($tokenEstimate)->toBeGreaterThan(0); +}); + +test('prompt builder respects max tokens by truncating context', function (): void { + $builder = new SimplePromptBuilder; + $context = [ + ['content' => str_repeat('Word ', 500), 'score' => 0.95, 'metadata' => []], + ['content' => str_repeat('Text ', 500), 'score' => 0.87, 'metadata' => []], + ]; + + $prompt = $builder->build($context, 'Short query', 50); + + expect($prompt)->toBeString() + ->and($prompt)->toContain('Short query') + ->and($prompt)->toContain('Answer:'); +}); + +test('text data source throws on non-existent file', function (): void { + $source = new TextDataSource; + $source->load('/non/existent/path.txt'); +})->throws(RuntimeException::class, 'File not found'); diff --git a/tests/Pest.php b/tests/Pest.php index 7fe1500..f65a9af 100644 --- a/tests/Pest.php +++ b/tests/Pest.php @@ -1,5 +1,5 @@ in(__DIR__); diff --git a/tests/TestCase.php b/tests/TestCase.php index 96a329e..9b53e3f 100644 --- a/tests/TestCase.php +++ b/tests/TestCase.php @@ -1,37 +1,23 @@ 'VendorName\\Skeleton\\Database\\Factories\\'.class_basename($modelName).'Factory' - ); - } - protected function getPackageProviders($app) { return [ - SkeletonServiceProvider::class, + RagServiceProvider::class, ]; } public function getEnvironmentSetUp($app) { config()->set('database.default', 'testing'); - - /* - foreach (\Illuminate\Support\Facades\File::allFiles(__DIR__ . '/../database/migrations') as $migration) { - (include $migration->getRealPath())->up(); - } - */ + config()->set('rag.embedding.api_key', 'test-key'); + config()->set('rag.llm.api_key', 'test-key'); } } diff --git a/tests/Unit/Drivers/OpenAIEmbeddingDriverTest.php b/tests/Unit/Drivers/OpenAIEmbeddingDriverTest.php new file mode 100644 index 0000000..06779f2 --- /dev/null +++ b/tests/Unit/Drivers/OpenAIEmbeddingDriverTest.php @@ -0,0 +1,36 @@ + Http::response(['data' => [['embedding' => array_fill(0, 1536, 0.1)]]], 200), + ]); + $driver = new OpenAIEmbeddingDriver(apiKey: 'test-key', model: 'text-embedding-3-small', dimension: 1536); + $embedding = $driver->embed('Hello world'); + expect($embedding)->toBeArray()->and(count($embedding))->toEqual(1536); +}); + +test('embedBatch returns embeddings for all inputs', function (): void { + Http::fake([ + '*' => Http::response(['data' => [['embedding' => array_fill(0, 1536, 0.1)], ['embedding' => array_fill(0, 1536, 0.2)]]], 200), + ]); + $driver = new OpenAIEmbeddingDriver(apiKey: 'test-key', model: 'text-embedding-3-small', dimension: 1536); + $embeddings = $driver->embedBatch(['text one', 'text two']); + expect($embeddings)->toHaveCount(2); +}); + +test('throws EmbeddingFailedException on API error', function (): void { + Http::fake(['*' => Http::response(['error' => ['message' => 'Unauthorized']], 401)]); + $driver = new OpenAIEmbeddingDriver(apiKey: 'bad-key', model: 'text-embedding-3-small', dimension: 1536); + expect(fn () => $driver->embed('test'))->toThrow(EmbeddingFailedException::class); +}); + +test('getDimension returns configured dimension', function (): void { + $driver = new OpenAIEmbeddingDriver(apiKey: 'key', model: 'model', dimension: 768); + expect($driver->getDimension())->toEqual(768); +}); diff --git a/tests/Unit/Drivers/OpenAILlmDriverTest.php b/tests/Unit/Drivers/OpenAILlmDriverTest.php new file mode 100644 index 0000000..4669c23 --- /dev/null +++ b/tests/Unit/Drivers/OpenAILlmDriverTest.php @@ -0,0 +1,32 @@ + Http::response(['choices' => [['message' => ['content' => 'The answer.']]]], 200)]); + $driver = new OpenAILlmDriver(apiKey: 'test-key', model: 'gpt-4o-mini'); + expect($driver->generate('What is RAG?'))->toEqual('The answer.'); +}); + +test('generateWithSystem includes system message', function (): void { + Http::fake(['*' => Http::response(['choices' => [['message' => ['content' => 'Response.']]]], 200)]); + $driver = new OpenAILlmDriver(apiKey: 'test-key', model: 'gpt-4o-mini'); + $result = $driver->generateWithSystem('You are a helpful assistant.', 'What is RAG?'); + expect($result)->toBeString(); + Http::assertSent(fn ($request) => collect($request->data()['messages'])->contains('role', 'system')); +}); + +test('throws GenerationFailedException on API error', function (): void { + Http::fake(['*' => Http::response(['error' => ['message' => 'Rate limit']], 429)]); + $driver = new OpenAILlmDriver(apiKey: 'test-key', model: 'gpt-4o-mini'); + expect(fn () => $driver->generate('test'))->toThrow(GenerationFailedException::class); +}); + +test('getMaxTokens returns configured value', function (): void { + $driver = new OpenAILlmDriver(apiKey: 'key', model: 'gpt-4o-mini', maxTokens: 2048); + expect($driver->getMaxTokens())->toEqual(2048); +}); diff --git a/tests/Unit/Drivers/TextDataSourceTest.php b/tests/Unit/Drivers/TextDataSourceTest.php new file mode 100644 index 0000000..e36aa48 --- /dev/null +++ b/tests/Unit/Drivers/TextDataSourceTest.php @@ -0,0 +1,30 @@ +load(__DIR__.'/../../fixtures/sample.txt'); + + expect($docs)->toBeArray() + ->and($docs)->toHaveCount(1) + ->and($docs[0])->toHaveKeys(['id', 'content', 'metadata']) + ->and($docs[0]['id'])->toBeString() + ->and($docs[0]['content'])->toContain('RAG systems') + ->and($docs[0]['metadata']['source'])->toEqual(__DIR__.'/../../fixtures/sample.txt'); +}); + +test('each call returns a unique document id', function (): void { + $source = new TextDataSource(); + $doc1 = $source->load(__DIR__.'/../../fixtures/sample.txt'); + $doc2 = $source->load(__DIR__.'/../../fixtures/sample.txt'); + expect($doc1[0]['id'])->not->toEqual($doc2[0]['id']); +}); + +test('throws when file does not exist', function (): void { + $source = new TextDataSource(); + expect(fn () => $source->load('/tmp/does-not-exist-xyz-abc.txt')) + ->toThrow(\Exception::class); +}); diff --git a/tests/Unit/Exceptions/ExceptionsTest.php b/tests/Unit/Exceptions/ExceptionsTest.php new file mode 100644 index 0000000..cd149bd --- /dev/null +++ b/tests/Unit/Exceptions/ExceptionsTest.php @@ -0,0 +1,32 @@ +toBeInstanceOf(EmbeddingFailedException::class) + ->and($e->getMessage())->toContain('API timeout'); +}); + +test('GenerationFailedException creates with reason', function (): void { + $e = GenerationFailedException::create('rate limit'); + expect($e)->toBeInstanceOf(GenerationFailedException::class) + ->and($e->getMessage())->toContain('rate limit'); +}); + +test('RetrievalFailedException creates with reason', function (): void { + $e = RetrievalFailedException::create('no results'); + expect($e)->toBeInstanceOf(RetrievalFailedException::class) + ->and($e->getMessage())->toContain('no results'); +}); + +test('StorageFailedException creates with reason', function (): void { + $e = StorageFailedException::create('connection failed'); + expect($e)->toBeInstanceOf(StorageFailedException::class) + ->and($e->getMessage())->toContain('connection failed'); +}); diff --git a/tests/Unit/Services/IngestionPipelineTest.php b/tests/Unit/Services/IngestionPipelineTest.php new file mode 100644 index 0000000..a1f71cd --- /dev/null +++ b/tests/Unit/Services/IngestionPipelineTest.php @@ -0,0 +1,75 @@ +shouldIgnoreMissing(); + return new IngestionPipeline($source, $chunker, $embedder, $store, $logger); +} + +test('ingest returns stats with stored count', function (): void { + $source = Mockery::mock(DataSource::class); + $source->shouldReceive('load')->once()->andReturn([ + ['id' => 'doc-1', 'content' => 'Hello world', 'metadata' => []], + ]); + $chunker = Mockery::mock(Chunker::class); + $chunker->shouldReceive('chunk')->once()->andReturn([ + ['id' => 'c-1', 'content' => 'Hello world', 'metadata' => [], 'index' => 0], + ]); + $embedder = Mockery::mock(EmbeddingDriver::class); + $embedder->shouldReceive('embedBatch')->once()->andReturn([array_fill(0, 1536, 0.1)]); + $store = Mockery::mock(VectorStore::class); + $store->shouldReceive('storeMany')->once(); + + $stats = makePipeline($source, $chunker, $embedder, $store)->ingest('/path/to/file.txt'); + + expect($stats)->toHaveKeys(['stored', 'errors', 'source']) + ->and($stats['stored'])->toBeGreaterThanOrEqual(1) + ->and($stats['errors'])->toEqual(0); +}); + +test('ingest calls storeMany with embedding and chunk data', function (): void { + $source = Mockery::mock(DataSource::class); + $source->shouldReceive('load')->andReturn([ + ['id' => 'doc-1', 'content' => 'Test', 'metadata' => []], + ]); + $chunker = Mockery::mock(Chunker::class); + $chunker->shouldReceive('chunk')->andReturn([ + ['id' => 'c-1', 'content' => 'Test', 'metadata' => [], 'index' => 0], + ]); + $embedder = Mockery::mock(EmbeddingDriver::class); + $embedder->shouldReceive('embedBatch')->andReturn([array_fill(0, 1536, 0.5)]); + $store = Mockery::mock(VectorStore::class); + $store->shouldReceive('storeMany')->once()->withArgs(function (array $items): bool { + return count($items) > 0; + }); + + makePipeline($source, $chunker, $embedder, $store)->ingest('test.txt'); +}); + +test('ingest returns errors when embedder throws', function (): void { + $source = Mockery::mock(DataSource::class); + $source->shouldReceive('load')->andReturn([ + ['id' => 'doc-1', 'content' => 'Test', 'metadata' => []], + ]); + $chunker = Mockery::mock(Chunker::class); + $chunker->shouldReceive('chunk')->andReturn([ + ['id' => 'c-1', 'content' => 'Test', 'metadata' => [], 'index' => 0], + ]); + $embedder = Mockery::mock(EmbeddingDriver::class); + $embedder->shouldReceive('embedBatch')->andThrow(new \RuntimeException('API error')); + $store = Mockery::mock(VectorStore::class); + $store->shouldNotReceive('storeMany'); + + $stats = makePipeline($source, $chunker, $embedder, $store)->ingest('test.txt'); + expect($stats['errors'])->toBeGreaterThan(0); +}); diff --git a/tests/Unit/Services/QueryPipelineTest.php b/tests/Unit/Services/QueryPipelineTest.php new file mode 100644 index 0000000..ac2a1fa --- /dev/null +++ b/tests/Unit/Services/QueryPipelineTest.php @@ -0,0 +1,58 @@ +shouldIgnoreMissing(); + return new QueryPipeline($retriever, $builder, $llm, $logger); +} + +test('query returns answer, chunks, and query string', function (): void { + $chunks = [['content' => 'RAG is great.', 'score' => 0.9, 'metadata' => []]]; + $retriever = Mockery::mock(Retriever::class); + $retriever->shouldReceive('retrieve')->once()->andReturn($chunks); + $builder = Mockery::mock(PromptBuilder::class); + $builder->shouldReceive('build')->once()->andReturn('built prompt'); + $llm = Mockery::mock(LlmDriver::class); + $llm->shouldReceive('getMaxTokens')->andReturn(4096); + $llm->shouldReceive('generate')->once()->andReturn('RAG stands for Retrieval-Augmented Generation.'); + + $result = makeQueryPipeline($retriever, $builder, $llm)->query('What is RAG?'); + + expect($result)->toHaveKeys(['answer', 'chunks', 'query']) + ->and($result['answer'])->toEqual('RAG stands for Retrieval-Augmented Generation.') + ->and($result['query'])->toEqual('What is RAG?'); +}); + +test('query passes topK to retriever', function (): void { + $retriever = Mockery::mock(Retriever::class); + $retriever->shouldReceive('retrieve')->once()->withArgs(fn ($q, $k) => $k === 10)->andReturn([]); + $builder = Mockery::mock(PromptBuilder::class); + $builder->shouldReceive('build')->andReturn('prompt'); + $llm = Mockery::mock(LlmDriver::class); + $llm->shouldReceive('getMaxTokens')->andReturn(4096); + $llm->shouldReceive('generate')->andReturn('answer'); + + makeQueryPipeline($retriever, $builder, $llm)->query('query', 10); +}); + +test('query returns answer key when llm throws', function (): void { + $retriever = Mockery::mock(Retriever::class); + $retriever->shouldReceive('retrieve')->andReturn([['content' => 'some context', 'score' => 0.9, 'metadata' => []]]); + $builder = Mockery::mock(PromptBuilder::class); + $builder->shouldReceive('build')->andReturn('prompt'); + $llm = Mockery::mock(LlmDriver::class); + $llm->shouldReceive('getMaxTokens')->andReturn(4096); + $llm->shouldReceive('generate')->andThrow(new \RuntimeException('LLM error')); + + $result = makeQueryPipeline($retriever, $builder, $llm)->query('question'); + expect($result)->toHaveKey('answer'); +}); diff --git a/tests/Unit/Services/SimplePromptBuilderTest.php b/tests/Unit/Services/SimplePromptBuilderTest.php new file mode 100644 index 0000000..9425342 --- /dev/null +++ b/tests/Unit/Services/SimplePromptBuilderTest.php @@ -0,0 +1,38 @@ + 'RAG stands for Retrieval-Augmented Generation.', 'score' => 0.92, 'metadata' => []]]; + $prompt = $builder->build($chunks, 'What is RAG?', 4000); + expect($prompt)->toBeString()->not->toBeEmpty() + ->and($prompt)->toContain('What is RAG?') + ->and($prompt)->toContain('RAG stands for'); +}); + +test('build respects maxTokens limit by truncating chunks', function (): void { + $builder = new SimplePromptBuilder(); + $chunks = array_fill(0, 100, ['content' => str_repeat('word ', 200), 'score' => 0.9, 'metadata' => []]); + $prompt = $builder->build($chunks, 'query', 200); + expect(strlen($prompt))->toBeLessThan(20000); +}); + +test('buildWithSystem includes system prompt in output', function (): void { + $builder = new SimplePromptBuilder(); + $prompt = $builder->buildWithSystem( + 'You are a PHP expert.', + [['content' => 'PHP is great.', 'score' => 0.9, 'metadata' => []]], + 'Tell me about PHP', + 4000 + ); + expect($prompt)->toContain('You are a PHP expert.'); +}); + +test('estimateTokens returns a positive integer', function (): void { + $builder = new SimplePromptBuilder(); + $tokens = $builder->estimateTokens(implode(' ', array_fill(0, 100, 'word'))); + expect($tokens)->toBeGreaterThan(0); +}); diff --git a/tests/Unit/Services/TextChunkerTest.php b/tests/Unit/Services/TextChunkerTest.php new file mode 100644 index 0000000..6d92126 --- /dev/null +++ b/tests/Unit/Services/TextChunkerTest.php @@ -0,0 +1,43 @@ + 'doc-1', 'content' => str_repeat('word ', 30), 'metadata' => []]; + $chunks = $chunker->chunk($document); + expect(count($chunks))->toBeGreaterThan(1); +}); + +test('each chunk has required keys', function (): void { + $chunker = new TextChunker(maxChunkSize: 100, overlap: 20); + $document = ['id' => 'doc-1', 'content' => 'Hello world. This is a test document.', 'metadata' => []]; + $chunks = $chunker->chunk($document); + foreach ($chunks as $chunk) { + expect($chunk)->toHaveKeys(['id', 'content', 'metadata', 'index']); + } +}); + +test('chunk metadata inherits parent metadata', function (): void { + $chunker = new TextChunker(maxChunkSize: 500, overlap: 0); + $document = ['id' => 'doc-abc', 'content' => 'Some content here.', 'metadata' => ['source' => 'file.txt', 'author' => 'test']]; + $chunks = $chunker->chunk($document); + expect($chunks[0]['metadata']['source'])->toEqual('file.txt') + ->and($chunks[0]['metadata']['parent_id'])->toEqual('doc-abc'); +}); + +test('short text produces single chunk', function (): void { + $chunker = new TextChunker(maxChunkSize: 1000, overlap: 0); + $document = ['id' => 'doc-1', 'content' => 'Short text.', 'metadata' => []]; + $chunks = $chunker->chunk($document); + expect($chunks)->toHaveCount(1) + ->and($chunks[0]['index'])->toEqual(0); +}); + +test('getMaxChunkSize and getOverlap return configured values', function (): void { + $chunker = new TextChunker(maxChunkSize: 750, overlap: 50); + expect($chunker->getMaxChunkSize())->toEqual(750) + ->and($chunker->getOverlap())->toEqual(50); +}); diff --git a/tests/fixtures/sample.txt b/tests/fixtures/sample.txt new file mode 100644 index 0000000..2d1bb8c --- /dev/null +++ b/tests/fixtures/sample.txt @@ -0,0 +1 @@ +This is a test document about RAG systems. It has multiple sentences. Vector search is powerful.