diff --git a/.gitignore b/.gitignore index b42284e0..282463ad 100644 --- a/.gitignore +++ b/.gitignore @@ -2,3 +2,6 @@ utils/params.py .aider* .env __pycache__/ +.DS_Store +.cursor/ +.vscode/ diff --git a/learning/README.md b/learning/README.md index afeca845..58859f99 100644 --- a/learning/README.md +++ b/learning/README.md @@ -1,6 +1,6 @@ # Pedagogical Concept Graph: Interactive Learning System -An intelligent learning platform that transforms textbooks into interactive, guided learning experiences using concept graphs, Socratic dialogue, and retrieval-augmented generation (RAG). +An intelligent learning platform that transforms textbooks into interactive, guided learning experiences using concept graphs, Socratic dialogue, retrieval-augmented generation (RAG), and conversational spaced repetition. ## Overview @@ -76,6 +76,26 @@ To keep dialogue grounded in the actual textbook: **Result**: LLM uses Norvig's actual examples, terminology, and pedagogical approach +### 5. **Conversational Spaced Repetition** + +The system automatically maintains long-term retention through natural dialogue-based reviews: + +- **No Manual Cards**: The agent automatically extracts what's worth remembering during learning sessions +- **Proactive Reviews**: Instead of flipping flashcards, the agent initiates conversations: *"Hey, remember that concept we discussed last week? How does it work?"* +- **Natural Assessment**: The agent interprets your answers and grades understanding (0.0-1.0) without requiring self-grading +- **FSRS Algorithm**: Implements Free Spaced Repetition Scheduler with exponential decay forgetting curves +- **Smart Scheduling**: Concepts are reviewed when retrievability drops below 90%, prioritized by urgency and difficulty +- **Memory Persistence**: Past learning notes are used to personalize review questions + +**How it Works:** +1. During Socratic dialogue, the system automatically creates "memories" (notes about what you learned) +2. The FSRS algorithm tracks each concept's stability and retrievability over time +3. When concepts are due for review (retrievability < 90%), the Review Dialogue opens +4. The AI tutor probes your recall using past memories, assessing understanding naturally +5. Based on your responses, the system updates stability and schedules the next review + +**Technology**: FSRS algorithm with exponential decay, localStorage persistence (backend sync planned) + ## Architecture ``` @@ -86,12 +106,18 @@ learning/ │ └── embeddings.json # Vector embeddings (3072-dim) ├── app/ │ ├── api/ -│ │ └── socratic-dialogue/ # RAG + LLM dialogue endpoint +│ │ ├── socratic-dialogue/ # RAG + LLM dialogue endpoint +│ │ └── review-dialogue/ # Spaced repetition review endpoint │ ├── components/ │ │ ├── ConceptGraph.tsx # Interactive graph visualization │ │ ├── ConceptDetails.tsx # Concept info + prerequisites -│ │ └── SocraticDialogue.tsx # AI tutor modal +│ │ ├── SocraticDialogue.tsx # AI tutor modal +│ │ └── ReviewDialogue.tsx # Spaced repetition review modal │ └── page.tsx # Main application +├── lib/ +│ ├── memory-store.ts # Memory persistence & FSRS state +│ ├── spaced-repetition.ts # FSRS scheduling algorithm +│ └── gemini-utils.ts # Gemini API utilities ├── scripts/ │ ├── chunk-paip.ts # Semantic chunking script │ └── embed-chunks.ts # Vector embedding generation @@ -147,6 +173,16 @@ learning/ 4. **Track progress**: Watch as skills are marked as demonstrated 5. **Achieve mastery**: When ready, mark the concept as mastered 6. **Unlock new concepts**: Dependent concepts become available +7. **Automatic memory creation**: The system saves notes about what you learned for future reviews + +### Reviewing with Conversational Spaced Repetition + +1. **Automatic notifications**: When concepts are due for review, a notification appears +2. **Open Review Dialogue**: Click to start a natural conversation about concepts you've learned +3. **Answer naturally**: The AI tutor asks questions based on your past learning notes +4. **Automatic assessment**: Your understanding is evaluated from your responses (no self-grading needed) +5. **Smart scheduling**: The system adjusts when you'll see each concept again based on your performance +6. **Multi-concept reviews**: Multiple concepts can be reviewed in a single conversation ### Progress Tracking @@ -155,7 +191,7 @@ learning/ - **Green**: Ready to learn (prerequisites satisfied) ✅ - **Faded**: Locked (missing prerequisites) 🔒 -Progress persists in your browser's localStorage across sessions. +Progress and spaced repetition state persist in your browser's localStorage across sessions. ## Technical Highlights @@ -194,6 +230,28 @@ The LLM returns both dialogue and evaluation in structured JSON: } ``` +### FSRS Spaced Repetition Algorithm + +The system implements a simplified Free Spaced Repetition Scheduler (FSRS) with exponential decay: + +**Core Formula:** +$$ R(t) = 0.9^{\frac{t}{S}} $$ + +Where: +- $R(t)$ is retrievability (probability of recall) at time $t$ +- $t$ is days elapsed since last review +- $S$ is stability (days until retrievability drops to 90%) + +**Review Scheduling:** +- Concepts are marked "Due" when $R < 0.9$ +- Priority calculation considers: low retrievability, high difficulty, and overdue time +- Understanding scores (0.0-1.0) from AI assessment update stability: + - Good recall (≥0.7): Large interval increase (2.2-2.5x) + - Medium recall (0.3-0.7): Maintenance interval (0.8-1.5x) + - Poor recall (<0.3): Reset with shorter interval (0.2-0.5x) + +**Implementation**: See `lib/spaced-repetition.ts` for the complete algorithm + ## Development Scripts ```bash @@ -218,11 +276,19 @@ See [NOTES.md](NOTES.md) for comprehensive documentation including: - UI/UX design decisions - Performance metrics and optimizations +See [docs/conversational_spaced_repetition_implementation_summary.md](docs/conversational_spaced_repetition_implementation_summary.md) for detailed documentation on: +- Conversational spaced repetition motivation and design +- FSRS algorithm implementation +- Active recall through review dialogue +- Memory store architecture +- Future enhancements + ## Future Enhancements - [ ] Complete Pass 2 enrichment for all 33 concepts -- [ ] Spaced repetition for long-term retention +- [ ] Backend sync: Move spaced repetition state from localStorage to database - [ ] Voice interface with Gemini Live +- [ ] Parameter tuning: Advanced FSRS parameter optimization based on aggregate user data - [ ] Model selector (Flash vs Pro vs Thinking) - [ ] Multi-chapter support - [ ] Exercise session flow @@ -237,6 +303,7 @@ See [NOTES.md](NOTES.md) for comprehensive documentation including: - **Markdown Rendering**: react-markdown with syntax highlighting - **AI**: Google Gemini 2.5 Flash, Gemini Embedding 001 - **State Management**: React hooks + localStorage +- **Spaced Repetition**: FSRS algorithm with exponential decay forgetting curves ## License diff --git a/learning/app/api/review-dialogue/route.ts b/learning/app/api/review-dialogue/route.ts new file mode 100644 index 00000000..06e22341 --- /dev/null +++ b/learning/app/api/review-dialogue/route.ts @@ -0,0 +1,355 @@ +/** + * Copyright 2025 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { NextRequest, NextResponse } from 'next/server'; + +type Message = { + role: 'system' | 'user' | 'assistant'; + content: string; +}; + +// Memory type from memory-store +type Memory = { + id: string; + conceptId: string; + content: string; + understanding: number; + timestamp: number; + context?: string; +}; + +// Due concept with memories for review +type DueConceptForReview = { + conceptId: string; + conceptName: string; + retrievability: number; + memories: Memory[]; +}; + +// Response types +type ConceptProbed = { + conceptId: string; + understanding: number; + new_memory?: { + content: string; + understanding: number; + context?: string; + }; +}; + +export async function POST(request: NextRequest) { + try { + const { + conversationHistory, + dueConcepts, + libraryId, + } = await request.json(); + + console.log('\n━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━'); + console.log('🔄 NEW REVIEW DIALOGUE REQUEST'); + console.log('━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━'); + console.log('📌 Library ID:', libraryId); + console.log('📌 Conversation turns:', conversationHistory.length); + console.log('📌 Due concepts:', dueConcepts?.length || 0); + + if (dueConcepts?.length > 0) { + console.log('📌 Concepts to review:'); + dueConcepts.forEach((c: DueConceptForReview, i: number) => { + console.log(` ${i + 1}. ${c.conceptName} (${Math.round(c.retrievability * 100)}% retained, ${c.memories.length} memories)`); + }); + } + + // Get API key from environment + const apiKey = process.env.GOOGLE_API_KEY; + + if (!apiKey) { + return NextResponse.json( + { error: 'API key not configured. Please add GOOGLE_API_KEY to .env.local' }, + { status: 500 } + ); + } + + // Build system prompt for review conversation + const systemPrompt = buildReviewPrompt(dueConcepts as DueConceptForReview[]); + + console.log('\n📝 SYSTEM PROMPT CONSTRUCTED:'); + console.log(` - Total length: ${systemPrompt.length} characters`); + + // Convert conversation history to Gemini format + const geminiContents = convertToGeminiFormat(systemPrompt, conversationHistory); + + console.log('\n📤 SENDING TO GEMINI:'); + console.log(` - Total messages: ${geminiContents.length}`); + + // Call Google Gemini API with structured output + const model = 'gemini-2.5-flash'; + const response = await fetch( + `https://generativelanguage.googleapis.com/v1beta/models/${model}:generateContent?key=${apiKey}`, + { + method: 'POST', + headers: { + 'Content-Type': 'application/json', + }, + body: JSON.stringify({ + contents: geminiContents, + generationConfig: { + temperature: 0.7, + maxOutputTokens: 1500, + responseMimeType: 'application/json', + responseSchema: { + type: 'object', + properties: { + message: { + type: 'string', + description: 'The conversational response to the student', + }, + concepts_probed: { + type: 'array', + description: 'Array of concepts that were probed in this exchange', + items: { + type: 'object', + properties: { + conceptId: { + type: 'string', + description: 'The ID of the concept that was probed', + }, + understanding: { + type: 'number', + description: 'Understanding level demonstrated (0-1)', + }, + new_memory: { + type: 'object', + description: 'Optional memory note about this interaction', + properties: { + content: { + type: 'string', + description: 'Note about what the student demonstrated or struggled with', + }, + understanding: { + type: 'number', + description: 'Understanding level (0-1)', + }, + context: { + type: 'string', + description: 'What prompted this observation', + }, + }, + required: ['content', 'understanding'], + }, + }, + required: ['conceptId', 'understanding'], + }, + }, + review_complete: { + type: 'boolean', + description: 'True if all key concepts have been sufficiently probed and the review can end naturally', + }, + }, + required: ['message', 'concepts_probed', 'review_complete'], + }, + }, + }), + } + ); + + if (!response.ok) { + const error = await response.text(); + console.error('\n❌ GEMINI API ERROR:'); + console.error(error); + console.log('━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n'); + return NextResponse.json( + { error: 'Failed to get response from Gemini' }, + { status: response.status } + ); + } + + const data = await response.json(); + + console.log('\n📥 RECEIVED FROM GEMINI:'); + console.log(' - Usage metadata:', JSON.stringify(data.usageMetadata, null, 2)); + + // Find the text response + const textPart = data.candidates[0].content.parts.find( + (part: any) => part.text !== undefined + ); + + if (!textPart) { + console.error('\n❌ No text part found in response'); + return NextResponse.json( + { error: 'Invalid response structure from Gemini' }, + { status: 500 } + ); + } + + const responseText = textPart.text; + console.log('\n📄 RAW RESPONSE TEXT:'); + console.log(responseText.substring(0, 500) + (responseText.length > 500 ? '...' : '')); + + // Parse the JSON response + let parsedResponse; + try { + parsedResponse = JSON.parse(responseText); + console.log('\n✅ PARSED RESPONSE:'); + console.log(' - Message length:', parsedResponse.message.length); + console.log(' - Concepts probed:', parsedResponse.concepts_probed?.length || 0); + console.log(' - Review complete:', parsedResponse.review_complete); + + if (parsedResponse.concepts_probed?.length > 0) { + parsedResponse.concepts_probed.forEach((cp: ConceptProbed) => { + console.log(` - ${cp.conceptId}: understanding ${Math.round(cp.understanding * 100)}%`); + }); + } + } catch (e) { + console.error('\n❌ JSON PARSE ERROR:', e); + + parsedResponse = { + message: '⚠️ I encountered an error. Please click retry to try again.', + concepts_probed: [], + review_complete: false, + }; + } + + console.log('━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n'); + + return NextResponse.json({ + message: parsedResponse.message, + concepts_probed: parsedResponse.concepts_probed || [], + review_complete: parsedResponse.review_complete || false, + usage: data.usageMetadata, + }); + + } catch (error) { + console.error('\n💥 UNEXPECTED ERROR:'); + console.error(error); + console.log('━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n'); + return NextResponse.json( + { error: 'Internal server error' }, + { status: 500 } + ); + } +} + +// Convert OpenAI-style messages to Gemini format +function convertToGeminiFormat(systemPrompt: string, conversationHistory: Message[]) { + const contents: any[] = []; + + // If conversation is empty, add system prompt as first user message + if (conversationHistory.length === 0) { + contents.push({ + role: 'user', + parts: [{ text: systemPrompt }], + }); + } else { + // Prepend system prompt to the first user message + const firstUserIndex = conversationHistory.findIndex((msg) => msg.role === 'user'); + + conversationHistory.forEach((msg, index) => { + if (msg.role === 'user') { + const text = index === firstUserIndex + ? `${systemPrompt}\n\n---\n\nStudent: ${msg.content}` + : msg.content; + + contents.push({ + role: 'user', + parts: [{ text }], + }); + } else if (msg.role === 'assistant') { + contents.push({ + role: 'model', + parts: [{ text: msg.content }], + }); + } + }); + } + + return contents; +} + +// Build a review-focused prompt for multi-concept conversations +function buildReviewPrompt(dueConcepts: DueConceptForReview[]): string { + // Format concepts with their memories + const conceptsList = dueConcepts.map((c, i) => { + const retentionPercent = Math.round(c.retrievability * 100); + const memorySummary = c.memories.length > 0 + ? c.memories.slice(-3).map(m => `"${m.content}"`).join('; ') + : 'No previous notes'; + + return `${i + 1}. **${c.conceptName}** (ID: ${c.conceptId}) + - Current retention: ${retentionPercent}% + - Past notes: ${memorySummary}`; + }).join('\n\n'); + + return `You are a friendly tutor having a review conversation with a student. Your job is to naturally probe their recall of concepts they've learned previously. + +**CONCEPTS DUE FOR REVIEW:** + +${conceptsList} + +**YOUR OBJECTIVES:** + +1. Start with a casual greeting, then naturally ask about the concepts +2. Focus on concepts with lower retention first (they need the most reinforcement) +3. Use your past notes to personalize questions - reference specific struggles or successes +4. Probe understanding through questions, not by lecturing +5. If they struggle, give hints rather than answers +6. Assess their understanding from their responses (0-1 scale) +7. Write new memory notes when you observe something notable about their understanding +8. When you've adequately probed the key concepts, wrap up the conversation naturally + +**CONVERSATION GUIDELINES:** + +- Be conversational and encouraging, not clinical +- Don't make it feel like a test - it should feel like catching up with a tutor +- Weave between concepts naturally based on the conversation flow +- You don't have to cover every concept - focus on the most important ones +- Keep responses concise (2-4 sentences with a question) +- Set review_complete to true when you've covered the key concepts and it's a natural ending point + +**ASSESSMENT GUIDELINES:** + +- 0.0-0.3: Major confusion, forgot key points, needs significant review +- 0.4-0.6: Partial recall, got the gist but missing details +- 0.7-0.9: Good recall, understood with minor gaps +- 1.0: Perfect recall, quick and confident response + +**MEMORY WRITING GUIDELINES:** + +- Write a new_memory when the student shows notable understanding or confusion +- Keep notes concise but specific: what they understood, what they confused +- Include context about what prompted the observation +- Don't write a memory for every interaction - only when noteworthy + +**RESPONSE FORMAT:** + +Return JSON with: +{ + "message": "Your conversational response here", + "concepts_probed": [ + { + "conceptId": "concept_id", + "understanding": 0.75, + "new_memory": { + "content": "Student explained X clearly but still confuses Y with Z", + "understanding": 0.75, + "context": "When asked about recursion" + } + } + ], + "review_complete": false +} + +Begin with a friendly opening that leads into probing their recall of the first concept.`; +} diff --git a/learning/app/api/socratic-dialogue/route.ts b/learning/app/api/socratic-dialogue/route.ts index af070c86..92abc424 100644 --- a/learning/app/api/socratic-dialogue/route.ts +++ b/learning/app/api/socratic-dialogue/route.ts @@ -23,6 +23,22 @@ type Message = { content: string; }; +// Memory types for conversational spaced repetition +type Memory = { + id: string; + conceptId: string; + content: string; + understanding: number; + timestamp: number; + context?: string; +}; + +type DueConcept = { + conceptId: string; + retrievability: number; + priority: number; +}; + // Compute cosine similarity between two vectors function cosineSimilarity(a: number[], b: number[]): number { if (a.length !== b.length) { @@ -187,7 +203,16 @@ async function loadConceptContext( export async function POST(request: NextRequest) { try { - const { conceptId, conversationHistory, conceptData, textbookContext, embeddingsPath } = await request.json(); + const { + conceptId, + conversationHistory, + conceptData, + textbookContext, + embeddingsPath, + // New fields for conversational spaced repetition + memories, // Past memories for this concept + dueConcepts, // Concepts that need probing (from FSRS) + } = await request.json(); console.log('\n━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━'); console.log('🎓 NEW SOCRATIC DIALOGUE REQUEST'); @@ -196,6 +221,8 @@ export async function POST(request: NextRequest) { console.log('📌 Concept Name:', conceptData.name); console.log('📌 Conversation turns:', conversationHistory.length); console.log('📌 Textbook context:', textbookContext ? 'CACHED ✅' : 'NEEDS SEARCH 🔍'); + console.log('📌 Memories:', memories?.length || 0); + console.log('📌 Due concepts:', dueConcepts?.length || 0); // Get API key from environment (prefer GOOGLE_API_KEY) const apiKey = process.env.GOOGLE_API_KEY; @@ -234,8 +261,13 @@ export async function POST(request: NextRequest) { console.log(textbookSections.substring(0, 500) + '...\n'); } - // Build system prompt with textbook grounding - const systemPrompt = buildSocraticPrompt(conceptData, textbookSections); + // Build system prompt with textbook grounding and memory context + const systemPrompt = buildSocraticPrompt( + conceptData, + textbookSections, + memories as Memory[] | undefined, + dueConcepts as DueConcept[] | undefined + ); console.log('\n📝 SYSTEM PROMPT CONSTRUCTED:'); console.log(` - Total length: ${systemPrompt.length} characters`); @@ -303,6 +335,36 @@ export async function POST(request: NextRequest) { }, required: ['indicators_demonstrated', 'confidence', 'ready_for_mastery'], }, + new_memory: { + type: 'object', + description: 'A memory note about this conversation turn (only if significant learning occurred)', + properties: { + content: { + type: 'string', + description: 'Free-form note about what the student demonstrated or struggled with', + }, + understanding: { + type: 'number', + description: 'Understanding level demonstrated (0-1)', + }, + context: { + type: 'string', + description: 'What prompted this observation', + }, + }, + required: ['content', 'understanding'], + }, + concept_assessment: { + type: 'object', + description: 'Overall assessment for FSRS scheduling', + properties: { + understanding: { + type: 'number', + description: 'Overall understanding level for this concept (0-1)', + }, + }, + required: ['understanding'], + }, }, required: ['message', 'mastery_assessment'], }, @@ -355,6 +417,13 @@ export async function POST(request: NextRequest) { console.log(' - Indicators demonstrated:', parsedResponse.mastery_assessment.indicators_demonstrated); console.log(' - Confidence:', parsedResponse.mastery_assessment.confidence); console.log(' - Ready for mastery:', parsedResponse.mastery_assessment.ready_for_mastery); + if (parsedResponse.new_memory) { + console.log(' - New memory:', parsedResponse.new_memory.content?.substring(0, 50) + '...'); + console.log(' - Understanding:', parsedResponse.new_memory.understanding); + } + if (parsedResponse.concept_assessment) { + console.log(' - Concept assessment:', parsedResponse.concept_assessment.understanding); + } } catch (e) { console.error('\n❌ JSON PARSE ERROR:', e); console.error(' - Raw response:', responseText); @@ -381,13 +450,32 @@ export async function POST(request: NextRequest) { console.log('━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n'); - return NextResponse.json({ + // Build response with new memory fields + const apiResponse: any = { message: parsedResponse.message, mastery_assessment: parsedResponse.mastery_assessment, textbookContext: textbookSections, sources: sourceChunks, usage: data.usageMetadata, - }); + }; + + // Include new memory if the agent created one + if (parsedResponse.new_memory) { + apiResponse.new_memory = { + ...parsedResponse.new_memory, + conceptId, + }; + } + + // Include concept assessment for FSRS + if (parsedResponse.concept_assessment) { + apiResponse.concept_assessment = { + conceptId, + ...parsedResponse.concept_assessment, + }; + } + + return NextResponse.json(apiResponse); } catch (error) { console.error('\n💥 UNEXPECTED ERROR:'); @@ -441,14 +529,38 @@ function convertToGeminiFormat(systemPrompt: string, conversationHistory: Messag // Build a Socratic teaching prompt using the concept's pedagogical data function buildSocraticPrompt( conceptData: any, - textbookSections?: string + textbookSections?: string, + memories?: Memory[], + dueConcepts?: DueConcept[] ): string { const { name, description, learning_objectives, mastery_indicators, examples, misconceptions } = conceptData; + // Build memory context section + let memoryContext = ''; + if (memories && memories.length > 0) { + memoryContext = ` +**YOUR NOTES FROM PREVIOUS CONVERSATIONS WITH THIS STUDENT:** +${memories.map(m => `- [${new Date(m.timestamp).toLocaleDateString()}] ${m.content} (understanding: ${Math.round(m.understanding * 100)}%)`).join('\n')} + +Use these notes to personalize your teaching. Reference past struggles or successes naturally. +`; + } + + // Build due concepts context section + let dueContext = ''; + if (dueConcepts && dueConcepts.length > 0) { + dueContext = ` +**CONCEPTS DUE FOR REVIEW (weave these in naturally if relevant):** +${dueConcepts.slice(0, 3).map(d => `- ${d.conceptId} (retrievability: ${Math.round(d.retrievability * 100)}%)`).join('\n')} + +If the conversation allows, probe these concepts to refresh the student's memory. +`; + } + return `You are a Socratic tutor teaching the concept: "${name}". **Concept Description:** ${description} - +${memoryContext}${dueContext} ${textbookSections ? ` **YOUR TEACHING MATERIAL (internalize this as your own knowledge):** @@ -502,9 +614,23 @@ Return JSON with: "confidence": 0.85, "ready_for_mastery": false, "next_focus": "Let's explore X next..." + }, + "new_memory": { // Optional - only include if significant learning occurred + "content": "Student demonstrated clear understanding of X but confused Y with Z", + "understanding": 0.7, + "context": "When explaining recursion" + }, + "concept_assessment": { // Optional - overall understanding for this exchange + "understanding": 0.75 } } +**Memory Writing Guidelines:** +- Write a "new_memory" when the student demonstrates or struggles with something notable +- Keep notes concise but specific (what they understood, what they confused) +- Set understanding 0-0.3 for confusion/errors, 0.4-0.6 for partial understanding, 0.7-1.0 for strong grasp +- Don't write a memory for every turn - only when there's something worth remembering + **Example Interaction Pattern:** - Start with an open question about their understanding - Ask follow-up questions based on their answers diff --git a/learning/app/api/storage/route.ts b/learning/app/api/storage/route.ts new file mode 100644 index 00000000..4c0a4c84 --- /dev/null +++ b/learning/app/api/storage/route.ts @@ -0,0 +1,58 @@ + +import { NextRequest, NextResponse } from 'next/server'; +import fs from 'fs'; +import path from 'path'; + +const STORAGE_FILE = path.join(process.cwd(), 'data', 'storage.json'); + +// Helper to ensure data directory exists +function ensureDirectory() { + const dir = path.dirname(STORAGE_FILE); + if (!fs.existsSync(dir)) { + fs.mkdirSync(dir, { recursive: true }); + } +} + +export async function GET() { + try { + if (!fs.existsSync(STORAGE_FILE)) { + return NextResponse.json({}); + } + const content = fs.readFileSync(STORAGE_FILE, 'utf-8'); + const data = JSON.parse(content); + return NextResponse.json(data); + } catch (error) { + console.error('Failed to read storage:', error); + return NextResponse.json({ error: 'Failed to read storage' }, { status: 500 }); + } +} + +export async function POST(req: NextRequest) { + try { + const changes = await req.json(); + + ensureDirectory(); + + // Read existing data + let currentData = {}; + if (fs.existsSync(STORAGE_FILE)) { + try { + const content = fs.readFileSync(STORAGE_FILE, 'utf-8'); + currentData = JSON.parse(content); + } catch (e) { + // Ignore parse errors, start fresh + } + } + + // Merge changes + const newData = { ...currentData, ...changes }; + + // Write back + fs.writeFileSync(STORAGE_FILE, JSON.stringify(newData, null, 2)); + + return NextResponse.json({ success: true }); + } catch (error) { + console.error('Failed to write storage:', error); + return NextResponse.json({ error: 'Failed to write storage' }, { status: 500 }); + } +} diff --git a/learning/app/components/ReviewDialogue.tsx b/learning/app/components/ReviewDialogue.tsx new file mode 100644 index 00000000..4650fe91 --- /dev/null +++ b/learning/app/components/ReviewDialogue.tsx @@ -0,0 +1,493 @@ +/** + * Copyright 2025 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +'use client'; + +import { useState, useRef, useEffect } from 'react'; +import ReactMarkdown from 'react-markdown'; +import remarkGfm from 'remark-gfm'; +import { Prism as SyntaxHighlighter } from 'react-syntax-highlighter'; +import { oneDark } from 'react-syntax-highlighter/dist/esm/styles/prism'; +import { + Dialog, + DialogContent, + DialogDescription, + DialogHeader, + DialogTitle, +} from '@/components/ui/dialog'; +import { Button } from '@/components/ui/button'; +import { Textarea } from '@/components/ui/textarea'; + +import { + ConceptState, + Memory, + loadConceptStates, + saveConceptStates, + batchUpdateFSRS, + ConceptAssessment, +} from '@/lib/memory-store'; +import { + DueConcept, + calculateRetrievability, + calculateNextInterval, +} from '@/lib/spaced-repetition'; + +type Message = { + role: 'user' | 'assistant'; + content: string; + conceptsProbed?: ConceptProbed[]; +}; + +type ConceptProbed = { + conceptId: string; + understanding: number; + new_memory?: { + content: string; + understanding: number; + context?: string; + }; +}; + +type ReviewDialogueProps = { + open: boolean; + onOpenChange: (open: boolean) => void; + dueConcepts: DueConcept[]; + conceptNames: Map; // conceptId -> name mapping + libraryId: string; + onReviewComplete?: (assessments: ConceptAssessment[]) => void; +}; + +export default function ReviewDialogue({ + open, + onOpenChange, + dueConcepts, + conceptNames, + libraryId, + onReviewComplete, +}: ReviewDialogueProps) { + const [messages, setMessages] = useState([]); + const [input, setInput] = useState(''); + const [isLoading, setIsLoading] = useState(false); + const [hasStarted, setHasStarted] = useState(false); + const [reviewComplete, setReviewComplete] = useState(false); + const [allAssessments, setAllAssessments] = useState([]); + const [conceptStates, setConceptStates] = useState>(new Map()); + const messagesEndRef = useRef(null); + const textareaRef = useRef(null); + + // Auto-scroll to bottom when new messages arrive + useEffect(() => { + messagesEndRef.current?.scrollIntoView({ behavior: 'smooth' }); + }, [messages]); + + // Load concept states on mount + useEffect(() => { + if (open) { + const states = loadConceptStates(libraryId); + setConceptStates(states); + } + }, [libraryId, open]); + + // Auto-focus textarea when loading completes + useEffect(() => { + if (!isLoading && textareaRef.current) { + textareaRef.current.focus(); + } + }, [isLoading]); + + // Start the dialogue when modal opens + useEffect(() => { + if (open && !hasStarted && dueConcepts.length > 0) { + startDialogue(); + } + }, [open, hasStarted, dueConcepts]); + + // Reset when modal closes + useEffect(() => { + if (!open) { + setMessages([]); + setHasStarted(false); + setInput(''); + setReviewComplete(false); + setAllAssessments([]); + } + }, [open]); + + const startDialogue = async () => { + setIsLoading(true); + setHasStarted(true); + + // Prepare due concepts with memories + const dueConceptsWithMemories = dueConcepts.map(dc => ({ + conceptId: dc.conceptId, + conceptName: conceptNames.get(dc.conceptId) || dc.conceptId, + retrievability: dc.retrievability, + memories: dc.state.memories, + })); + + try { + const response = await fetch('/api/review-dialogue', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ + conversationHistory: [], + dueConcepts: dueConceptsWithMemories, + libraryId, + }), + }); + + if (!response.ok) { + const errorData = await response.json(); + throw new Error(errorData.error || 'Failed to start review'); + } + + const data = await response.json(); + + // Process any concepts probed in the opening + if (data.concepts_probed?.length > 0) { + processConceptsProbed(data.concepts_probed); + } + + setMessages([{ + role: 'assistant', + content: data.message, + conceptsProbed: data.concepts_probed, + }]); + + setReviewComplete(data.review_complete); + } catch (error) { + console.error('Error starting review:', error); + setMessages([ + { + role: 'assistant', + content: error instanceof Error + ? `Error: ${error.message}` + : 'Sorry, I encountered an error starting the review.', + }, + ]); + } finally { + setIsLoading(false); + } + }; + + const processConceptsProbed = (conceptsProbed: ConceptProbed[]) => { + const newAssessments: ConceptAssessment[] = conceptsProbed.map(cp => ({ + conceptId: cp.conceptId, + understanding: cp.understanding, + newMemory: cp.new_memory, + })); + + setAllAssessments(prev => [...prev, ...newAssessments]); + }; + + const sendMessage = async () => { + const currentInput = input.trim(); + if (!currentInput || isLoading) return; + + const userMessage: Message = { role: 'user', content: currentInput }; + const updatedMessages = [...messages, userMessage]; + setMessages(updatedMessages); + setInput(''); + setIsLoading(true); + + // Prepare due concepts with memories + const dueConceptsWithMemories = dueConcepts.map(dc => ({ + conceptId: dc.conceptId, + conceptName: conceptNames.get(dc.conceptId) || dc.conceptId, + retrievability: dc.retrievability, + memories: dc.state.memories, + })); + + try { + const conversationHistory = updatedMessages.map(msg => ({ + role: msg.role, + content: msg.content, + })); + + const response = await fetch('/api/review-dialogue', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ + conversationHistory, + dueConcepts: dueConceptsWithMemories, + libraryId, + }), + }); + + if (!response.ok) { + const errorData = await response.json().catch(() => ({})); + throw new Error(errorData.error || `API error: ${response.status}`); + } + + const data = await response.json(); + + // Process any concepts probed + if (data.concepts_probed?.length > 0) { + processConceptsProbed(data.concepts_probed); + } + + setMessages([...updatedMessages, { + role: 'assistant', + content: data.message, + conceptsProbed: data.concepts_probed, + }]); + + if (data.review_complete) { + setReviewComplete(true); + } + } catch (error) { + console.error('Error sending message:', error); + const errorMessage = error instanceof Error ? error.message : 'Unknown error occurred'; + setMessages([ + ...updatedMessages, + { + role: 'assistant', + content: `⚠️ **Error:** ${errorMessage}\n\nPlease try again.`, + }, + ]); + } finally { + setIsLoading(false); + } + }; + + const handleKeyPress = (e: React.KeyboardEvent) => { + if (e.key === 'Enter' && !e.shiftKey) { + e.preventDefault(); + sendMessage(); + } + }; + + const handleFinishReview = () => { + // Save all assessments to localStorage + if (allAssessments.length > 0) { + const updatedStates = batchUpdateFSRS(conceptStates, allAssessments); + saveConceptStates(libraryId, updatedStates); + console.log('💾 Saved review assessments:', allAssessments.length); + } + + // Notify parent + if (onReviewComplete) { + onReviewComplete(allAssessments); + } + + onOpenChange(false); + }; + + if (dueConcepts.length === 0) return null; + + // Calculate review progress + const conceptsReviewed = new Set(allAssessments.map(a => a.conceptId)); + const progressPercent = Math.round((conceptsReviewed.size / dueConcepts.length) * 100); + + return ( + + + + Review Session + + {dueConcepts.length} concept{dueConcepts.length > 1 ? 's' : ''} to review + + + + {/* Progress indicator - matches SocraticDialogue style */} +
+
+ Progress: + + {conceptsReviewed.size} / {dueConcepts.length} concepts reviewed + +
+
+
+
+ + {/* Concept pills showing status */} +
+ {dueConcepts.map(dc => { + const isReviewed = conceptsReviewed.has(dc.conceptId); + const assessment = allAssessments.find(a => a.conceptId === dc.conceptId); + const understanding = assessment?.understanding; + + return ( + = 0.7 + ? 'bg-green-100 text-green-700' + : understanding && understanding >= 0.4 + ? 'bg-amber-100 text-amber-700' + : 'bg-red-100 text-red-700' + : 'bg-slate-200 text-slate-600' + }`} + > + {conceptNames.get(dc.conceptId) || dc.conceptId} + {isReviewed && understanding !== undefined && ( + {Math.round(understanding * 100)}% + )} + + ); + })} +
+ + {reviewComplete && ( +
+
+ + ✅ Review session complete! + + +
+ + {/* Show next review schedule */} + {allAssessments.length > 0 && ( +
+

📅 Next Reviews Scheduled:

+
+ {allAssessments.map((assessment) => { + const currentState = dueConcepts.find(dc => dc.conceptId === assessment.conceptId)?.state; + const currentStability = currentState?.stability || 1; + const newStability = calculateNextInterval( + currentStability, + assessment.understanding, + currentState?.difficulty || 0.3 + ); + const daysUntilNext = Math.round(newStability); + + return ( +
+ + {conceptNames.get(assessment.conceptId) || assessment.conceptId} + + + {daysUntilNext === 1 ? 'Tomorrow' : `in ${daysUntilNext} days`} + +
+ ); + })} +
+
+ )} +
+ )} +
+ + {/* Messages area - matches SocraticDialogue style */} +
+ {messages.map((msg, idx) => ( +
+
+
+
+ + {String(children).replace(/\n$/, '')} + + ) : ( + + {children} + + ); + }, + }} + > + {msg.content} + +
+
+
+
+ ))} + {isLoading && ( +
+
+

Thinking...

+
+
+ )} +
+
+ + {/* Input area - matches SocraticDialogue style */} +
+
+