load functions + pscale postgres

[bmdavis419]

Greptile Overview

Greptile Summary

This PR migrates the database from MySQL to PostgreSQL (PlanetScale → standard Postgres) and refactors data loading from client-side remote functions to server-side SvelteKit load functions.

Key Changes:

• Migrated database schema from MySQL to PostgreSQL with proper type conversions (datetime → timestamp, int → integer, mysqlEnum → pgEnum)
• Replaced client-side auth store with server-side form actions and session management
• Moved data fetching from remote functions to +page.server.ts load functions using the new runner Effect runtime
• Simplified comment bulk operations by replacing separate insert/update logic with PostgreSQL's onConflictDoUpdate upsert
• Switched AI provider from OpenRouter to Groq SDK with retry logic (3 attempts, 1 minute spacing)
• Added new CLI tools for database management (@r8y/db-cli) and backfilling (@r8y/backfill-cli)
• Improved backfill progress tracking using Effect's Ref for concurrent counter updates
• Removed local MySQL docker-compose setup
• Updated environment variables from MYSQL_URL to DATABASE_URL and OPENROUTER_API_KEY to GROQ_API_KEY

Architecture Impact:
The data loading pattern now follows SvelteKit best practices with server-side load functions, reducing client-side JavaScript and enabling proper SSR. The Effect runtime is properly managed with shutdown handlers.

Confidence Score: 4/5

• Safe to merge with one SQL injection risk that should be addressed
• The migration is well-executed with proper PostgreSQL type conversions, clean refactoring from client-side to server-side data loading, and good use of Effect patterns. The score is 4 instead of 5 due to a potential SQL injection vulnerability in the upsert logic using sql.raw with template interpolation instead of sql tagged templates.
• packages/channel-sync/src/db/index.ts requires attention for the SQL injection risk in the upsert logic

Important Files Changed

File Analysis

Filename	Score	Overview
packages/db/src/schema.ts	5/5	Migrated from MySQL to PostgreSQL - changed imports from drizzle-orm/mysql-core to drizzle-orm/pg-core, replaced datetime with timestamp, int with integer, and mysqlEnum with pgEnum. All index names updated to be unique across tables.
packages/db/src/connection.ts	5/5	Changed database driver from drizzle-orm/mysql2 to drizzle-orm/node-postgres, removed MySQL-specific mode: 'default' configuration.
packages/db/src/cli.ts	5/5	New CLI tool for database management with wipe, push, and seed commands using Effect CLI. Properly handles database connection cleanup with acquireRelease and client termination.
apps/web/src/lib/server/runner.ts	5/5	New server-side Effect runtime manager for handling Effect-based operations in SvelteKit. Provides runner and authedRunner functions that convert Effect errors to SvelteKit error responses, with proper shutdown handlers.
apps/web/src/routes/+page.svelte	5/5	Removed client-side auth store, migrated to server-side form actions with SvelteKit's use:enhance for signin, added loading state and server-side error display.
apps/web/src/routes/app/+page.server.ts	5/5	New server load function using authedRunner to fetch channels data on the server-side, replacing client-side remote functions. Added signout action.
packages/channel-sync/src/db/index.ts	4/5	Changed env var from MYSQL_URL to DATABASE_URL, replaced complex insert/update logic with PostgreSQL upsert using onConflictDoUpdate. Uses sql.raw for excluded values which may have PostgreSQL-specific behavior.
packages/channel-sync/src/ai/index.ts	5/5	Switched AI provider from OpenRouter to Groq SDK, added retry schedule with 3 attempts spaced 1 minute apart for AI operations.

Sequence Diagram

sequenceDiagram
    participant User
    participant Browser
    participant SvelteKit as SvelteKit Server
    participant Runner as Effect Runner
    participant DbService
    participant AuthService
    participant PostgreSQL as PostgreSQL DB
    
    Note over User,PostgreSQL: Authentication Flow
    User->>Browser: Enter password
    Browser->>SvelteKit: POST /signin (form action)
    SvelteKit->>Runner: runner(effect)
    Runner->>AuthService: signIn(event, password)
    AuthService->>SvelteKit: Set session cookie
    SvelteKit->>Browser: Redirect to /app
    
    Note over User,PostgreSQL: Server-Side Data Loading
    Browser->>SvelteKit: GET /app
    SvelteKit->>Runner: authedRunner(event, fn)
    Runner->>AuthService: checkAuthAndFail(event)
    AuthService-->>Runner: Authenticated
    Runner->>DbService: getChannelsWithStats()
    DbService->>PostgreSQL: SELECT with aggregations
    PostgreSQL-->>DbService: Channel data
    DbService-->>Runner: Effect.succeed(data)
    Runner-->>SvelteKit: Return channels
    SvelteKit->>Browser: Render page with data
    
    Note over User,PostgreSQL: Background Sync (channel-sync package)
    participant BgWorker as Background Worker
    participant YouTube as YouTube API
    participant Groq as Groq AI
    
    BgWorker->>DbService: getAllChannels()
    DbService->>PostgreSQL: SELECT channels
    PostgreSQL-->>DbService: Channels list
    BgWorker->>YouTube: getLatestVideos()
    YouTube-->>BgWorker: Video metadata
    BgWorker->>Groq: classifyComment() [with retry]
    Groq-->>BgWorker: Classification result
    BgWorker->>DbService: bulkUpsertComments()
    DbService->>PostgreSQL: INSERT ... ON CONFLICT DO UPDATE
    PostgreSQL-->>DbService: Success

Loading

[greptile-apps]

_{64 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}