Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
184 changes: 184 additions & 0 deletions backend/DATABASE_SETUP.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,184 @@
# Database Setup Guide

This guide explains how to set up the PostgreSQL database for the PayeD backend.

## Prerequisites

You need either:

- Docker and Docker Compose (recommended), OR
- PostgreSQL installed locally

## Option 1: Using Docker Compose (Recommended)

1. **Start the database**:

```bash
cd backend
docker-compose up -d postgres
```

2. **Create .env file**:

```bash
cp .env.example .env
```

3. **Update .env with database credentials**:

```env
DATABASE_URL=postgresql://payd_user:payd_password@localhost:5432/payd_db
```

4. **Run migrations**:

```bash
npm run db:migrate
```

5. **Verify schema**:
```bash
npm run db:verify-schema
```

## Option 2: Using Local PostgreSQL

1. **Install PostgreSQL** (if not already installed):

```bash
# Ubuntu/Debian
sudo apt-get install postgresql postgresql-contrib

# macOS
brew install postgresql
```

2. **Start PostgreSQL service**:

```bash
# Ubuntu/Debian
sudo systemctl start postgresql

# macOS
brew services start postgresql
```

3. **Create database and user**:

```bash
sudo -u postgres psql
```

Then in the PostgreSQL prompt:

```sql
CREATE USER payd_user WITH PASSWORD 'payd_password';
CREATE DATABASE payd_db OWNER payd_user;
GRANT ALL PRIVILEGES ON DATABASE payd_db TO payd_user;
\q
```

4. **Create .env file**:

```bash
cd backend
cp .env.example .env
```

5. **Update .env with database credentials**:

```env
DATABASE_URL=postgresql://payd_user:payd_password@localhost:5432/payd_db
```

6. **Run migrations**:

```bash
npm run db:migrate
```

7. **Verify schema**:
```bash
npm run db:verify-schema
```

## Migration Scripts

The following npm scripts are available:

- `npm run db:migrate` - Run all pending migrations
- `npm run db:migrate:dry-run` - Preview migrations without executing
- `npm run db:verify-schema` - Verify database schema is correct

## Payroll Scheduler Migrations

The payroll scheduler feature adds two new tables:

### schedules table

Stores payroll schedule configurations including:

- Schedule frequency (once, weekly, biweekly, monthly)
- Time of day for execution
- Payment configuration (recipients, amounts, assets)
- Execution tracking (next run, last run, status)

### execution_history table

Records all schedule execution attempts including:

- Execution status (success, failed, partial)
- Blockchain transaction details
- Error information for failed executions

## Verification

After running migrations, the verify-schema script will check:

1. ✓ Tables exist (schedules, execution_history)
2. ✓ All columns are present with correct types
3. ✓ Primary key constraints are configured
4. ✓ Foreign key constraints are configured
5. ✓ Check constraints are enforced
6. ✓ Indexes are created
7. ✓ Foreign key relationships work correctly
8. ✓ Check constraints reject invalid data

## Troubleshooting

### "DATABASE_URL environment variable is not set"

- Make sure you have a `.env` file in the `backend` directory
- Verify the `.env` file contains `DATABASE_URL=postgresql://...`

### "Connection refused" or "ECONNREFUSED"

- Check if PostgreSQL is running: `docker ps` or `sudo systemctl status postgresql`
- Verify the host and port in DATABASE_URL match your PostgreSQL instance

### "database does not exist"

- Create the database using the SQL commands in Option 2, step 3
- Or use Docker Compose which creates the database automatically

### "permission denied for schema public"

- Grant privileges to your user: `GRANT ALL PRIVILEGES ON DATABASE payd_db TO payd_user;`
- Or use Docker Compose which handles permissions automatically

### Migration already applied

- Migrations are tracked in the `schema_migrations` table
- If you need to re-run a migration, delete its entry from `schema_migrations`
- Or drop and recreate the database for a clean slate

## Next Steps

After successful database setup:

1. Start the backend server: `npm run dev`
2. The server will connect to the database automatically
3. The cron job will start monitoring for due schedules
4. Use the API endpoints to create and manage schedules

For API documentation, see the main README.md file.
209 changes: 209 additions & 0 deletions backend/MIGRATION_STATUS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,209 @@
# Migration Status Report - Task 1.3

## Summary

Task 1.3 requires running migrations and verifying the schema for the payroll scheduler feature. The migration files have been created and the verification tooling is in place.

## Completed Steps

### 1. Migration Files Created (Tasks 1.1 & 1.2)

- ✅ `014_create_schedules.sql` - Creates schedules table with all required columns, constraints, and indexes
- ✅ `015_create_execution_history.sql` - Creates execution_history table with foreign key to schedules

### 2. Verification Tooling Created

- ✅ Created `src/db/verify-schema.ts` - Comprehensive schema verification script
- ✅ Added `db:verify-schema` npm script to package.json
- ✅ Fixed ESM module compatibility issues in migrate.ts and verify-schema.ts

### 3. Documentation Created

- ✅ Created `DATABASE_SETUP.md` - Complete guide for database setup with Docker or local PostgreSQL
- ✅ Created this status report

## What the Verification Script Tests

The `verify-schema.ts` script performs comprehensive checks:

1. **Table Existence**: Verifies both schedules and execution_history tables exist
2. **Column Verification**: Checks all 13 columns in schedules table and 9 columns in execution_history table
3. **Constraint Verification**:
- Primary key constraints
- Foreign key constraints (schedules → organizations, execution_history → schedules)
- Check constraints (frequency enum, status enums)
4. **Index Verification**: Checks all 6 indexes across both tables
5. **Functional Testing**:
- Tests foreign key enforcement (prevents orphaned execution_history records)
- Tests check constraint enforcement (rejects invalid frequency/status values)

## Next Steps - Database Setup Required

To complete task 1.3, a PostgreSQL database must be available. Choose one option:

### Option A: Docker Compose (Recommended)

```bash
cd backend
docker-compose up -d postgres
cp .env.example .env
# Edit .env to set DATABASE_URL=postgresql://payd_user:payd_password@localhost:5432/payd_db
npm run db:migrate
npm run db:verify-schema
```

### Option B: Local PostgreSQL

```bash
# Install and start PostgreSQL
sudo apt-get install postgresql # or brew install postgresql on macOS
sudo systemctl start postgresql

# Create database and user
sudo -u postgres psql
CREATE USER payd_user WITH PASSWORD 'payd_password';
CREATE DATABASE payd_db OWNER payd_user;
GRANT ALL PRIVILEGES ON DATABASE payd_db TO payd_user;
\q

# Configure and run migrations
cd backend
cp .env.example .env
# Edit .env to set DATABASE_URL=postgresql://payd_user:payd_password@localhost:5432/payd_db
npm run db:migrate
npm run db:verify-schema
```

### Option C: Use Existing Database

If a database is already configured:

```bash
cd backend
# Ensure .env file exists with DATABASE_URL
npm run db:migrate
npm run db:verify-schema
```

## Expected Output

### Successful Migration

```
[migrate] Starting migration runner
[migrate] Target database: postgresql://payd_user:***@localhost:5432/payd_db
[migrate] ✓ schema_migrations table ready
[migrate] Found 15 migration file(s) in /path/to/migrations
[migrate] ↷ Skipped 001_*.sql (already applied)
...
[migrate] ↷ Skipped 013_*.sql (already applied)
[migrate] ✓ Applied 014_create_schedules.sql (XX ms)
[migrate] ✓ Applied 015_create_execution_history.sql (XX ms)
─────────────────────────────────────────
[migrate] Summary (XXX ms total)
Applied : 2
Skipped : 13
Drift : 0
─────────────────────────────────────────
[migrate] Done.
```

### Successful Verification

```
[verify-schema] Starting schema verification...

1. Checking schedules table...
✓ schedules table exists

2. Verifying schedules table columns...
✓ All 13 columns present

3. Verifying schedules table constraints...
✓ Primary key constraint exists
✓ Foreign key constraint exists
✓ Check constraints exist

4. Verifying schedules table indexes...
✓ All 3 indexes present

5. Checking execution_history table...
✓ execution_history table exists

6. Verifying execution_history table columns...
✓ All 9 columns present

7. Verifying execution_history table constraints...
✓ Primary key constraint exists
✓ Foreign key constraint exists
✓ Check constraints exist

8. Verifying execution_history table indexes...
✓ All 3 indexes present

9. Testing foreign key constraints...
✓ Foreign key constraint properly enforced

10. Testing check constraints...
✓ Check constraint on frequency properly enforced

────────────────────────────────────────────────────────────
[verify-schema] ✓ All schema verifications passed!
────────────────────────────────────────────────────────────
```

## Files Modified/Created

### Modified

- `backend/src/db/migrate.ts` - Fixed ESM compatibility (added \_\_dirname polyfill)
- `backend/package.json` - Added db:verify-schema script

### Created

- `backend/src/db/verify-schema.ts` - Schema verification script
- `backend/DATABASE_SETUP.md` - Database setup guide
- `backend/MIGRATION_STATUS.md` - This status report

## Technical Details

### Migration Files

Both migration files follow PostgreSQL best practices:

- Use `IF NOT EXISTS` for idempotency
- Include proper CHECK constraints for enum validation
- Define foreign keys with CASCADE delete for referential integrity
- Create indexes on frequently queried columns
- Include metadata columns (created_at, updated_at)

### Schema Design

- **schedules table**: 13 columns, 3 indexes, 2 check constraints, 1 foreign key
- **execution_history table**: 9 columns, 3 indexes, 1 check constraint, 1 foreign key
- Both tables use SERIAL primary keys
- JSONB columns for flexible payment_config and error_details storage
- Proper timestamp tracking for execution history

## Validation Against Requirements

This implementation satisfies:

- **Requirement 1.1**: Database schema created with proper structure
- **Requirement 1.3**: Migrations are ready to execute
- **Requirement 1.4**: Next run timestamp calculation supported by schema
- **Requirement 5.4**: Execution history tracking enabled
- **Requirement 5.5**: Error tracking columns included

## Current Status

✅ **Ready for Execution** - All migration files and verification tools are prepared
⏸️ **Waiting for Database** - Requires PostgreSQL database connection to proceed

Once a database is available, run:

```bash
npm run db:migrate && npm run db:verify-schema
```

This will complete task 1.3 and verify the schema is correctly implemented.
Loading