feat: file upload security, provider schema change detection, preferences versioning, and ML transaction classification

[Sundayabel222]

Overview

This PR implements four independent features across four commits:

1. Antivirus scanning for uploads — security pipeline for every file upload
2. Provider API schema change detection — monitoring for silent contract changes
3. User preferences versioning — concurrency control + audit trail + change webhooks
4. ML transaction type classification — auto-categorisation of transactions

---

1. Antivirus scanning for uploads (3e19c2e)

Problem: Uploads (KYC documents, dispute evidence) are accepted based on extension/MIME claims only; malicious files could be stored without any scanning.

Changes:

• New FileSecurityService (src/services/fileSecurityService.ts) with a full pipeline:
  • Antivirus scanning — ClamAV (clamd INSTREAM protocol) when CLAMAV_HOST/CLAMAV_PORT are configured, falling back to an embedded signature scanner (EICAR, executables, dangerous scripts, PDF JavaScript) otherwise
  • File type validation beyond extension — magic-byte sniffing (PDF/PNG/JPEG/GIF/ZIP) that catches spoofed extensions and polyglots
  • Quarantine period before approval — inconclusive scans are held for a configurable period (UPLOAD_QUARANTINE_MINUTES, default 24h) with auto-release of expired quarantines and admin approve/reject endpoints
  • File integrity checking — SHA-256 hash recorded for every upload, verified on approval (tamper detection)
• New upload_security_records table + rollback migration
• Wired into the KYC document upload route: infected/spoofed files are rejected before reaching S3; scan outcome + hash are recorded in S3 metadata
• 27 unit tests covering scanning, sniffing, integrity, quarantine lifecycle and gating

Acceptance criteria:

• ✅ Implement antivirus scanning for uploads
• ✅ Add file type validation beyond extension
• ✅ Create quarantine period before approval
• ✅ Implement file integrity checking
• ✅ Add security scanning test coverage

Env vars: CLAMAV_HOST, CLAMAV_PORT, CLAMAV_TIMEOUT_MS, UPLOAD_QUARANTINE_MINUTES

---

2. Provider API schema change detection (1d1167e)

Problem: Provider API contracts can change silently and break integrations with no monitoring.

Changes:

• New ProviderSchemaMonitorService (src/services/providerSchemaMonitorService.ts):
  • Schema capture + change detection — canonicalised JSON Schema snapshots hashed (SHA-256) and diffed against the last recorded version
  • Breaking change classification — removed required fields, type changes and removed enum values are flagged as breaking; additive optional fields are non-breaking
  • Breaking change notification system — structured logs (error level for breaking) + optional webhook via PROVIDER_SCHEMA_ALERT_WEBHOOK_URL
  • Schema versioning tracking — every capture persisted with semver (MAJOR on breaking, MINOR on additive), full history queryable
• New provider_api_schema_versions table + rollback migration
• 20 unit tests covering diffing, breaking classification, version bumps, alert delivery and the full monitoring pipeline

Acceptance criteria:

• ✅ Implement provider API schema change detection
• ✅ Send alert when provider APIs change
• ✅ Create breaking change notification system
• ✅ Add schema versioning tracking
• ✅ Create test coverage for change detection

Env vars: PROVIDER_SCHEMA_ALERT_WEBHOOK_URL

---

3. User preferences versioning (e2c52c0)

Problem: User preferences had no versioning, so concurrent sessions could silently overwrite each other's changes.

Changes:

• Version tracking — UserSettings now carries a monotonic version, incremented on every mutation
• Conflict resolution for concurrent changes — updateSettings supports optimistic concurrency via expectedVersion; stale updates return a 409 conflict with the current state instead of clobbering. Explicit resolveSettingsConflict strategies: server-wins, client-wins, merge
• Preference change audit trail — append-only preference_change_log table records every update/reset/delete with version transitions and changed fields (PreferenceChangeService)
• Webhook for preference change notifications — preference.changed events enqueued into the shared webhook_outbox for at-least-once delivery with retries
• PATCH endpoint accepts expectedVersion; conflict responses carry the latest settings so clients can re-sync
• 47 unit tests covering versioning, conflicts, resolution strategies, audit trail and webhook enqueueing

Acceptance criteria:

• ✅ Add version tracking to user preferences
• ✅ Implement conflict resolution for concurrent changes
• ✅ Create preference change audit trail
• ✅ Add webhook for preference change notifications
• ✅ Create test coverage for preference conflicts

---

4. ML transaction type classification (27ce618)

Problem: Transaction types are manually assigned; ML classification can auto-categorise transactions for better reporting.

Changes:

• New TransactionClassifierService (src/services/transactionClassifierService.ts) — multinomial Naive Bayes classifier with Laplace smoothing over 6 categories (deposit, withdraw, payment, payout, refund, fee), seeded with domain priors so it works before any training data
• Classification confidence scoring — normalised posterior probability per category
• Training data collection pipeline — recordTrainingExample persists labelled samples from real transactions (transaction_classifier_training) and folds them into the live model
• Human feedback loop — submitHumanFeedback stores corrections (transaction_classifier_feedback) and applies them to the model immediately via online learning; trainModel performs batch retraining and persists versioned model snapshots
• Classification accuracy monitoring — getClassificationAccuracy evaluates against human-labelled samples and exports Prometheus metrics (accuracy gauge, per-category counters, confidence histogram, feedback/training-sample counts)
• New transaction_classifier_training / transaction_classifier_feedback / transaction_classifier_models tables + rollback migration
• 17 unit tests covering feature extraction, classification, confidence scoring, online learning, training pipeline, feedback loop and accuracy monitoring

Acceptance criteria:

• ✅ Implement transaction type classification model
• ✅ Add training data collection pipeline
• ✅ Create classification confidence scoring
• ✅ Implement human feedback loop for training improvement
• ✅ Add classification accuracy monitoring

---

Testing

• 111 new unit tests across the four features, all passing
• Full-suite regression run: zero new failures vs main (remaining failures are pre-existing environment issues — missing native sharp binary, no live Postgres/Redis)
• tsc --noEmit: no new type errors (20 pre-existing errors in src/stellar/sep02.ts unchanged)
• ESLint: no errors (only a few pre-existing-style any warnings)

Migrations

4 new migrations (all with rollback files):

• 20260825_create_upload_security_records(.down).sql
• 20260825_create_provider_api_schema_versions(.down).sql
• 20260825_create_preference_change_log(.down).sql
• 20260825_create_transaction_classifier_tables(.down).sql

Run with npm run migrate:up.

[drips-wave]

Hey @Sundayabel222! 👋 It looks like this PR isn't linked to any issue.

If this PR is for one of the issues assigned to you as part of a Wave, please link it to ensure your contribution is tracked properly. You can do this by adding a keyword to the PR description (e.g., Closes #123), or by clicking a button below:

Issue	Title
#425	Add Transaction Type Classification Machine Learning	Link to this issue
#429	Add Provider Contract Change Notification	Link to this issue
#430	Implement Secure File Upload Quarantine	Link to this issue
#428	Implement User Preference Versioning	Link to this issue

ℹ️ Learn more about linking PRs to issues