Skip to content

TODO.md

Latest commit

 

History

History
220 lines (161 loc) · 6.11 KB

TODO.md

File metadata and controls

220 lines (161 loc) · 6.11 KB

📋 TODO - Distributed AI Inference Platform

Last updated: February 2, 2026 Current completion: ~92%

🔴 HIGH PRIORITY

1. Code TODOs (In Source Files)

File Line Description
services/api-gateway/internal/handlers/inference.go 279 Query metadata service or database for job status (currently returns mock)

2. Missing Unit Tests

These source files have no corresponding test files:

API Gateway

  • services/api-gateway/internal/handlers/inference_test.go
  • services/api-gateway/internal/middleware/metrics_test.go
  • services/api-gateway/internal/middleware/logging_test.go
  • services/api-gateway/internal/middleware/tracing_test.go
  • services/api-gateway/internal/observability/metrics_test.go
  • services/api-gateway/internal/observability/tracing_test.go
  • services/api-gateway/internal/config/config_test.go

Inference Orchestrator

  • services/inference-orchestrator/internal/handlers/inference_test.go
  • services/inference-orchestrator/internal/config/config_test.go

Model Router

  • services/model-router/internal/handlers/route_test.go
  • services/model-router/internal/config/config_test.go

Batch Worker

  • services/batch-worker/internal/storage/minio_test.go
  • services/batch-worker/internal/config/config_test.go

Metadata Service

  • services/metadata-service/internal/models/model_test.go
  • services/metadata-service/internal/repository/model_repository_test.go
  • services/metadata-service/internal/config/config_test.go

🟡 MEDIUM PRIORITY

3. Integration Tests (Require Full Stack)

The integration tests exist but need running infrastructure:

# Start infrastructure first
docker-compose up -d

# Then run without -short flag
cd tests && go test ./... -v

Tests to verify:

  • tests/integration/batch_processing_test.go - Batch job end-to-end
  • tests/integration/circuit_breaker_test.go - Circuit breaker behavior
  • tests/integration/model_registry_test.go - Model CRUD operations
  • tests/integration/realtime_inference_test.go - Real-time inference flow
  • tests/e2e/full_pipeline_test.go - Complete system test

4. Load Testing Scripts

Existing:

  • ✅ scripts/loadtest/inference.js - Real-time inference load test

Missing:

  • scripts/loadtest/batch-inference.js - Batch job load testing
  • scripts/loadtest/model-registry.js - Model registry stress test
  • scripts/loadtest/stress-test.js - Sustained high-load test

5. Security Configuration

  • .golangci.yml - Go linter configuration
  • .github/workflows/security.yml - Security scanning workflow
    • Trivy for container scanning
    • gosec for Go security analysis
    • Dependabot for dependency updates

🟢 LOW PRIORITY (Nice to Have)

6. Documentation

Architecture Decision Records (ADRs)

  • docs/adr/001-microservices-architecture.md
  • docs/adr/002-circuit-breaker-pattern.md
  • docs/adr/003-kafka-for-batch-jobs.md
  • docs/adr/004-redis-caching-strategy.md

Runbooks

  • docs/runbooks/deployment.md
  • docs/runbooks/troubleshooting.md
  • docs/runbooks/monitoring.md
  • docs/runbooks/disaster-recovery.md

API Documentation Enhancements

  • Swagger UI integration
  • Postman collection (docs/postman/collection.json)

7. Observability

Grafana Dashboards

  • config/grafana/dashboards/api-gateway.json
  • config/grafana/dashboards/model-router.json
  • config/grafana/dashboards/batch-worker.json
  • config/grafana/dashboards/inference-orchestrator.json
  • config/grafana/dashboards/metadata-service.json
  • config/grafana/datasources.yml

Alerting Rules

  • config/prometheus/alerts.yml - Prometheus alerting rules

8. Code Quality

  • .editorconfig - Editor configuration
  • Pre-commit hooks configuration
  • Code coverage thresholds in CI

✅ COMPLETED

Services (100%)

  • ✅ API Gateway
  • ✅ Model Router
  • ✅ Inference Orchestrator
  • ✅ Batch Worker
  • ✅ Metadata Service

Infrastructure (100%)

  • ✅ Docker Compose setup
  • ✅ Kubernetes manifests (Deployments, Services, HPAs)
  • ✅ All Dockerfiles

Tests (Partial)

  • ✅ API Gateway middleware tests (auth, cors, ratelimit)
  • ✅ API Gateway handler tests (health)
  • ✅ Model Router tests (router)
  • ✅ Inference Orchestrator tests (triton client)
  • ✅ Batch Worker tests (consumer, worker pool, postgres storage)
  • ✅ Metadata Service tests (cache, handlers)

CI/CD (100%)

  • ✅ GitHub Actions workflow
  • ✅ Multi-service build pipeline

Documentation (Partial)

  • ✅ README.md
  • ✅ GETTING_STARTED.md
  • ✅ CONTRIBUTING.md
  • ✅ OpenAPI specs for all services

📊 Completion Summary

Category Status Notes
Core Services ✅ 100% All 5 microservices
Infrastructure ✅ 100% Docker, K8s, CI/CD
Unit Tests ⚠️ 60% 16 test files missing
Integration Tests ⚠️ 80% Tests exist, need infra
Load Tests ⚠️ 33% 1 of 3 scripts
API Docs ✅ 90% OpenAPI done, Swagger UI missing
Observability ⚠️ 70% Prometheus ✅, Grafana ❌
Security ⚠️ 50% Auth ✅, Scanning ❌

🚀 Quick Start Commands

Run Unit Tests

# All services
make test

# Individual service
cd services/api-gateway && go test ./... -v

Run Integration Tests

# Start infrastructure
docker-compose up -d postgres redis kafka minio jaeger prometheus

# Build and start services
docker-compose up -d --build

# Run tests (without -short flag)
cd tests && go test ./... -v -timeout 120s

Run Load Tests

# Install k6
brew install k6  # macOS
# or: sudo apt-get install k6  # Linux

# Run load test
k6 run scripts/loadtest/inference.js

📝 Notes

  1. For job interviews: Focus on unit tests and integration tests. Skip Grafana dashboards and ADRs.

  2. Production readiness: Add security scanning and Grafana dashboards before production deployment.

  3. Test coverage goal: Aim for 80%+ coverage on core business logic.