Background Task Queue & Webhook SystemBackground Task Queue & Webhook…

[ChukwuemekaP1]

Implementation Complete: Background Task Queue & Webhook System (Closes #71)

This PR introduces a background task processing system for SoroScope, enabling non-blocking job execution with webhook notifications.

Summary of Changes

New/Updated Files

• core/src/jobs.rs – Background task queue implementation
• core/src/main.rs – Integrated job system and API routes
• core/Cargo.toml – Added required dependencies (dashmap, uuid)

---

Key Features Implemented

• Background job queue using DashMap for thread-safe in-memory tracking
• POST /jobs/submit – Submit job and receive immediate job_id
• GET /jobs/{id} – Retrieve job status
• Webhook callback support upon job completion (success or failure)
• Job cancellation support
• Configurable job timeouts and automatic cleanup of expired jobs

---

Job Lifecycle

• Queued – Awaiting processing
• Processing – Actively running
• Completed – Finished successfully
• Failed – Execution error
• Cancelled – Terminated by user

---

Architecture & Design

• Thread-safe state management via DashMap
• Integrated with existing Arc<AppState> pattern
• Background processing using tokio::spawn
• Webhook delivery executed asynchronously to prevent blocking
• Consistent error handling using AppError
• Secured with existing JWT authentication
• OpenAPI documentation added for new endpoints

---

Reliability Enhancements

• Timeout protection for webhook delivery
• Periodic cleanup of expired jobs
• Isolated job failures to prevent system-wide impact
• Safe concurrent access

Close #71

[ayomideadeniran]

@ChukwuemekaP1 resolve conflict.

[ayomideadeniran]

@ChukwuemekaP1 and make sure the ci/cd pipeline pass.

[ChukwuemekaP1]

Done @ayomideadeniran

Sorry for the delay

[ayomideadeniran]

@ChukwuemekaP1 your checks are failing.

[ayomideadeniran]

@ChukwuemekaP1 make sure they pass.

[ayomideadeniran]

@ChukwuemekaP1 Resolve conflicts.

[ChukwuemekaP1]

@ayomideadeniran
Doing that now
Apologies for the delay

[ayomideadeniran]

@ChukwuemekaP1 Fix conflicts.

[ChukwuemekaP1]

@ayomideadeniran FIx
Thanks

[ChukwuemekaP1]

@ayomideadeniran I know its late, pls close before its close

[ayomideadeniran]

@ChukwuemekaP1 Make sure all the checks pass. Now one is failing.