New cleanup_jobs command to remove async jobs and related files

[stefanorosanelli]

This PR introduces a new cleanup_jobs command that removes async jobs and their associated files from the system before a specified date. The feature provides both dry-run functionality and actual deletion capabilities.

• Adds a CLI command to clean up old async jobs and related files with date-based filtering
• Implements file cleanup functionality for both local filesystem and S3 storage
• Provides comprehensive test coverage for the new cleanup functionality

[Copilot]

Pull Request Overview

This PR introduces a new cleanup_jobs command that removes async jobs and their associated files from the system before a specified date. The feature provides both dry-run functionality and actual deletion capabilities.

• Adds a CLI command with date-based filtering and dry-run support
• Implements file cleanup functionality for both local filesystem and S3 storage
• Provides comprehensive test coverage for the new cleanup functionality

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
pyproject.toml	Adds the new cleanup_jobs command entry point
brevia/commands.py	Implements the CLI command with date parsing and confirmation prompts
brevia/async_jobs.py	Adds core cleanup functionality with database operations and file cleanup
brevia/utilities/output.py	Implements file cleanup methods for both local and S3 storage
tests/test_commands.py	Comprehensive tests for the CLI command functionality
tests/test_async_jobs.py	Tests for the core cleanup async jobs functionality
tests/utilities/test_output.py	Tests for file cleanup methods and S3 operations

Comments suppressed due to low confidence (1)

brevia/commands.py:197

• There is no test coverage for the case where the cleanup_async_jobs function throws an exception. Consider adding error handling or tests for failure scenarios.

    num = cleanup_async_jobs(before_date=before_date, dry_run=dry_run)