-
Notifications
You must be signed in to change notification settings - Fork 3
Add downloadable services and tasks file outputs #97
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add downloadable services and tasks file outputs #97
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR implements file output functionality for services tasks, including support for generating downloadable txt files and handling S3 uploads. Key changes include:
- New tests for file output and download endpoints are added.
- A new service (RefineTextAnalysisToTxtService) is implemented in services.py leveraging PublicFileOutput.
- Settings, routers, and async jobs are updated to support file output operations.
Reviewed Changes
Copilot reviewed 13 out of 14 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/utilities/test_output.py | Added tests to validate both local and S3 file output paths. |
| tests/test_services.py | Added tests for the new RefineTextAnalysisToTxtService service. |
| tests/routers/test_download_router.py | Introduced tests for download endpoints handling file download scenarios. |
| tests/conftest.py | Updated fixture with file output base path configuration. |
| brevia/utilities/output.py | Added PublicFileOutput class with S3 upload and file writing logic. |
| brevia/settings.py | Configured default settings for file output paths and URLs. |
| brevia/services.py | Implemented RefineTextAnalysisToTxtService leveraging file output. |
| brevia/routers/download_router.py | Created endpoint support for file downloads with error handling for S3. |
| brevia/routers/app_routers.py | Included download_router in application router configuration. |
| brevia/routers/analyze_router.py | Set default file_name in payload if not provided. |
| brevia/async_jobs.py | Updated async jobs to include job_id in the payload. |
Files not reviewed (1)
- tests/files/1234/test.txt: Language not supported
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds file output functionality for services tasks and associated endpoints, along with new tests to cover local and S3 file outputs. Key changes include:
- Implementation of PublicFileOutput for handling file paths and S3 uploads.
- Addition of RefineTextAnalysisToTxtService to generate a txt file output.
- New download router endpoints and accompanying tests for file retrieval.
Reviewed Changes
Copilot reviewed 14 out of 15 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/utilities/test_output.py | Tests covering file_path, file_url, and write functionality |
| tests/test_services.py | Tests for new RefineTextAnalysisToTxtService service |
| tests/routers/test_download_router.py | Tests to verify download endpoint responses for local/S3 paths |
| tests/conftest.py | Updates settings with file output base path |
| brevia/utilities/output.py | Implements file output operations and S3 upload logic |
| brevia/settings.py | Updates default file output path and URL settings |
| brevia/services.py | Introduces RefineTextAnalysisToTxtService service |
| brevia/routers/download_router.py | Adds endpoint for file download |
| brevia/routers/app_routers.py | Registers the download router |
| brevia/routers/analyze_router.py | Sets default file_name for analyze endpoint |
| brevia/base_retriever.py | Adjusts type signature for async retriever methods |
| brevia/async_jobs.py | Injects job_id into payload before service execution |
Files not reviewed (1)
- tests/files/1234/test.txt: Language not supported
Co-authored-by: Copilot <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds file output functionality and tests for service tasks, including generating file paths/URLs, handling S3 uploads, and providing a download endpoint. Key changes include:
- Introducing LinkedFileOutput for managing file output for both local and S3 storage.
- Adding tests for output utilities, services, download router, and updating settings.
- Integrating file output download routes into the application routers.
Reviewed Changes
Copilot reviewed 14 out of 16 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/utilities/test_output.py | Tests for local and S3 file output functionality |
| tests/test_services.py | Added test for RefineTextAnalysisToTxtService and settings override |
| tests/routers/test_download_router.py | Tests covering file download responses (local and S3) |
| tests/conftest.py | Updates file output base path configuration via a fixture |
| brevia/utilities/output.py | Introduces LinkedFileOutput for file management and S3 upload logic |
| brevia/settings.py | Adds file output configuration fields |
| brevia/services.py | Adds RefineTextAnalysisToTxtService which writes output to a txt file |
| brevia/routers/download_router.py | Defines file download endpoint with S3 download restrictions |
| brevia/routers/app_routers.py | Includes the download router to the app |
| brevia/routers/analyze_router.py | Sets default file_name for file analyzing endpoints |
| brevia/base_retriever.py | Updates to async callback type hints |
| brevia/async_jobs.py | Sets job_id in the job payload for later retrieval of output |
Files not reviewed (2)
- .env.sample: Language not supported
- tests/files/1234/test.txt: Language not supported
This PR adds downloadable file output functionality for services and tasks, including generating file paths/URLs, handling S3 uploads, and providing a download endpoint.
Key changes include:
LinkedFileOutputfor managing file output for both local and S3 storage.