Skip to content

feat: Implement DataFrame management and batch deletion endpoints #461

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 9 additions & 8 deletions .kiro/specs/dataframe-web-interface/tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,30 +33,31 @@
- Create error handling for invalid expressions and operations
- _Requirements: 3.1, 3.2, 3.3, 3.4, 3.5, 3.6_

- [ ] 2. Create DataFrame management and file upload functionality
- [x] 2. Create DataFrame management and file upload functionality
- Implement file upload handling with multiple format support
- Create DataFrame deletion and cleanup operations
- Add data loading from URLs and local files
- _Requirements: 5.4, 5.5, 5.6, 7.4, 7.5_

- [ ] 2.1 Implement file upload and data loading endpoints
- [x] 2.1 Implement file upload and data loading endpoints
- Code POST `/api/dataframes/upload` endpoint for file uploads
- Code POST `/api/dataframes/load-url` endpoint for URL-based data loading
- Implement multipart file handling and temporary file management
- Add support for CSV, JSON, Excel, and Parquet formats with options
- _Requirements: 5.4, 5.5, 5.6_

- [ ] 2.2 Implement DataFrame deletion and cleanup endpoints
- [x] 2.2 Implement DataFrame deletion and cleanup endpoints
- Code DELETE `/api/dataframes/{df_id}` endpoint for individual DataFrame deletion
- Code POST `/api/dataframes/cleanup` endpoint for expired DataFrame cleanup
- Code POST `/api/dataframes/batch-delete` endpoint for batch deletion operations
- Implement confirmation mechanisms and batch operations
- Add cleanup statistics and reporting
- _Requirements: 5.1, 5.2, 5.3, 7.4, 7.5_

- [ ] 2.3 Implement data export functionality
- [x] 2.3 Implement data export functionality
- Code POST `/api/dataframes/{df_id}/export` endpoint for data export
- Add support for CSV and JSON export formats
- Implement streaming for large dataset exports
- Add support for CSV, JSON, Excel, and Parquet export formats
- Implement file generation and base64 encoding for download
- Create export progress tracking and file download handling
- _Requirements: 6.1, 6.2, 6.3, 6.4, 6.5_

Expand Down Expand Up @@ -134,7 +135,7 @@
- Update navigation template to include DataFrame link
- _Requirements: 1.1, 1.2_

- [ ] 5.2 Integrate DataFrame API with existing API structure
- [x] 5.2 Integrate DataFrame API with existing API structure
- Import DataFrame API routes in `server/api/__init__.py`
- Add DataFrame endpoints to the main API routes list
- Ensure consistent error handling with existing APIs
Expand All @@ -147,7 +148,7 @@
- Add performance tests for large DataFrame handling
- _Requirements: All requirements for validation_

- [ ] 6.1 Implement API endpoint unit tests
- [x] 6.1 Implement API endpoint unit tests
- Create `test_dataframes_api.py` with tests for all endpoints
- Test CRUD operations, error handling, and edge cases
- Add tests for file upload and data loading functionality
Expand Down
2 changes: 2 additions & 0 deletions server/api/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@
api_execute_dataframe_operation,
api_delete_dataframe,
api_cleanup_expired_dataframes,
api_batch_delete_dataframes,
api_upload_dataframe,
api_load_dataframe_from_url,
api_export_dataframe,
Expand Down Expand Up @@ -117,6 +118,7 @@
Route("/api/dataframes", endpoint=api_list_dataframes, methods=["GET"]),
Route("/api/dataframes/stats", endpoint=api_get_storage_stats, methods=["GET"]),
Route("/api/dataframes/cleanup", endpoint=api_cleanup_expired_dataframes, methods=["POST"]),
Route("/api/dataframes/batch-delete", endpoint=api_batch_delete_dataframes, methods=["POST"]),
Route("/api/dataframes/upload", endpoint=api_upload_dataframe, methods=["POST"]),
Route("/api/dataframes/load-url", endpoint=api_load_dataframe_from_url, methods=["POST"]),
Route("/api/dataframes/{df_id}", endpoint=api_get_dataframe_detail, methods=["GET"]),
Expand Down
Loading