feat(event_handler): add File parameter support for multipart/form-data uploads in OpenAPI utility #7132
Conversation
…ploads - Add public File parameter class extending _File - Support multipart/form-data parsing with WebKit boundary compatibility - OpenAPI schema generation with format: binary for file uploads - Enhanced dependant logic to handle File + Form parameter combinations - Clean implementation based on upstream develop branch Changes: - params.py: Add File(_File) public class with proper documentation - dependant.py: Add File parameter support in body field info logic - openapi_validation.py: Add multipart parsing with boundary detection - test_file_form_validation.py: Basic test coverage for File parameters This provides customers with File parameter support using the same pattern as Query, Path, Header parameters with Annotated types.
- Add File parameter class in openapi/params.py with binary format schema - Implement comprehensive multipart/form-data parsing in openapi_validation.py * Support for WebKit and standard boundary formats * Base64-encoded request handling for AWS Lambda * Mixed file and form data parsing - Update dependant.py to handle File parameters in body field resolution - Add comprehensive test suite (13 tests) covering: * Basic file upload parsing and validation * WebKit boundary format support * Base64-encoded multipart data * Multiple file uploads * File size constraints validation * Optional file parameters * Error handling for invalid boundaries and missing files - Add file_parameter_example.py demonstrating various File parameter use cases - Clean up unnecessary imports and pragma comments Resolves file upload functionality with full OpenAPI schema generation and validation support.
pull-request-size
bot
added
the
size/XL
- Break down _parse_multipart_data method into smaller helper methods - Reduce cognitive complexity from 43 to under 15 per SonarCloud requirement - Improve code readability and maintainability - All existing tests continue to pass Helper methods created: - _decode_request_body: Handle base64 decoding - _extract_boundary_bytes: Extract multipart boundary - _parse_multipart_sections: Parse sections into data dict - _parse_multipart_section: Handle individual section parsing - _split_section_headers_and_content: Split headers/content - _decode_form_field_content: Decode form field as string Addresses SonarCloud cognitive complexity violation while maintaining all existing functionality for File parameter multipart parsing.
oyiz-michael
changed the title
oyiz-michael
changed the title
- Add missing __future__ annotations imports - Remove unused pytest imports from test files - Remove unused json import from example - Fix line length violations in test files - All File parameter tests continue to pass (13/13) Addresses ruff linting violations: - FA102: Missing future annotations for PEP 604 unions - F401: Unused imports - E501: Line too long violations
|
Hi @oyiz-michael, I see you are working on this PR and please let me know when you need a first round of review or any help. |
- Replace bytes | None with Union[bytes, None] for broader compatibility - Replace str | None with Union[str, None] in examples - Add noqa: UP007 comments to suppress linter preference for newer syntax - Ensures compatibility with Python environments that don't support PEP 604 unions - Fixes test failure: 'Unable to evaluate type annotation bytes | None' All File parameter tests continue to pass (13/13) across Python versions.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## develop #7132 +/- ##
===========================================
- Coverage 96.49% 96.43% -0.06%
===========================================
Files 275 276 +1
Lines 13110 13366 +256
Branches 985 1035 +50
===========================================
+ Hits 12650 12890 +240
- Misses 356 366 +10
- Partials 104 110 +6 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
@leandrodamascena fixing some failing test and should be ready for a review and feed back |
pull-request-size
bot
added
size/XXL
|
Hi @oyiz-michael, a quick tip: run |
- Replaced 'assert file_param is not None' with meaningful assertions - Check file_param.description and hasattr for json_schema_extra - Addresses SonarCloud quality issue S5727: comparison to None should not be constant - Improves test meaningfulness and code maintainability
- Added tests for WebKit boundary extraction in multipart parsing - Added base64 decoding error handling test coverage - Added multipart section name extraction tests - Added AttributeError handling tests for field value extraction - Targets specific missing lines identified in codecov patch report - Increases total test count from 21 to 25 comprehensive tests - All tests pass with proper formatting and linting validation - Addresses codecov target of 96.36% diff coverage
- Removed 'mock_body is not None and value is None' redundant condition - Simplified test to directly test AttributeError handling scenario - Addresses linter warning about identity check that will always be True - Maintains same test coverage while improving code quality - Follows proper make format && make pr workflow validation
|
|
Hi @oyiz-michael, a quick update here. We have several PRs making changes that directly affect the OpenAPI and the event handler, two critical utilities we have, and we can't merge all the code at once because if there's a regression, we can easily roll it back. Our coverage is good, we have e2e tests, but we never know all the edge cases. So, the order in which I'll be merging the PRs is: 1/ Today I merged this PR #7227 - and it will be included in the next release on September 25th. 2/ I'll work to merge this PR #7253 sometime after the 25th and include it in the release on October 7th. 3/ After October 7th, I will work to merge this PR and include it in the release on October 21st. Thank you very much for your work and patience. I know as a OpenSource contributor sometimes we want to have the code in production, but we care a lot do not to introduce breaking changes to our customers, and this PR of yours is a bit more complex than the others. |
leandrodamascena
added
on-hold
|
Hi @oyiz-michael! After merging the PRs I mentioned in the previous comment, I see we have some merge conflicts here. Can you fix them before we move forward? Thanks |
Merge conflicts resolved after upstream PRs aws-powertools#7227 and aws-powertools#7253: Core Changes: - Updated File class rename from File -> _File with public File alias - Fixed Union import for _resolve_field_type function - Removed unused _File import in dependant.py - Maintained all UploadFile and validation functionality Testing: - All 25 comprehensive tests passing - 96.36%+ codecov coverage maintained - Code quality checks passing (make format && make pr) Compatibility: - Preserves public File API for backward compatibility - Maintains UploadFile OpenAPI schema generation - All validation middleware features intact
- Changed back from '_File with alias' to direct 'class File(Form)' - All 25 tests passing with 96.36%+ coverage - Code quality checks passing (make format && make pr)
Add type: ignore[misc] comment to suppress mypy error about inheriting from final FieldInfo class in Pydantic. This matches the upstream pattern used for similar classes and allows the code to pass mypy type checking in Python 3.10+. - Fix mypy error: Cannot inherit from final class 'FieldInfo' - Matches upstream _File class pattern with type ignore - All tests passing (25/25) - All quality checks passing (format, lint, mypy)
- Added UploadFile-to-bytes conversion in _normalize_field_value() - Handles type annotations including Annotated[bytes, File()] - Fixes 24 failing tests in test_file_parameter.py - All tests now return 200 OK instead of 422 validation errors Resolves multipart form data parsing issue where UploadFile instances weren't being converted to bytes before Pydantic validation.
…to feature/file-parameter-clean
| # Convert UploadFile to bytes if the expected type is bytes | ||
| if isinstance(value, UploadFile): |
There was a problem hiding this comment.
This function was originally meant to normalize values, not convert them. Maybe we should keep conversions separate to avoid changing its intent.
What do you think?
There was a problem hiding this comment.
I agree. Normalization should stay free of IO/side-effects. I'll move the UploadFile→bytes logic out of the normalizer into a dedicated conversion helper that we call only when the field's expected type requires it (and add tests). If you'd prefer immediate compatibility, I can first add a flag to the normalizer to preserve current behavior and deprecate it in a follow-up. Which approach do you prefer?
There was a problem hiding this comment.
I wouldn't deprecate that part. It isn't public.
| This turns the low-level function signature into typed, validated Pydantic models for consumption. | ||
| """ | ||
|
|
||
| __all__ = ["Path", "Query", "Header", "Body", "Form", "File", "UploadFile"] |
There was a problem hiding this comment.
Adding __all__ here alters what the module exposes. Was the goal to define a stable public surface?
There was a problem hiding this comment.
yes, that was intentional: all is used here to define the stable public surface for openapi params so downstream imports and docs are explicit. I'll add a short comment explaining the intent and keep the list maintained. If you'd rather we leave it implicit, I can remove it instead.
…to ensure_upload_file_schema_references; tidy openapi package
|
Not all issues are linked correctly. Please link each issue to the PR either manually or using a closing keyword in the format If mentioning more than one issue, separate them with commas: i.e. |
|
3 participants
Issue number: #7124
closes #7124
Summary
This PR adds comprehensive File parameter support for handling file uploads in multipart/form-data requests within the AWS Lambda Powertools Python Event Handler with OpenAPI validation.
Changes
Added
Fileclass inaws_lambda_powertools/event_handler/openapi/params.pyFormwithformat: binaryin OpenAPI schemaEnhanced multipart parsing in
aws_lambda_powertools/event_handler/middlewares/openapi_validation.py_parse_multipart_datamethod for parsing multipart/form-dataComprehensive test suite with 13 test scenarios covering:
Complete usage example in
examples/event_handler_rest/src/file_parameter_example.pyUser experience
Before: Users could not handle file uploads in multipart/form-data requests with OpenAPI validation. They had to manually parse request bodies or disable validation entirely.
After: Users can now use type-annotated
Fileparameters that automatically:format: binaryChecklist
If your change doesn't seem to apply, please leave them unchecked.
Is this a breaking change?
RFC issue number: N/AThis is not a breaking change - it's a new feature addition that doesn't modify existing functionality.
Checklist:
Acknowledgment
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.