fix: issue with path is returned from download_files is always set t… #731
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR fixes a bug where the download_files method in the parquet input handler was not returning the downloaded file path, causing the code to always use the original remote path instead of the local downloaded path.
- Removes unnecessary local
pathvariable assignment - Updates
download_filescall to not capture return value since it modifiesself.pathdirectly - Ensures
pd.read_parquet()uses the correct path after download
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| await self.download_files(self.path) | ||
| # Use pandas native read_parquet which can handle both single files and directories | ||
| return pd.read_parquet(path) | ||
| return pd.read_parquet(self.path) |
There was a problem hiding this comment.
The fix assumes that download_files modifies self.path in-place, but this behavior is not evident from the code shown. If download_files doesn't update self.path to point to the downloaded local file, this will still read from the original remote path. Consider either ensuring download_files updates self.path or capturing its return value if it returns the local path.
Copilot uses AI. Check for mistakes.
application_sdk/inputs/parquet.py
Outdated
| if self.input_prefix and self.path: | ||
| path = await self.download_files(self.path) | ||
| await self.download_files(self.path) | ||
| # Use pandas native read_parquet which can handle both single files and directories | ||
| return pd.read_parquet(path) | ||
| return pd.read_parquet(self.path) |
There was a problem hiding this comment.
The fix assumes that download_files modifies self.path in-place, but this behavior is not evident from the code shown. If download_files doesn't update self.path to point to the downloaded local file, this will still read from the original remote path. Consider either ensuring download_files updates self.path or capturing its return value if it returns the local path.
Copilot uses AI. Check for mistakes.
Changelog
path was always set to none as the underlying methods inside
download_filesis returning NoneAdditional context (e.g. screenshots, logs, links)
Checklist
Copyleft License Compliance