Ignore Azure paths when validating directories #179
Conversation
- Update `FormatDirectoryPathEvaluator` and `FormatFilePathEvaluator` to detect Azure paths (scheme == 'az'). - Skip directory/file checks for Azure paths, since Azure Blob Storage does not have true directories. - This prevents false validation failures for Azure storage paths.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
- Update evaluators to check for Azure paths early using value.startsWith('az://')
- Skip validation for Azure paths when nf-azure plugin is missing
- Add comprehensive tests for Azure path validation
- Ensure non-Azure cloud paths (S3, GCS) are still validated normally
This allows pipelines using Azure storage paths to pass validation even when
the nf-azure plugin is not available in the test environment.
Previously, the validators for directory and file path formats would bypass validation and return success if the nf-azure plugin was missing and the path was an Azure path. This commit removes that bypass, so that missing plugin errors will now result in a validation failure, providing clearer feedback to users when the plugin is not installed or configured.
I can't seem to import the nf-azure plugin, have you got any ideas? |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
CHANGELOG.md
Outdated
| ## Bug fixes | ||
|
|
||
| 1. Fixed a bug where non-local samplesheets couldn't be validated and converted. | ||
| 2. Fixed a bug where Azure blob storage paths were incorrectly validated as directories. |
There was a problem hiding this comment.
Can you add this version 2.6.0?
There was a problem hiding this comment.
Done with c844ee0
It should do this automatically :/ I'll take a look when I find some time |
It's the use of I will roll it back to use simple string matching ( |
|
|
||
| // Skip validation if it's a directory | ||
| if(file.isDirectory() || value.startsWith('az://')) { | ||
| log.debug("Could not validate the file ${file.toString()} - path is a directory") |
There was a problem hiding this comment.
Bug: Azure Path Validation Skips Files
The condition if(file.isDirectory() || value.startsWith('az://')) skips schema validation for all Azure paths, including files. This contradicts the goal of validating Azure file paths and only skipping validation for directories. Additionally, the log message "path is a directory" is misleading when an Azure file path triggers this condition.
This pull request updates the file and directory path evaluators to better support Azure blob storage paths. The main improvement is that directory checks are now skipped for Azure paths, since Azure blob storage does not have true directories. This change ensures that validation works correctly for Azure file and directory references. Additionally, a new test verifies the correct handling of Azure storage file paths.
Azure path handling improvements:
FormatDirectoryPathEvaluator,FormatFilePathEvaluator, andFormatFilePathPatternEvaluatorto skip directory checks for Azure storage paths by inspecting the URI scheme. This avoids incorrect validation failures for Azure blob paths, which do not have true directories. [1] [2] [3]SchemaEvaluatorso that validation is only skipped for directories on non-Azure paths, allowing Azure file paths to be validated as expected.Testing:
ValidateParametersTest.groovyto ensure Azure storage file and directory paths are validated without errors.Fixes #16