Fix storage and containment validation for registerView#4963
Conversation
There was a problem hiding this comment.
Pull request overview
This PR closes a storage-governance gap in LocalIcebergCatalog.registerView by adding the same location/containment validation that already exists for table registration, preventing view registration from reading user-supplied metadata from disallowed locations.
Changes:
- Add
validateLocationForTableLike(...)toregisterViewbefore loading/reading the metadata file. - Add
validateMetadataFileInTableDir(...)after parsing view metadata to ensure the metadata file is contained under the view’s declared location. - Add targeted tests asserting
registerViewrejects (1) metadata in disallowed storage locations and (2) metadata files outside the view directory.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/LocalIcebergCatalog.java | Adds pre-read location validation and post-parse containment validation to registerView. |
| runtime/service/src/test/java/org/apache/polaris/service/catalog/iceberg/AbstractLocalIcebergCatalogViewTest.java | Adds regression tests for disallowed metadata locations and metadata-file containment enforcement. |
| @@ -1025,6 +1028,7 @@ public View registerView(TableIdentifier identifier, String metadataFileLocation | |||
|
|
|||
| InputFile metadataFile = fileIO.newInputFile(metadataFileLocation); | |||
| ViewMetadata metadata = ViewMetadataParser.read(metadataFile); | |||
| validateMetadataFileInTableDir(identifier, metadata.location(), metadataFileLocation); | |||
| ops.commit(null, metadata); | |||
There was a problem hiding this comment.
I noticed this while writing the tests—validateMetadataFileInTableDir strictly reads from the realm-level configuration and ignores the catalog-level overrides, meaning a catalog configured with ALLOW_EXTERNAL_TABLE_LOCATION=true will still be blocked by this check.
Since this exact same split-brain configuration bug currently exists for registerTable, I opted to keep the implementation as-is to bring registerView into strict behavioral parity with tables.
I plan to chase that separately if people agree that ain't deliberate rather than expanding the scope here
There was a problem hiding this comment.
ALLOW_EXTERNAL_METADATA_FILE_LOCATION is not overridable at catalog level, but ALLOW_EXTERNAL_TABLE_LOCATION is.
This looks like a bug to me. Do you mind opening an issue for this?
|
|
||
| InputFile metadataFile = fileIO.newInputFile(metadataFileLocation); | ||
| ViewMetadata metadata = ViewMetadataParser.read(metadataFile); | ||
| validateMetadataFileInTableDir(identifier, metadata.location(), metadataFileLocation); |
There was a problem hiding this comment.
I think you also need to call validateLocationForTableLike – the metadata.location() value must be valid wrt to the storage configuration from the table hierarchy, AND wrt to the table directory, if escaping the directory is not allowed:
| validateMetadataFileInTableDir(identifier, metadata.location(), metadataFileLocation); | |
| validateLocationForTableLike(identifier, metadata.location(), resolvedParent); | |
| validateMetadataFileInTableDir(identifier, metadata.location(), metadataFileLocation); |
|
Hey @dimas-b this was a different issue, not sure why this PR got closed. I don't think I have access to reopen either. Should I create another PR? |
|
Oops, the closing was not intentional, sorry 😅 |
58015dc to
b029a7b
Compare
|
Thanx @dimas-b for the help. Must be some bug in Github Auto close, some missing |
b029a7b to
db40db6
Compare
db40db6 to
26df577
Compare
26df577 to
66c2037
Compare
|
It seems that this PR is superseded by #5114, FYI. |
Currently,
LocalIcebergCatalog.registerNewTableproperly enforces catalog storage governance by callingvalidateLocationForTableLikeandvalidateMetadataFileInTableDiron the user-providedmetadataFileLocation.However,
registerViewomits these checks, directly reading the provided metadata file URI before any authorization or structural containment checks occur. This creates a governance gap where a user could register a view using a metadata file stored outside of the catalog's allowed locations, potentially bypassing storage RBAC policies.This PR brings
registerViewinto parity withregisterTableby adding the missing validation checksChecklist
CHANGELOG.md(if needed)site/content/in-dev/unreleased(if needed)