fix(services): encode version id for cos and tos#7888
Conversation
Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com>
dosubot
Bot
added
size:L
|
Can we have a test for versioned object in behavior test? |
Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com>
@erickguan Thanks for the suggestion. I have updated the behavior tests to cover versioned object operations more explicitly. PTAL 👀 |
|
Alright, I tried S3. S3 seems to respond with proper URI without breaking anything at least in behavior tests. Will have a look. |
| fn restore_encoded_query( | ||
| uri: &mut Uri, | ||
| encoded_query: Option<&str>, | ||
| keep_signed_query: bool, |
There was a problem hiding this comment.
Do we need this? For URI to be correct, I believe we always will need encoded query param.
| .await | ||
| .map_err(|e| new_request_sign_error(e.into()))?; | ||
|
|
||
| Self::restore_encoded_query(&mut parts.uri, encoded_query.as_deref(), true)?; |
There was a problem hiding this comment.
I didn't look into details. Can we build a new parts instead of mutating it?
Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com>
There was a problem hiding this comment.
I believe the real problem is reqsign didn't keep the raw query as is, we should fix at reqsign side so we don't need to have this query restore logic.
And before we are going too crazy: version id returned by services is always picked and not a randomly generated string. So I'm ok to really fix it but I don't want to merge PR like this one.
Both percent_encode_path and percent_decode_path here are all no-op (at least for 99% cases, I didn't see a real report).
@Xuanwo Thanks for the review. I agree that the root fix should happen in reqsign-core instead of adding service-local query restoration logic in OpenDAL. I've opened an issue to track it: apache/opendal-reqsign#778 I'll pause this PR and will fix the reqsign side firstly, then revisit this change with the workaround removed. |
3 participants
Which issue does this PR close?
Closes #.
Rationale for this change
COS and TOS used decoded version IDs when building
versionIdquery parameters for read/stat/delete requests. If a version ID contains query-reserved characters like&,=,+, or%, the generated URI can be parsed as different query parameters instead of a single encodedversionId.What changes are included in this PR?
versionIdquery values before appending them to request URIs.Are there any user-facing changes?
Yes. COS and TOS read/stat/delete operations with version IDs containing query-reserved characters now build valid request URIs.
AI Usage Statement
I used OpenAI Codex to implement the test in this change.