NSFS | content dir versioning LIST and tagging operations #8814
Conversation
nadavMiz
force-pushed
the
directory-object-list-operation
branch
6 times, most recently
from
ccbf9f6 to
573c2d1
Compare
docs/design/NsfsVersioning.md
Outdated
| content dir structure has two modes: | ||
| 1. versioning disabled mode,at this mode xattr are located at the directory. we create `.folder` file inside the directory only if the object has data in it. in that case `.folder` file contains the objects body. | ||
| example bucket tree: |
There was a problem hiding this comment.
I would give some more background and organize it a bit more. This is an example for disabled mode, we need the same for the other modes
| content dir structure has two modes: | |
| 1. versioning disabled mode,at this mode xattr are located at the directory. we create `.folder` file inside the directory only if the object has data in it. in that case `.folder` file contains the objects body. | |
| example bucket tree: | |
| Versioning related xattr - | |
| 1. `user.noobaa.version_id` - stores the version_id string. | |
| 2. `user.noobaa.delete_marker` - boolean. | |
| 3. `user.noobaa.content_dir` - stores the size of the directory object. | |
| Directory object / content directory filesystem structure has two modes - | |
| 1. versioning disabled mode - | |
| On this mode, the object body and the xattr (extended attributes) are splitted. The data of the directory object will be stored in `<dir_path>/.folder`, while the xattr of the directory object will be stored on the directory itself - <dir_path>. `<dir_path>/.folder` file is created only if the object's data size > 0. When size = 0, only the xattr will be stored in `<dir_path>`. |
| const fs_context = this.prepare_fs_context(object_sdk); | ||
| const file_path = await this._find_version_path(fs_context, params, true); |
There was a problem hiding this comment.
have you checked that this change is compatible to S3? especially when versioning is suspended, delete tagging from the version id file is expected per the protocol?
There was a problem hiding this comment.
there shouldn't be a difference between enabled and suspended mode. but I will validate this. I will also check what happens if we are in disabled mode and have version as a parameter
| const key_list_versions = list_res.Versions.filter(v => v.Key === key); | ||
| assert.equal(key_list_versions.length, version_ids.size); | ||
| key_list_versions.forEach(v => { | ||
| assert(version_ids.has(v.VersionId)); |
There was a problem hiding this comment.
can you add an assert of the key and the version id do not contain .folder?
|
|
||
| await s3_uid6.putBucketVersioning({ Bucket: content_dir_bucket_name, VersioningConfiguration: { MFADelete: 'Disabled', Status: 'Enabled' } }); |
There was a problem hiding this comment.
after validating the behavior on AWS, can you add a test for suspended mode as well?
| }); | ||
|
|
||
| mocha.it('content directory - list object versions', async function() { | ||
| const key = "list_key/"; |
There was a problem hiding this comment.
do we need a test of list object versions versioning suspended?
| const key = "list_key/"; | ||
| const version_ids = new Set(); | ||
| let push_res = await s3_uid6.putObject({ Bucket: content_dir_bucket_name, Key: key, Body: body1 }); | ||
| version_ids.add(push_res.VersionId); |
There was a problem hiding this comment.
can you also add a delete marker of content dir in this test / another test?
|
@nadavMiz please add a link in the description to all the content directory PRs the original issue, I think it's 8531 |
nadavMiz
force-pushed
the
directory-object-list-operation
branch
from
573c2d1 to
629a556
Compare
…ing operations Signed-off-by: nadav mizrahi <[email protected]>
nadavMiz
force-pushed
the
directory-object-list-operation
branch
from
629a556 to
b59fbb2
Compare
Explain the changes
Issues: Fixed #xxx / Gap #xxx
Content-dir versioning issue: #8531
continuation of #8773
####GAPS
Testing Instructions: