Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add moveFileAtomic method #2586

Merged
merged 18 commits into from
Mar 31, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -180,6 +180,7 @@ Samples are in the [`samples/`](https://github.com/googleapis/nodejs-storage/tre
| Storage Make Bucket Public. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/makeBucketPublic.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/makeBucketPublic.js,samples/README.md) |
| Make Public | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/makePublic.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/makePublic.js,samples/README.md) |
| Move File | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/moveFile.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/moveFile.js,samples/README.md) |
| Move File Atomic | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/moveFileAtomic.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/moveFileAtomic.js,samples/README.md) |
| Print Bucket Acl | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/printBucketAcl.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/printBucketAcl.js,samples/README.md) |
| Print Bucket Acl For User | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/printBucketAclForUser.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/printBucketAclForUser.js,samples/README.md) |
| Print File Acl | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/printFileAcl.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/printFileAcl.js,samples/README.md) |
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@
"dependencies": {
"@google-cloud/paginator": "^5.0.0",
"@google-cloud/projectify": "^4.0.0",
"@google-cloud/promisify": "^4.0.0",
"@google-cloud/promisify": "<4.1.0",
"abort-controller": "^3.0.0",
"async-retry": "^1.3.3",
"duplexify": "^4.1.3",
Expand Down
18 changes: 18 additions & 0 deletions samples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@ objects to users via direct download.
* [Storage Make Bucket Public.](#storage-make-bucket-public.)
* [Make Public](#make-public)
* [Move File](#move-file)
* [Move File Atomic](#move-file-atomic)
* [Print Bucket Acl](#print-bucket-acl)
* [Print Bucket Acl For User](#print-bucket-acl-for-user)
* [Print File Acl](#print-file-acl)
Expand Down Expand Up @@ -1594,6 +1595,23 @@ __Usage:__



### Move File Atomic

View the [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/moveFileAtomic.js).

[![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/moveFileAtomic.js,samples/README.md)

__Usage:__


`node samples/moveFileAtomic.js`


-----




### Print Bucket Acl

View the [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/printBucketAcl.js).
Expand Down
76 changes: 76 additions & 0 deletions samples/moveFileAtomic.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

/**
* This application demonstrates how to perform basic operations on files with
* the Google Cloud Storage API.
*
* For more information, see the README.md under /storage and the documentation
* at https://cloud.google.com/storage/docs.
*/

function main(
bucketName = 'my-bucket',
srcFileName = 'test.txt',
destFileName = 'test2.txt',
destinationGenerationMatchPrecondition = 0
) {
// [START storage_move_object]
/**
* TODO(developer): Uncomment the following lines before running the sample.
*/
// The ID of your GCS bucket
// const bucketName = 'your-source-bucket';

// The ID of your GCS file
// const srcFileName = 'your-file-name';

// The new ID for your GCS file
// const destFileName = 'your-new-file-name';

// Imports the Google Cloud client library
const {Storage} = require('@google-cloud/storage');

// Creates a client
const storage = new Storage();

async function moveFileAtomic() {
// Optional:
// Set a generation-match precondition to avoid potential race conditions
// and data corruptions. The request to copy is aborted if the object's
// generation number does not match your precondition. For a destination
// object that does not yet exist, set the ifGenerationMatch precondition to 0
// If the destination object already exists in your bucket, set instead a
// generation-match precondition using its generation number.
const moveOptions = {
preconditionOpts: {
ifGenerationMatch: destinationGenerationMatchPrecondition,
},
};

// Moves the file automatically within the HNS enabled bucket
await storage
.bucket(bucketName)
.file(srcFileName)
.moveFileAtomic(destFileName, moveOptions);

console.log(
`gs://${bucketName}/${srcFileName} moved to gs://${bucketName}/${destFileName}`
);
}

moveFileAtomic().catch(console.error);
// [END storage_move_object]
}
main(...process.argv.slice(2));
29 changes: 29 additions & 0 deletions samples/system-test/files.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ const storage = new Storage();
const cwd = path.join(__dirname, '..');
const bucketName = generateName();
const bucket = storage.bucket(bucketName);
const hnsBucketName = generateName();
const hnsBucket = storage.bucket(hnsBucketName);
const objectRetentionBucketName = generateName();
const objectRetentionBucket = storage.bucket(objectRetentionBucketName);
const fileContents = 'these-are-my-contents';
Expand Down Expand Up @@ -597,6 +599,33 @@ describe('file', () => {
assert(metadata.retention.mode.toUpperCase(), 'UNLOCKED');
});
});

describe('HNS Bucket Move Object', () => {
before(async () => {
await storage.createBucket(hnsBucketName, {
hierarchicalNamespace: {enabled: true},
iamConfiguration: {
uniformBucketLevelAccess: {
enabled: true,
},
},
});
});

it('should move a file', async () => {
const file = hnsBucket.file(fileName);
await file.save(fileName);
const output = execSync(
`node moveFileAtomic.js ${hnsBucketName} ${fileName} ${movedFileName} ${doesNotExistPrecondition}`
);
assert.include(
output,
`gs://${hnsBucketName}/${fileName} moved to gs://${hnsBucketName}/${movedFileName}`
);
const [exists] = await hnsBucket.file(movedFileName).exists();
assert.strictEqual(exists, true);
});
});
});

function generateName() {
Expand Down
197 changes: 197 additions & 0 deletions src/file.ts
Original file line number Diff line number Diff line change
Expand Up @@ -282,6 +282,14 @@ export interface MakeFilePublicCallback {
(err?: Error | null, apiResponse?: unknown): void;
}

interface MoveFileAtomicQuery {
userProject?: string;
ifGenerationMatch?: number | string;
ifGenerationNotMatch?: number | string;
ifMetagenerationMatch?: number | string;
ifMetagenerationNotMatch?: number | string;
}

export type MoveResponse = [unknown];

export interface MoveCallback {
Expand All @@ -297,6 +305,10 @@ export interface MoveOptions {
preconditionOpts?: PreconditionOptions;
}

export type MoveFileAtomicOptions = MoveOptions;
export type MoveFileAtomicCallback = MoveCallback;
export type MoveFileAtomicResponse = MoveResponse;

export type RenameOptions = MoveOptions;
export type RenameResponse = MoveResponse;
export type RenameCallback = MoveCallback;
Expand Down Expand Up @@ -3426,6 +3438,191 @@ class File extends ServiceObject<File, FileMetadata> {
}/${encodeURIComponent(this.name)}`;
}

moveFileAtomic(
destination: string | File,
options?: MoveFileAtomicOptions
): Promise<MoveFileAtomicResponse>;
moveFileAtomic(
destination: string | File,
callback: MoveFileAtomicCallback
): void;
moveFileAtomic(
destination: string | File,
options: MoveFileAtomicOptions,
callback: MoveFileAtomicCallback
): void;
/**
* @typedef {array} MoveFileAtomicResponse
* @property {File} 0 The moved {@link File}.
* @property {object} 1 The full API response.
*/
/**
* @callback MoveFileAtomicCallback
* @param {?Error} err Request error, if any.
* @param {File} movedFile The moved {@link File}.
* @param {object} apiResponse The full API response.
*/
/**
* @typedef {object} MoveFileAtomicOptions Configuration options for File#moveFileAtomic(). See an
* {@link https://cloud.google.com/storage/docs/json_api/v1/objects#resource| Object resource}.
* @property {string} [userProject] The ID of the project which will be
* billed for the request.
* @property {object} [preconditionOpts] Precondition options.
* @property {number} [preconditionOpts.ifGenerationMatch] Makes the operation conditional on whether the object's current generation matches the given value.
*/
/**
* Move this file within the same HNS-enabled bucket.
* The source object must exist and be a live object.
* The source and destination object IDs must be different.
* Overwriting the destination object is allowed by default, but can be prevented
* using preconditions.
* If the destination path includes non-existent parent folders, they will be created.
*
* See {@link https://cloud.google.com/storage/docs/json_api/v1/objects/move| Objects: move API Documentation}
*
* @throws {Error} If the destination file is not provided.
*
* @param {string|File} destination Destination file name or File object within the same bucket..
* @param {MoveFileAtomicOptions} [options] Configuration options. See an
* @param {MoveFileAtomicCallback} [callback] Callback function.
* @returns {Promise<MoveFileAtomicResponse>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
*
* //-
* // Assume 'my-hns-bucket' is an HNS-enabled bucket.
* //-
* const bucket = storage.bucket('my-hns-bucket');
* const file = bucket.file('my-image.png');
*
* //-
* // If you pass in a string for the destination, the file is copied to its
* // current bucket, under the new name provided.
* //-
* file.moveFileAtomic('moved-image.png', function(err, movedFile, apiResponse) {
* // `my-hns-bucket` now contains:
* // - "moved-image.png"
*
* // `movedFile` is an instance of a File object that refers to your new
* // file.
* });
*
* //-
* // Move the file to a subdirectory, creating parent folders if necessary.
* //-
* file.moveFileAtomic('new-folder/subfolder/moved-image.png', function(err, movedFile, apiResponse) {
* // `my-hns-bucket` now contains:
* // - "new-folder/subfolder/moved-image.png"
* });
*
* //-
* // Prevent overwriting an existing destination object using preconditions.
* //-
* file.moveFileAtomic('existing-destination.png', {
* preconditionOpts: {
* ifGenerationMatch: 0 // Fails if the destination object exists.
* }
* }, function(err, movedFile, apiResponse) {
* if (err) {
* // Handle the error (e.g., the destination object already exists).
* } else {
* // Move successful.
* }
* });
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* file.moveFileAtomic('moved-image.png).then(function(data) {
* const newFile = data[0];
* const apiResponse = data[1];
* });
*
* ```
* @example <caption>include:samples/files.js</caption>
* region_tag:storage_move_file_hns
* Another example:
*/
moveFileAtomic(
destination: string | File,
optionsOrCallback?: MoveFileAtomicOptions | MoveFileAtomicCallback,
callback?: MoveFileAtomicCallback
): Promise<MoveFileAtomicResponse> | void {
const noDestinationError = new Error(
FileExceptionMessages.DESTINATION_NO_NAME
);

if (!destination) {
throw noDestinationError;
}

let options: MoveFileAtomicOptions = {};
if (typeof optionsOrCallback === 'function') {
callback = optionsOrCallback;
} else if (optionsOrCallback) {
options = {...optionsOrCallback};
}

callback = callback || util.noop;

let destName: string;
let newFile: File;

if (typeof destination === 'string') {
const parsedDestination = GS_URL_REGEXP.exec(destination);
if (parsedDestination !== null && parsedDestination.length === 3) {
destName = parsedDestination[2];
} else {
destName = destination;
}
} else if (destination instanceof File) {
destName = destination.name;
newFile = destination;
} else {
throw noDestinationError;
}

newFile = newFile! || this.bucket.file(destName);

if (
!this.shouldRetryBasedOnPreconditionAndIdempotencyStrat(
options?.preconditionOpts
)
) {
this.storage.retryOptions.autoRetry = false;
}
const query = {} as MoveFileAtomicQuery;
if (options.userProject !== undefined) {
query.userProject = options.userProject;
delete options.userProject;
}
if (options.preconditionOpts?.ifGenerationMatch !== undefined) {
query.ifGenerationMatch = options.preconditionOpts?.ifGenerationMatch;
delete options.preconditionOpts;
}

this.request(
{
method: 'POST',
uri: `/moveTo/o/${encodeURIComponent(newFile.name)}`,
qs: query,
json: options,
},
(err, resp) => {
this.storage.retryOptions.autoRetry = this.instanceRetryValue;
if (err) {
callback!(err, null, resp);
return;
}

callback!(null, newFile, resp);
}
);
}

move(
destination: string | Bucket | File,
options?: MoveOptions
Expand Down
3 changes: 3 additions & 0 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,9 @@ export {
MoveCallback,
MoveOptions,
MoveResponse,
MoveFileAtomicOptions,
MoveFileAtomicCallback,
MoveFileAtomicResponse,
PolicyDocument,
PolicyFields,
PredefinedAcl,
Expand Down
Loading