Skip to content

Implement Java sync improved bulk write API and unified spec tests #1486

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 76 commits into from
Sep 24, 2024
Merged
Show file tree
Hide file tree
Changes from 49 commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
48b9614
Create and document Java sync improved bulk write API
stIncMale Jul 23, 2024
af854ed
Remove the type parameter from `ClientWriteModel`
stIncMale Jul 23, 2024
1eb7466
Remove `ClientBulkWriteException.create` as we can get by with the co…
stIncMale Jul 25, 2024
5567324
Merge branch 'master' into JAVA-5527
stIncMale Jul 27, 2024
644d561
Merge branch 'master' into JAVA-5527
stIncMale Aug 13, 2024
f566303
Do minor improvements
stIncMale Aug 15, 2024
3d13ffd
Make changes needed for the implementation
stIncMale Aug 15, 2024
cf46b46
Fix formatting in ClientUpdateManyModel
stIncMale Aug 15, 2024
e049234
Make a few minor changes
stIncMale Aug 15, 2024
0e16427
Merge branch 'master' into JAVA-5527
stIncMale Aug 15, 2024
0d518d8
Add more info to the API docs, add `ClientWriteModel` subtypes
stIncMale Aug 22, 2024
6b77f78
Add `ClientWriteModelWithNamespace`
stIncMale Aug 22, 2024
2ed8e87
Implement
stIncMale Aug 8, 2024
07bac95
Sync spec tests
stIncMale Aug 15, 2024
16f100b
Implement required test runner changes
stIncMale Aug 15, 2024
9f8ce2c
Improve how `indexedNamespaces` are computed
stIncMale Aug 22, 2024
fdb90d2
Remove `throws` declarations from the API
stIncMale Aug 22, 2024
39386fe
Make wording on `ClientWriteModel` methods consistent with that on `C…
stIncMale Aug 22, 2024
f67af3f
Move constructor methods to `ClientNamespacedWriteModel`
stIncMale Aug 22, 2024
53f6883
Merge branch 'JAVA-5527' into JAVA-5528
stIncMale Aug 22, 2024
9a7b668
Fix errors caused by the merge
stIncMale Aug 22, 2024
395af7a
Use `Optional` to express verbose/summary results
stIncMale Aug 22, 2024
8f504b0
Merge branch 'JAVA-5527' into JAVA-5528
stIncMale Aug 22, 2024
b02a638
Fix errors caused by the merge
stIncMale Aug 22, 2024
a11e5f6
Make an API doc improvement
stIncMale Aug 27, 2024
bfbc1cc
Refactor `shouldAttemptToRetryWriteAndAddRetryableLabel`
stIncMale Aug 27, 2024
1c3c590
Improve `CrudProseTest.insertMustGenerateIdAtMostOnce`
stIncMale Aug 27, 2024
1c9c19e
Move `MixedBulkWriteOperation.validateAndGetEffectiveWriteConcern`/`c…
stIncMale Aug 27, 2024
061e605
Add a comment in `toClientNamespacedWriteModel`
stIncMale Aug 27, 2024
8320216
Use `CommandResultDocumentCodec` in `ClientBulkWriteOperation`
stIncMale Aug 28, 2024
f41ed59
Use `Integer` for indexes in `ClientBulkWriteResult.Verbose`. Make `C…
stIncMale Aug 28, 2024
4846e0b
Use `Integer` for indexes in `ClientBulkWriteException`.
stIncMale Aug 28, 2024
dbf9a26
Take `ClientBulkWriteException` into account in `OperationExecutor.ex…
stIncMale Aug 28, 2024
a0005cd
Merge branch 'JAVA-5527' into JAVA-5528
stIncMale Aug 28, 2024
35eee96
Fixes after the merge
stIncMale Aug 28, 2024
21bb22e
Make `ClientInsertOneResult.getInsertedId` return `Optional`
stIncMale Aug 28, 2024
182b2f9
Merge branch 'JAVA-5527' into JAVA-5528
stIncMale Aug 28, 2024
fb32fde
Fixes after the merge
stIncMale Aug 28, 2024
f026ee3
Update the documentation of `ClientBulkWriteException` and remove a TODO
stIncMale Aug 28, 2024
7372f13
Merge branch 'JAVA-5527' into JAVA-5528
stIncMale Aug 28, 2024
0fb65d4
Synchronize unified tests with https://github.com/mongodb/specificati…
stIncMale Aug 29, 2024
e36898f
Refactor `CrudProseTest` to support the reactive client
stIncMale Aug 29, 2024
70bb422
Add methods for all prose tests. Where possible, implement them
stIncMale Aug 30, 2024
3584de5
Move results to `com.mongodb.client.model.bulk`
stIncMale Aug 30, 2024
5a17ea0
Merge branch 'master' into JAVA-5527
stIncMale Aug 30, 2024
8e1d770
Move internal results to `com.mongodb.internal.client.model.bulk`
stIncMale Aug 30, 2024
e973615
Document that `bulkWrite` is not supported by serverless instances
stIncMale Aug 30, 2024
33ec764
Merge branch 'JAVA-5527' into JAVA-5528
stIncMale Aug 30, 2024
ee893f2
Fixes after the merge
stIncMale Aug 30, 2024
e33f592
Add subtypes of `ClientNamespacedWriteModel` and hide `ClientWriteModel`
stIncMale Sep 3, 2024
f94ad58
Merge branch 'JAVA-5527' into JAVA-5528
stIncMale Sep 3, 2024
310426a
Fixes after the merge
stIncMale Sep 3, 2024
301a2ba
Remove a garbage comment
stIncMale Sep 3, 2024
1a57fb6
Merge branch 'JAVA-5527' into JAVA-5528
stIncMale Sep 3, 2024
6f01e61
Replace `ConcreteClientNamespacedWriteModel` with multiple more speci…
stIncMale Sep 9, 2024
c3224e9
Rearrange `ClientWriteModel` inheritance: "one" should extend "many",…
stIncMale Sep 9, 2024
c057390
Make internal `AbstractClientNamespacedWriteModel` public
stIncMale Sep 9, 2024
dbb6ec8
Rearrange `ClientWriteModel` inheritance: neither "one" nor "many" sh…
stIncMale Sep 9, 2024
cb11c44
Merge branch 'JAVA-5527' into JAVA-5528
stIncMale Sep 9, 2024
8bd7a6a
Fixes after the merge
stIncMale Sep 9, 2024
f781bca
Make internal `AbstractClientUpdateModel`, `AbstractClientDeleteModel…
stIncMale Sep 9, 2024
ec88c02
Merge branch 'JAVA-5527' into JAVA-5528
stIncMale Sep 9, 2024
14bb86e
Improve code comments in `ClientBulkWriteOperation`
stIncMale Sep 10, 2024
f186b31
Fix `server-selection/logging/operation-id.json:Failed client bulkWri…
stIncMale Sep 16, 2024
3442a73
Rename an incorrectly named variable
stIncMale Sep 17, 2024
3fc86bb
Stop linking to `Filters` from the documentation of `ClientNamespaced…
stIncMale Sep 17, 2024
86e5234
Update driver-core/src/main/com/mongodb/client/model/bulk/ClientBulkW…
stIncMale Sep 18, 2024
fb134f8
Update driver-core/src/main/com/mongodb/client/model/bulk/ClientDelet…
stIncMale Sep 18, 2024
a4bf4d0
Fix typos in API docs
stIncMale Sep 18, 2024
2837235
Change code to always refer to `ClientBulkWriteResult.Verbose` with t…
stIncMale Sep 18, 2024
b6702a6
Rename `ClientBulkWriteResult.Verbose` to `ClientBulkWriteResult.Verb…
stIncMale Sep 23, 2024
24ce6db
Improve partial result API documentation wording
stIncMale Sep 24, 2024
ca5d19a
Merge branch 'JAVA-5527' into JAVA-5528
stIncMale Sep 24, 2024
6d8a3e6
Fixes after the merge
stIncMale Sep 24, 2024
076d39a
Merge branch 'master' into JAVA-5528
stIncMale Sep 24, 2024
6b2b7a8
Merge branch 'JAVA-4586_bulk-write' into JAVA-5528
stIncMale Sep 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
154 changes: 154 additions & 0 deletions driver-core/src/main/com/mongodb/ClientBulkWriteException.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
/*
* Copyright 2008-present MongoDB, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.mongodb;

import com.mongodb.bulk.WriteConcernError;
import com.mongodb.client.model.bulk.ClientWriteModel;
import com.mongodb.client.model.bulk.ClientBulkWriteResult;
import com.mongodb.lang.Nullable;

import java.util.List;
import java.util.Map;
import java.util.Optional;

import static com.mongodb.assertions.Assertions.isTrueArgument;
import static com.mongodb.assertions.Assertions.notNull;
import static com.mongodb.internal.operation.ClientBulkWriteOperation.Exceptions.serverAddressFromException;
import static java.util.Collections.emptyList;
import static java.util.Collections.emptyMap;
import static java.util.Collections.unmodifiableList;
import static java.util.Collections.unmodifiableMap;
import static java.util.Optional.ofNullable;

/**
* The result of an unsuccessful or partially unsuccessful client-level bulk write operation.
* Note that the {@linkplain #getCode() code} and {@linkplain #getErrorLabels() labels} from this exception are not useful.
* An application should use those from the {@linkplain #getError() top-level error}.
*
* @see ClientBulkWriteResult
* @since 5.3
* @serial exclude
*/
public final class ClientBulkWriteException extends MongoServerException {
private static final long serialVersionUID = 1;

@Nullable
private final MongoException error;
private final List<WriteConcernError> writeConcernErrors;
private final Map<Integer, WriteError> writeErrors;
@Nullable
private final ClientBulkWriteResult partialResult;

/**
* Constructs a new instance.
*
* @param error The {@linkplain #getError() top-level error}.
* @param writeConcernErrors The {@linkplain #getWriteConcernErrors() write concern errors}.
* @param writeErrors The {@linkplain #getWriteErrors() write errors}.
* @param partialResult The {@linkplain #getPartialResult() partial result}.
* @param serverAddress The {@linkplain MongoServerException#getServerAddress() server address}.
* If {@code error} is a {@link MongoServerException} or a {@link MongoSocketException}, then {@code serverAddress}
* must be equal to the {@link ServerAddress} they bear.
*/
public ClientBulkWriteException(
@Nullable final MongoException error,
@Nullable final List<WriteConcernError> writeConcernErrors,
@Nullable final Map<Integer, WriteError> writeErrors,
@Nullable final ClientBulkWriteResult partialResult,
final ServerAddress serverAddress) {
super(
message(
error, writeConcernErrors, writeErrors, partialResult,
notNull("serverAddress", serverAddress)),
validateServerAddress(error, serverAddress));
isTrueArgument("At least one of `writeConcernErrors`, `writeErrors`, `partialResult` must be non-null or non-empty",
!(writeConcernErrors == null || writeConcernErrors.isEmpty())
|| !(writeErrors == null || writeErrors.isEmpty())
|| partialResult != null);
this.error = error;
this.writeConcernErrors = writeConcernErrors == null ? emptyList() : unmodifiableList(writeConcernErrors);
this.writeErrors = writeErrors == null ? emptyMap() : unmodifiableMap(writeErrors);
this.partialResult = partialResult;
}

private static String message(
@Nullable final MongoException error,
@Nullable final List<WriteConcernError> writeConcernErrors,
@Nullable final Map<Integer, WriteError> writeErrors,
@Nullable final ClientBulkWriteResult partialResult,
final ServerAddress serverAddress) {
return "Client-level bulk write operation error on server " + serverAddress + "."
+ (error == null ? "" : " Top-level error: " + error + ".")
+ (writeErrors == null || writeErrors.isEmpty() ? "" : " Write errors: " + writeErrors + ".")
+ (writeConcernErrors == null || writeConcernErrors.isEmpty() ? "" : " Write concern errors: " + writeConcernErrors + ".")
+ (partialResult == null ? "" : " Partial result: " + partialResult + ".");
}

private static ServerAddress validateServerAddress(@Nullable final MongoException error, final ServerAddress serverAddress) {
serverAddressFromException(error).ifPresent(serverAddressFromError ->
isTrueArgument("`serverAddress` must be equal to that of the `error`", serverAddressFromError.equals(serverAddress)));
return error instanceof MongoServerException
? ((MongoServerException) error).getServerAddress()
: serverAddress;
}

/**
* The top-level error. That is an error that is neither a {@linkplain #getWriteConcernErrors() write concern error},
* nor is an {@linkplain #getWriteErrors() error of an individual write operation}.
*
* @return The top-level error. {@linkplain Optional#isPresent() Present} only if a top-level error occurred.
*/
public Optional<MongoException> getError() {
return ofNullable(error);
}

/**
* The {@link WriteConcernError}s that occurred while executing the client-level bulk write operation.
* <p>
* There are no guarantees on mutability of the {@link List} returned.</p>
*
* @return The {@link WriteConcernError}s.
*/
public List<WriteConcernError> getWriteConcernErrors() {
return writeConcernErrors;
}

/**
* The indexed {@link WriteError}s.
* The {@linkplain Map#keySet() keys} are the indexes of the corresponding {@link ClientWriteModel}s
* in the corresponding client-level bulk write operation.
* <p>
* There are no guarantees on mutability or iteration order of the {@link Map} returned.</p>
*
* @return The indexed {@link WriteError}s.
* @see ClientBulkWriteResult.Verbose#getInsertResults()
* @see ClientBulkWriteResult.Verbose#getUpdateResults()
* @see ClientBulkWriteResult.Verbose#getDeleteResults()
*/
public Map<Integer, WriteError> getWriteErrors() {
return writeErrors;
}

/**
* The result of the successful part of a client-level bulk write operation.
*
* @return The successful partial result. {@linkplain Optional#isPresent() Present} only if at least one
* {@linkplain ClientWriteModel individual write operation} succeed.
*/
public Optional<ClientBulkWriteResult> getPartialResult() {
return ofNullable(partialResult);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
/*
* Copyright 2008-present MongoDB, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.mongodb.client.model.bulk;

import com.mongodb.annotations.Sealed;
import com.mongodb.client.model.Filters;
import com.mongodb.internal.client.model.bulk.ConcreteClientBulkWriteOptions;
import com.mongodb.lang.Nullable;
import org.bson.BsonValue;
import org.bson.conversions.Bson;

/**
* The options to apply when executing a client-level bulk write operation.
*
* @since 5.3
*/
@Sealed
public interface ClientBulkWriteOptions {
/**
* Creates the default options.
*
* @return The default options.
*/
static ClientBulkWriteOptions clientBulkWriteOptions() {
return new ConcreteClientBulkWriteOptions();
}

/**
* Enables or disables ordered execution of {@linkplain ClientWriteModel individual write operations}.
* In an ordered execution a failure of an individual operation prevents the rest of them
* from being executed.
* In an unordered execution failures of individual operations do not prevent the rest of them
* from being executed.
*
* @param ordered The ordered flag. If {@code null}, the client defaults to {@code true}.
* @return {@code this}.
*/
ClientBulkWriteOptions ordered(@Nullable Boolean ordered);

/**
* Disables or enables checking against document validation rules, a.k.a., schema validation.
*
* @param bypassDocumentValidation The flag specifying whether to bypass the document validation rules.
* {@code null} represents the server default.
* @return {@code this}.
*/
ClientBulkWriteOptions bypassDocumentValidation(@Nullable Boolean bypassDocumentValidation);

/**
* Sets variables that can be referenced from {@linkplain ClientWriteModel individual write operations}
* with the {@code "$$"} syntax, which in turn requires using {@link Filters#expr(Object)} when specifying filters.
* Values must be constants or expressions that do not reference fields.
*
* @param let The variables. {@code null} represents the server default.
* @return {@code this}.
* @mongodb.driver.manual reference/aggregation-variables/ Variables in Aggregation Expressions
*/
ClientBulkWriteOptions let(@Nullable Bson let);

/**
* Sets the comment to attach to the {@code bulkWrite} administration command.
*
* @param comment The comment. {@code null} represents the server default.
* @return {@code this}.
*/
ClientBulkWriteOptions comment(@Nullable BsonValue comment);

/**
* Enables or disables requesting {@linkplain ClientBulkWriteResult#getVerbose() verbose results}.
*
* @param verboseResults The flag specifying whether to request verbose results.
* If {@code null}, the client defaults to {@code false}.
* This value corresponds inversely to the {@code errorsOnly} field of the {@code bulkWrite} administration command.
* @return {@code this}.
*/
ClientBulkWriteOptions verboseResults(@Nullable Boolean verboseResults);
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
/*
* Copyright 2008-present MongoDB, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.mongodb.client.model.bulk;

import com.mongodb.ClientBulkWriteException;
import com.mongodb.WriteConcern;
import com.mongodb.annotations.Evolving;
import com.mongodb.bulk.WriteConcernError;

import java.util.Map;
import java.util.Optional;

/**
* The result of a successful or partially successful client-level bulk write operation.
* Note that if only some of the {@linkplain ClientWriteModel individual write operations} succeed,
* or if there are {@link WriteConcernError}s, then the successful partial result
* is still accessible via {@link ClientBulkWriteException#getPartialResult()}.
*
* @see ClientBulkWriteException
* @since 5.3
*/
@Evolving
public interface ClientBulkWriteResult {
/**
* Indicated whether this result was {@linkplain WriteConcern#isAcknowledged() acknowledged}.
* If not, then all other methods throw {@link UnsupportedOperationException}.
*
* @return Whether this result was acknowledged.
*/
boolean isAcknowledged();

/**
* The number of documents that were inserted across all insert operations.
*
* @return The number of documents that were inserted.
* @throws UnsupportedOperationException If this result is not {@linkplain #isAcknowledged() acknowledged}.
*/
long getInsertedCount();

/**
* The number of documents that were upserted across all update and replace operations.
*
* @return The number of documents that were upserted.
* @throws UnsupportedOperationException If this result is not {@linkplain #isAcknowledged() acknowledged}.
*/
long getUpsertedCount();

/**
* The number of documents that matched the filters across all operations with filters.
*
* @return The number of documents that were matched.
* @throws UnsupportedOperationException If this result is not {@linkplain #isAcknowledged() acknowledged}.
*/
long getMatchedCount();

/**
* The number of documents that were modified across all update and replace operations.
*
* @return The number of documents that were modified.
* @throws UnsupportedOperationException If this result is not {@linkplain #isAcknowledged() acknowledged}.
*/
long getModifiedCount();

/**
* The number of documents that were deleted across all delete operations.
*
* @return The number of documents that were deleted.
* @throws UnsupportedOperationException If this result is not {@linkplain #isAcknowledged() acknowledged}.
*/
long getDeletedCount();

/**
* The verbose results of individual operations.
*
* @return {@link Optional} verbose results of individual operations.
* @throws UnsupportedOperationException If this result is not {@linkplain #isAcknowledged() acknowledged}.
* @see ClientBulkWriteOptions#verboseResults(Boolean)
*/
Optional<Verbose> getVerbose();

/**
* The {@linkplain ClientBulkWriteResult#getVerbose() verbose results} of individual operations.
*
* @since 5.3
*/
@Evolving
interface Verbose {
/**
* The indexed {@link ClientInsertOneResult}s.
* The {@linkplain Map#keySet() keys} are the indexes of the corresponding {@link ClientWriteModel}s
* in the client-level bulk write operation.
* <p>
* There are no guarantees on mutability or iteration order of the {@link Map} returned.</p>
*
* @return The indexed {@link ClientInsertOneResult}s.
* @see ClientBulkWriteException#getWriteErrors()
*/
Map<Integer, ClientInsertOneResult> getInsertResults();

/**
* The indexed {@link ClientUpdateResult}s.
* The {@linkplain Map#keySet() keys} are the indexes of the corresponding {@link ClientWriteModel}s
* in the client-level bulk write operation.
* <p>
* There are no guarantees on mutability or iteration order of the {@link Map} returned.</p>
*
* @return The indexed {@link ClientUpdateResult}s.
* @see ClientBulkWriteException#getWriteErrors()
*/
Map<Integer, ClientUpdateResult> getUpdateResults();

/**
* The indexed {@link ClientDeleteResult}s.
* The {@linkplain Map#keySet() keys} are the indexes of the corresponding {@link ClientWriteModel}s
* in the client-level bulk write operation.
* <p>
* There are no guarantees on mutability or iteration order of the {@link Map} returned.</p>
*
* @return The indexed {@link ClientDeleteResult}s.
* @see ClientBulkWriteException#getWriteErrors()
*/
Map<Integer, ClientDeleteResult> getDeleteResults();
}
}
Loading