Skip to content

Commit 1d56433

Browse files
feat: Add FindNearest API to the stable branch (#1333)
* feat: Support for field update operators in the Datastore API and resolution strategies when there is a conflict at write time PiperOrigin-RevId: 683253625 Source-Link: googleapis/googleapis@3effbf2 Source-Link: googleapis/googleapis-gen@5dd983c Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNWRkOTgzYzc2NDE3ZjJhZDg4ZjlkNDc0MzhjNDhjMjdkNWFjMGUyNyJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * feat: Add FindNearest API to the stable branch PiperOrigin-RevId: 684905940 Source-Link: googleapis/googleapis@2196d48 Source-Link: googleapis/googleapis-gen@05df6fa Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMDVkZjZmYTE2YWI5M2JkOWRhMTdiNTZlZGQzNzliNDM5NjNkZTE2NyJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: danieljbruce <[email protected]>
1 parent 519e33b commit 1d56433

File tree

5 files changed

+1698
-11
lines changed

5 files changed

+1698
-11
lines changed

protos/google/datastore/v1/datastore.proto

+133-8
Original file line numberDiff line numberDiff line change
@@ -501,6 +501,18 @@ message ReserveIdsResponse {}
501501

502502
// A mutation to apply to an entity.
503503
message Mutation {
504+
// The possible ways to resolve a conflict detected in a mutation.
505+
enum ConflictResolutionStrategy {
506+
// Unspecified. Defaults to `SERVER_VALUE`.
507+
STRATEGY_UNSPECIFIED = 0;
508+
509+
// The server entity is kept.
510+
SERVER_VALUE = 1;
511+
512+
// The whole commit request fails.
513+
FAIL = 3;
514+
}
515+
504516
// The mutation operation.
505517
//
506518
// For `insert`, `update`, and `upsert`:
@@ -542,6 +554,11 @@ message Mutation {
542554
google.protobuf.Timestamp update_time = 11;
543555
}
544556

557+
// The strategy to use when a conflict is detected. Defaults to
558+
// `SERVER_VALUE`.
559+
// If this is set, then `conflict_detection_strategy` must also be set.
560+
ConflictResolutionStrategy conflict_resolution_strategy = 10;
561+
545562
// The properties to write in this mutation.
546563
// None of the properties in the mask may have a reserved name, except for
547564
// `__key__`.
@@ -551,6 +568,112 @@ message Mutation {
551568
// updated, others are left untouched.
552569
// Properties referenced in the mask but not in the entity are deleted.
553570
PropertyMask property_mask = 9;
571+
572+
// Optional. The transforms to perform on the entity.
573+
//
574+
// This field can be set only when the operation is `insert`, `update`,
575+
// or `upsert`. If present, the transforms are be applied to the entity
576+
// regardless of the property mask, in order, after the operation.
577+
repeated PropertyTransform property_transforms = 12
578+
[(google.api.field_behavior) = OPTIONAL];
579+
}
580+
581+
// A transformation of an entity property.
582+
message PropertyTransform {
583+
// A value that is calculated by the server.
584+
enum ServerValue {
585+
// Unspecified. This value must not be used.
586+
SERVER_VALUE_UNSPECIFIED = 0;
587+
588+
// The time at which the server processed the request, with millisecond
589+
// precision. If used on multiple properties (same or different entities)
590+
// in a transaction, all the properties will get the same server timestamp.
591+
REQUEST_TIME = 1;
592+
}
593+
594+
// Optional. The name of the property.
595+
//
596+
// Property paths (a list of property names separated by dots (`.`)) may be
597+
// used to refer to properties inside entity values. For example `foo.bar`
598+
// means the property `bar` inside the entity property `foo`.
599+
//
600+
// If a property name contains a dot `.` or a backlslash `\`, then that name
601+
// must be escaped.
602+
string property = 1 [(google.api.field_behavior) = OPTIONAL];
603+
604+
// The transformation to apply to the property.
605+
oneof transform_type {
606+
// Sets the property to the given server value.
607+
ServerValue set_to_server_value = 2;
608+
609+
// Adds the given value to the property's current value.
610+
//
611+
// This must be an integer or a double value.
612+
// If the property is not an integer or double, or if the property does not
613+
// yet exist, the transformation will set the property to the given value.
614+
// If either of the given value or the current property value are doubles,
615+
// both values will be interpreted as doubles. Double arithmetic and
616+
// representation of double values follows IEEE 754 semantics.
617+
// If there is positive/negative integer overflow, the property is resolved
618+
// to the largest magnitude positive/negative integer.
619+
Value increment = 3;
620+
621+
// Sets the property to the maximum of its current value and the given
622+
// value.
623+
//
624+
// This must be an integer or a double value.
625+
// If the property is not an integer or double, or if the property does not
626+
// yet exist, the transformation will set the property to the given value.
627+
// If a maximum operation is applied where the property and the input value
628+
// are of mixed types (that is - one is an integer and one is a double)
629+
// the property takes on the type of the larger operand. If the operands are
630+
// equivalent (e.g. 3 and 3.0), the property does not change.
631+
// 0, 0.0, and -0.0 are all zero. The maximum of a zero stored value and
632+
// zero input value is always the stored value.
633+
// The maximum of any numeric value x and NaN is NaN.
634+
Value maximum = 4;
635+
636+
// Sets the property to the minimum of its current value and the given
637+
// value.
638+
//
639+
// This must be an integer or a double value.
640+
// If the property is not an integer or double, or if the property does not
641+
// yet exist, the transformation will set the property to the input value.
642+
// If a minimum operation is applied where the property and the input value
643+
// are of mixed types (that is - one is an integer and one is a double)
644+
// the property takes on the type of the smaller operand. If the operands
645+
// are equivalent (e.g. 3 and 3.0), the property does not change. 0, 0.0,
646+
// and -0.0 are all zero. The minimum of a zero stored value and zero input
647+
// value is always the stored value. The minimum of any numeric value x and
648+
// NaN is NaN.
649+
Value minimum = 5;
650+
651+
// Appends the given elements in order if they are not already present in
652+
// the current property value.
653+
// If the property is not an array, or if the property does not yet exist,
654+
// it is first set to the empty array.
655+
//
656+
// Equivalent numbers of different types (e.g. 3L and 3.0) are
657+
// considered equal when checking if a value is missing.
658+
// NaN is equal to NaN, and the null value is equal to the null value.
659+
// If the input contains multiple equivalent values, only the first will
660+
// be considered.
661+
//
662+
// The corresponding transform result will be the null value.
663+
ArrayValue append_missing_elements = 6;
664+
665+
// Removes all of the given elements from the array in the property.
666+
// If the property is not an array, or if the property does not yet exist,
667+
// it is set to the empty array.
668+
//
669+
// Equivalent numbers of different types (e.g. 3L and 3.0) are
670+
// considered equal when deciding whether an element should be removed.
671+
// NaN is equal to NaN, and the null value is equal to the null value.
672+
// This will remove all equivalent values if there are duplicates.
673+
//
674+
// The corresponding transform result will be the null value.
675+
ArrayValue remove_all_from_array = 7;
676+
}
554677
}
555678

556679
// The result of applying a mutation.
@@ -578,6 +701,11 @@ message MutationResult {
578701
// Whether a conflict was detected for this mutation. Always false when a
579702
// conflict detection strategy field is not set in the mutation.
580703
bool conflict_detected = 5;
704+
705+
// The results of applying each
706+
// [PropertyTransform][google.datastore.v1.PropertyTransform], in the same
707+
// order of the request.
708+
repeated Value transform_results = 8;
581709
}
582710

583711
// The set of arbitrarily nested property paths used to restrict an operation to
@@ -611,16 +739,13 @@ message ReadOptions {
611739
EVENTUAL = 2;
612740
}
613741

614-
// For Cloud Datastore, if read_consistency is not specified, then lookups and
615-
// ancestor queries default to `read_consistency`=`STRONG`, global queries
616-
// default to `read_consistency`=`EVENTUAL`.
617-
//
618-
// For Cloud Firestore in Datastore mode, if read_consistency is not specified
619-
// then lookups and all queries default to `read_consistency`=`STRONG`.
742+
// For Cloud Firestore in Datastore mode, if you don't specify
743+
// read_consistency then all lookups and queries default to
744+
// `read_consistency`=`STRONG`. Note that, in Cloud Datastore, global queries
745+
// defaulted to `read_consistency`=`EVENTUAL`.
620746
//
621747
// Explicitly setting `read_consistency`=`EVENTUAL` will result in eventually
622-
// consistent lookups & queries in both Cloud Datastore & Cloud Firestore in
623-
// Datastore mode.
748+
// consistent lookups and queries.
624749
oneof consistency_type {
625750
// The non-transactional read consistency to use.
626751
ReadConsistency read_consistency = 1;

protos/google/datastore/v1/query.proto

+80
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,15 @@ message EntityResult {
8282
}
8383

8484
// A query for entities.
85+
//
86+
// The query stages are executed in the following order:
87+
// 1. kind
88+
// 2. filter
89+
// 3. projection
90+
// 4. order + start_cursor + end_cursor
91+
// 5. offset
92+
// 6. limit
93+
// 7. find_nearest
8594
message Query {
8695
// The projection to return. Defaults to returning all properties.
8796
repeated Projection projection = 2;
@@ -127,6 +136,13 @@ message Query {
127136
// Unspecified is interpreted as no limit.
128137
// Must be >= 0 if specified.
129138
google.protobuf.Int32Value limit = 12;
139+
140+
// Optional. A potential Nearest Neighbors Search.
141+
//
142+
// Applies after all other filters and ordering.
143+
//
144+
// Finds the closest vector embeddings to the given query vector.
145+
FindNearest find_nearest = 13 [(google.api.field_behavior) = OPTIONAL];
130146
}
131147

132148
// Datastore query for running an aggregation over a
@@ -436,6 +452,70 @@ message PropertyFilter {
436452
Value value = 3;
437453
}
438454

455+
// Nearest Neighbors search config. The ordering provided by FindNearest
456+
// supersedes the order_by stage. If multiple documents have the same vector
457+
// distance, the returned document order is not guaranteed to be stable between
458+
// queries.
459+
message FindNearest {
460+
// The distance measure to use when comparing vectors.
461+
enum DistanceMeasure {
462+
// Should not be set.
463+
DISTANCE_MEASURE_UNSPECIFIED = 0;
464+
465+
// Measures the EUCLIDEAN distance between the vectors. See
466+
// [Euclidean](https://en.wikipedia.org/wiki/Euclidean_distance) to learn
467+
// more. The resulting distance decreases the more similar two vectors are.
468+
EUCLIDEAN = 1;
469+
470+
// COSINE distance compares vectors based on the angle between them, which
471+
// allows you to measure similarity that isn't based on the vectors
472+
// magnitude. We recommend using DOT_PRODUCT with unit normalized vectors
473+
// instead of COSINE distance, which is mathematically equivalent with
474+
// better performance. See [Cosine
475+
// Similarity](https://en.wikipedia.org/wiki/Cosine_similarity) to learn
476+
// more about COSINE similarity and COSINE distance. The resulting COSINE
477+
// distance decreases the more similar two vectors are.
478+
COSINE = 2;
479+
480+
// Similar to cosine but is affected by the magnitude of the vectors. See
481+
// [Dot Product](https://en.wikipedia.org/wiki/Dot_product) to learn more.
482+
// The resulting distance increases the more similar two vectors are.
483+
DOT_PRODUCT = 3;
484+
}
485+
486+
// Required. An indexed vector property to search upon. Only documents which
487+
// contain vectors whose dimensionality match the query_vector can be
488+
// returned.
489+
PropertyReference vector_property = 1
490+
[(google.api.field_behavior) = REQUIRED];
491+
492+
// Required. The query vector that we are searching on. Must be a vector of no
493+
// more than 2048 dimensions.
494+
Value query_vector = 2 [(google.api.field_behavior) = REQUIRED];
495+
496+
// Required. The Distance Measure to use, required.
497+
DistanceMeasure distance_measure = 3 [(google.api.field_behavior) = REQUIRED];
498+
499+
// Required. The number of nearest neighbors to return. Must be a positive
500+
// integer of no more than 100.
501+
google.protobuf.Int32Value limit = 4 [(google.api.field_behavior) = REQUIRED];
502+
503+
// Optional. Optional name of the field to output the result of the vector
504+
// distance calculation. Must conform to [entity
505+
// property][google.datastore.v1.Entity.properties] limitations.
506+
string distance_result_property = 5 [(google.api.field_behavior) = OPTIONAL];
507+
508+
// Optional. Option to specify a threshold for which no less similar documents
509+
// will be returned. The behavior of the specified `distance_measure` will
510+
// affect the meaning of the distance threshold. Since DOT_PRODUCT distances
511+
// increase when the vectors are more similar, the comparison is inverted.
512+
//
513+
// For EUCLIDEAN, COSINE: WHERE distance <= distance_threshold
514+
// For DOT_PRODUCT: WHERE distance >= distance_threshold
515+
google.protobuf.DoubleValue distance_threshold = 6
516+
[(google.api.field_behavior) = OPTIONAL];
517+
}
518+
439519
// A [GQL
440520
// query](https://cloud.google.com/datastore/docs/apis/gql/gql_reference).
441521
message GqlQuery {

0 commit comments

Comments
 (0)