Enhance Alternator API by m-szymon · Pull Request #158 · scylladb/latte

m-szymon · 2026-04-29T09:49:59Z

Refactor ConnectionConf and add custom credentials support for Alternator

Refectored ConnectionConf so it has a database specific
DbConnectionConf field. This way the database specific code is contained
inside the scripting folder.
Added Alternator specific credential options to its DbConnectionConf
and used them in connect.
Edited both the CQL and Alternator config to not serialize
passwords/secrets. This stops the secrets from being saved to generated
raports.

The Alternator credential options work as follows:

if aws-credentials is specified, the default AWS SDK credentials
flow will be used, i.e. the SDK will try to fetch the credentials from
the environment
The access-key-id and secret-access-key options can be used to
specify the keys, otherwise the default empty values ("") are used

Add support for DynamoDB Sets to Alternator.

Implemented support for passing and receiving of DynamoDB Sets to Rune.

They are implemented as objects with exactly one key with a special
name. The values are stored as a vector of appropriate types.
Convenience functions used to create the sets are provided to Rune.

Implement batch_get_item and batch_write_item.

Implemented the functions and added workflows that demonstrates the
usage.

Batch operations support the get_unprocessed option that disables the
auto pagination feature and allows the user to do it manually from rune.

Implementing the auto pagination feature required editing some existing
traits.

Follow-up #138

ConnectionConf now only has fields that are shared between different databases. Database specific fields are moved to DbConnectionConf inside the scripting folder. (cherry picked from commit 5e4661e)

The config allows for specifying the credentials as command line arguments. By specifying the aws-credentials option the SDK will fetch credentials from the environment. (cherry picked from commit 1d37fa9)

Used serde(skip_serializing) to avoid serializing secret fields. (cherry picked from commit 62669ca)

Sets are represented as special objects with a single specific key. The items of the sets are stored in a vector. (cherry picked from commit d775dfc)

…les. (cherry picked from commit a1dd717)

PaginationToken will allow for retrying both normal Query requests and batch requests. (cherry picked from commit 0313ea2)

Implemented the functions with automatic retry and added a workflow that demonstrates the usage. (cherry picked from commit 7cac86f)

Added new `get_unprocessed` to batch write and get. Modified `handle_request` and added `format_batch_result` to properly return the unprocessed items. (cherry picked from commit a0f9ecb)

vponomaryov · 2026-04-30T18:13:09Z

+    /// Use AWS credentials and region from the environment.
+    /// If this flag is set, access key ID, secret access key, and region args will be ignored.
+    #[clap(long("aws-credentials"))]
+    pub aws_credentials: bool,


I think that the aws-credentials must be mutually exclusive to other 3 on the config validation level.

So, user will know he needs to specify this information in one or another way.
It will not make him to remember the order of significance...

Used conflicts_with_all to mark this as mutually exclusive with access-key-id, secret-access-key and region.

vponomaryov · 2026-04-30T18:20:23Z

+
+/// Marks a list of items as an Alternator string set.
+#[rune::function]
+pub fn sset(items: Vec<Value>) -> VmResult<Value> {


Such short names are pretty cryptic, IMHO.
I would like to see more intuitive ones reading rune scripts - such as string_set, number_set and bytes_set (or binary_set)?

DynamoDB uses even more cryptic names: Ss, Ns and Bs haha.

But good point, changed them to the names you proposed.

vponomaryov · 2026-04-30T18:31:38Z

+}
+
+pub async fn run(db, i) {
+    let batch_size = 5;


What happens when we use bigger sizes?
What do we expect to happen?

Shouldn't the oversize be covered in rune scripts?

In Alternator/DynamoDB the maximum batch size is limited to 100 by default, but this is configurable on the DB side (alternator_max_items_in_batch_write).

Providing more items will cause the request to fail and will be reported to rune as an error. I don't think we should do anything about it in our wrapper, the database already provides a descriptive error for it.

vponomaryov · 2026-04-30T18:33:58Z

+            type: "put",
+            item: #{
+                pk: base_id + j.to_string(),
+                data: "batch_item_" + j.to_string()


Double transformation of the j into a string.
Also, it may be written shorter removing double transformation:

pk: `base_id${j}`, data: `batch_item_${j}`

vponomaryov · 2026-04-30T18:34:46Z

+
+pub async fn run(db, i) {
+    let batch_size = 5;
+    let base_id = "user_" + i.to_string() + "_";


let base_id = `user_${i}_`;

?

vponomaryov · 2026-04-30T18:37:52Z

+        if let Some(unprocessed) = res.get("unprocessed_items") {
+            current_writes = unprocessed;
+        } else {
+            current_writes = []; // All items processed


Shouldn't the break do what you need?
It looks exactly the case for it.

m-szymon · 2026-05-05T09:45:25Z

@vponomaryov thanks for already reviewing.
@stopnoanime please have a look on the comments.

stopnoanime · 2026-05-05T19:06:22Z

I made a PR with the fixes in our repo: scylladb-zpp-2025-alternator-rs-driver#45

vponomaryov · 2026-05-05T19:55:20Z

@m-szymon
Why is it in draft state?
What is the expectation for the review changes merging? Going to be part of this PR? or a follow-up one?

m-szymon · 2026-05-06T08:52:42Z

@m-szymon Why is it in draft state? What is the expectation for the review changes merging? Going to be part of this PR? or a follow-up one?

@vponomaryov you were too quick with review - I created draft, but I was not sure if we want to add something, especially that we were syncing with some other changes from master.

I will rebase and include fixes in this PR.

stopnoanime added 8 commits April 29, 2026 11:30

Refactor ConnectionConf to allow for database specific options.

79655ad

ConnectionConf now only has fields that are shared between different databases. Database specific fields are moved to DbConnectionConf inside the scripting folder. (cherry picked from commit 5e4661e)

Add credentials to Alternator's DbConnectionConf and use them in connect

3b7eb8f

The config allows for specifying the credentials as command line arguments. By specifying the aws-credentials option the SDK will fetch credentials from the environment. (cherry picked from commit 1d37fa9)

Changed to not save secrets to generated reports.

c258dc1

Used serde(skip_serializing) to avoid serializing secret fields. (cherry picked from commit 62669ca)

Add support for Alternator Sets to Rune type converter.

217a963

Sets are represented as special objects with a single specific key. The items of the sets are stored in a vector. (cherry picked from commit d775dfc)

Add helper functions for creating Alternator Sets and add usage examp…

6ecf5f4

…les. (cherry picked from commit a1dd717)

Refactor Alternator Output trait to add PaginationToken.

8916eb6

PaginationToken will allow for retrying both normal Query requests and batch requests. (cherry picked from commit 0313ea2)

Implement batch_get_item and batch_write_item.

58b315e

Implemented the functions with automatic retry and added a workflow that demonstrates the usage. (cherry picked from commit 7cac86f)

Add ability to manually handle batch unprocessed items.

eabbc19

Added new `get_unprocessed` to batch write and get. Modified `handle_request` and added `format_batch_result` to properly return the unprocessed items. (cherry picked from commit a0f9ecb)

vponomaryov reviewed Apr 30, 2026

View reviewed changes

Conversation

m-szymon commented Apr 29, 2026

Refactor ConnectionConf and add custom credentials support for Alternator

Add support for DynamoDB Sets to Alternator.

Implement batch_get_item and batch_write_item.

Uh oh!

vponomaryov Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vponomaryov Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vponomaryov Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

m-szymon commented May 5, 2026

Uh oh!

stopnoanime commented May 5, 2026

Uh oh!

vponomaryov commented May 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

m-szymon commented May 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

vponomaryov Apr 30, 2026 •

edited

Loading

vponomaryov Apr 30, 2026 •

edited

Loading

vponomaryov Apr 30, 2026 •

edited

Loading

vponomaryov commented May 5, 2026 •

edited

Loading