Updated RIOT example

.gitignore

@@ -326,3 +326,7 @@ Homestead.yaml
 Homestead.json
 /.vagrant
 .phpunit.result.cache
+
+# MacOS Directories
+
+.DS_Store

riot-example/LICENSE

@@ -0,0 +1,16 @@
+MIT No Attribution
+
+Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of
+this software and associated documentation files (the "Software"), to deal in
+the Software without restriction, including without limitation the rights to
+use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
+the Software, and to permit persons to whom the Software is furnished to do so.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
+FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
+COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

riot-example/README.md

@@ -1,50 +1,131 @@
-# riot-example
+# riot-migration
 
-# Introduction
+## Purpose
 
-This is a simple example of how to use the riot utility to migrate data from a Valkey / Redis OSS cluster to Amazon ElastiCache. This guide is only an example, it does not provide prescriptive guidance.
+This repo contains files to assist in migrations from an existing
+self-managed Redis OSS cluster to Amazon ElastiCache or
+Amazon MemoryDB.
 
-## Step 1 - Install Valkey CLI on EC2
+## Files
 
-On an EC2 system with Amazon Linux 2023 image running, run the following commands to install the Valkey server and Valkey CLI:
+**`env.sh`** - manages all of the env variables. If any of these variables are
+commented out, they simply won't be included when launching the `replicate` or
+`generate` scripts.
 
-```console
-wget https://download.valkey.io/releases/valkey-7.2.5-focal-x86_64.tar.gz
-tar xvzf valkey-7.2.5-focal-x86_64.tar.gz
-rm valkey-7.2.5-focal-x86_64.tar.gz
-mv valkey-7.2.5-focal-x86_64 valkey
-echo "export PATH=\$PATH:$HOME/valkey/bin" >> ~/.bashrc
-source ~/.bashrc
+**`generate`** - calls the RIOT tool with the 'generate' action, passing all
+parameters from the `env.sh` file. Execute via `./generate`
+
+**`replicate`** - calls the RIOT tool with the 'replicate' action, passing all
+parameters from the `env.sh` file. Execute via `./replicate`
+
+**`admin`** - a shell script that allows you to perform some helpful tasks
+against primary, such as summarizing key count, memory usage, etc. Run
+`./admin --help` for more information.
+
+**`userdata.sh`** - This file can be pasted into an Amazon EC2 `userdata`
+section to be used upon initial boot of an EC2 node. Useful to provide in
+a Launch Template to automatically setup a Redis OSS environment across
+multiple nodes. Currently deploys Redis OSS 5.0.10 in cluster mode. This
+section will need comprehensive documentation in the future.
+
+## Requirements
+
+- Ensure Java is installed.
+- Have at least one Valkey or Redis OSS cluster as a SOURCE clusterthat
+you can reach from the node you install this on. (see `env.sh` for more).
+
+## Install
+
+1. Clone this repo
+
+2. Download the RIOT binaries into a local folder:
+
+    ```sh
+    wget https://github.com/redis/riot/releases/download/early-access/riot-early-access.zip
+    ```
+
+3. Unzip the zip file.
+
+4. Change directories into the newly created directory, and copy to the repo
+directory from step 1. Example:
+
+```sh
+cd riot-4.2.3-SNAPSHOT
+cp -r * myhomedir/amazon-elasticache-samples/riot-example
 ```
 
-## Step 2 - Download riot docker image
+## Running
+
+1. Modify the `env.sh` file appropriately
 
-This docker image will perform the actual migration work:
+2. Run a test via `./admin source primaries` to see if it can connect to show
+the source cluster primary nodes.
 
+3. Run a test via `./admin target primaries` to see if it can connect to show
+the source cluster replica nodes.
+
+## admin usage
+
+As mentioned earlier, the `admin` script is provided as a helpful tool during a
+migration to view source and target cluster information. Run `./admin --help`
+for more information. Here are some examples of its usage:
+
+---
+
+`./admin source dups`
+
+```text
+Fetching keys from 172.31.20.133...
+Fetching keys from 172.31.24.184...
+Fetching keys from 172.31.30.162...
+No duplicates
 ```
-docker pull riotx/riot
+
+`./admin source memory`
+
+```text
+172.31.20.133 : 864 MB
+172.31.24.184 : 864 MB
+172.31.30.162 : 864 MB
+Total : 2,592 MB
 ```
 
-## Step 3 - Update env variables
+`./admin source keys`
+
+```text
+172.31.20.133 : 184,182
+172.31.24.184 : 184,161
+172.31.30.162 : 184,257
+Total: 552,600
+```
 
-Update the file `env.sh`. It has most of the values that the tool uses, plus credentials for clusters. Update it with source and target cluster information, such as:
+`./admin source bgsave`
 
+```text
+Executing BGSAVE on 172.31.20.133...Background saving started
+Executing BGSAVE on 172.31.24.184...Background saving started
+Executing BGSAVE on 172.31.30.162...Background saving started
 ```
-export SOURCE_CLUSTER=dns_name_of_cluster
-export SOURCE_USER=default
-export SOURCE_PASS=your_password
-export SOURCE_PORT=6379
-
-export ELASTICACHE_SLS=elasticache_endpoint.use1.cache.amazonaws.com
-export ELASTICACHE_USER=default
-export ELASTICACHE_PASS=elasticache_user_password
-export ELASTICACHE_PORT=6379
+
+`./admin source flush`
+
+```text
+Flushing 172.31.20.133...OK
+Flushing 172.31.24.184...OK
+Flushing 172.31.30.162...OK
 ```
 
-## Step 4 - Generate data
+## RIOT usage
+
+This repo is designed to call the RIOT application via scripts that include
+parameters that are passed into RIOT.
 
-Use the [generate.sh](generate.sh) script to create data in the source cluster.
+Modify the `env.sh` file, ensuring the `GLOBAL_DRY_RUN` variable is set to
+`--dry-run`. Then run the `./replicate` script to see a dry run of the
+migration effort.
 
-## Step 5 - Migrate data
+## Populating data for testing
 
-Use the [replicate.sh](replicate.sh) script to replicate data from source cluster to target cluster.
+You may also use the `./generate` script after modifying the `env.sh` file to
+generate mock data. This is helpful to see if the keys from the source cluster
+(generated via `./generate`) exist after migration (via `./replicate`).