Merge pull request #11 from smithmicro/feature-10

Using ECS-CLI to streamline Lucy
smithmicro · Jan 10, 2018 · e561b4e · e561b4e
2 parents 2e1fdbf + 05cd2dc
commit e561b4e
Show file tree

Hide file tree

Showing 6 changed files with 159 additions and 257 deletions.
diff --git a/README.md b/README.md
@@ -12,7 +12,7 @@ The `smithmicro/lucy` Docker image can be run as-is with a number of required en
 
 Prerequisites to use this image:
 * Create a VPC with at least one subnet as ECS requires the use of VPC **
-* Create a VPC security group that allows ports 22, 1099, 50000 and 51000 (tcp) to the VPC **
+* Create a VPC security group that allows ports 22, 1099, 50000 and 51000 (tcp) and 4445 (udp) to the VPC **
 * Create a security key pair and place in the `keys` subdirectory
 * Have your AWS CLI Access Key ID/Secret Access Key handy
 * Replace or edit the included `plans/demo.jmx` to run your specific tests
@@ -31,7 +31,6 @@ docker run -v <oath to jmx>:/plans -v <path to pem>:/keys -v <path to logs>:/log
     --env SUBNET_ID=<subnet ID within your VPC> \
     --env KEY_NAME=<key pair name without extension> \
     --env MINION_COUNT=<number of minions> \
-    --enc INSTANCE_TYPE=<valid ECS instance type> \
     smithmicro/lucy /plans/demo.jmx
 ```
 For 5 test instances in N. Virginia, `docker run` would look like this, assuming your `jmeter-key.pem` file is located in the `keys` subdirectory:
@@ -44,50 +43,44 @@ docker run -v $PWD/plans:/plans -v $PWD/keys:/keys -v $PWD/logs:/logs \
     --env SUBNET_ID=subnet-12345678 \
     --env KEY_NAME=jmeter-key \
     --env MINION_COUNT=5 \
-    --enc INSTANCE_TYPE=t2.small \
     smithmicro/lucy /plans/demo.jmx
 ```
 
 ## Architecture
-This Docker image replaces the JMeter master/slave nomenclature with *Gru*, *Minion* and *Lucy*.  *Gru* manages the *Minions* from within EC2, but *Lucy* orchestrates the entire process.
+This Docker image replaces the JMeter master/slave nomenclature with *Gru*, *Minion* and *Lucy*.  *Gru* manages the *Minions* from within ECS, but *Lucy* orchestrates the entire process.
 
 ```
-+-------------------------------------+
-|  EC2           +-----------------+  |
-|                |  ECS            |  |
-|                | +--------+      |  |
-|  +---------+   | | +--------+    |  |      +--------+
-|  |         |---->| | +--------+ ---------->|        |
-|  |   Gru   |<----| | |        | ---------->| Target |
-|  |         |   | +-| | Minion | ---------->|        |
-|  +---------+   |   +-|        |  |  |      +--------+
-|      ^ |       |     +--------+  |  |
-|      | |       +-----------------+  |
-+------|-|----------------------------+
-       | |
-  .jmx | | .log/.jtl
-       | v
-   +----------+
-   |          |
-   |   Lucy   |
-   |          |
-   +----------+
++--------------------------------------+
+|  EC2                                 |
+|  +--------------------------------+  |
+|  |  ECS                           |  |
+|  |                +--------+      |  |
+|  |  +-------+     | +--------+    |  |      +--------+
+|  |  |       |---->| | +--------+ ---------->|        |
+|  |  |  Gru  |<----| | |        | ---------->| Target |
+|  |  |       |     +-| | Minion | ---------->|        |
+|  |  +-------+       +-|        |  |  |      +--------+
+|  |     ^ |            +--------+  |  |
+|  +-----|-|------------------------+  |
++--------|-|---------------------------+
+         | |
+    .jmx | | .log/.jtl
+         | v
+     +----------+
+     |          |
+     |   Lucy   |
+     |          |
+     +----------+
 ```
 
 *Lucy* runs the `lucy.sh` script to perform the following steps:
-* Step 1 - Create an ECS Cluster
-* Step 2 - Create all instances and register them with the Cluster
-* Step 3 - Create the Minion ECS task
-* Step 4 - Wait until the instances are running and registered with the Cluster
-* Step 5 - Fetch our Contatiner Instance IDs
-* Step 6 - Run a Minion Task with the requested instance count
-* Step 7 - Get public IP addresses from Gru and Minions
-* Step 8 - Run Gru with the specified JMX
-  * JMeter does its thing here
-  * Once complete, copy the jmeter.log and results.jtl files from Gru to Lucy
-* Step 9 - Stop all Tasks
-* Step 10 - Terminate all instances
-* Step 11 - Delete the cluster
+* Step 1 - Create 2 ECS Clusters
+* Step 2 - Fetch our Contatiner Instance IDs
+* Step 3 - Run a Minion Task with the requested instance count
+* Step 4 - Get IP addresses from Gru and Minions
+* Step 5 - Run Gru with the specified JMX
+* Step 6 - Fetch the results
+* Step 7 - Delete the clusters
 
 ### Volumes
 The `lucy` container uses 3 volumes:
@@ -128,6 +121,28 @@ docker-compose up
 ```
 Using the `docker-compose scale` command does not work as it creates hostnames like `minion_1`.  This causes an error in JMeter as it uses the hostname in URL form and sees the underscore as an illegal URL character.
 
+## Notes
+The following required and optional environment variables are supported:
+
+| Variable | Required | Default | Notes |
+|---|---|---|---|
+|AWS_DEFAULT_REGION|Yes|None|AWS Region (e.g. us-east-1)|
+|AWS_ACCESS_KEY_ID|Yes|None|AWS Access Key|
+|AWS_SECRET_ACCESS_KEY|Yes|None|AWS Secret Key|
+|INPUT_JMX|Yes|None|File path of JMeter Test file to run (.jmx).  You can optionally specify this as the first command line option of `docker run`|
+|KEY_NAME|Yes|None|AWS Security Key Pair .pem file (do not specify the .pem extension)|
+|SECURITY_GROUP|Yes|None|AWS Secuirty group that allows ports 22,1099,50000,51000/tcp and 4445/udp from all ports (e.g. sg-12345678)|
+|SUBNET_ID|Yes|None|One or more Subnets that are assigned to your VPC|
+|VPC_ID||VPC assigned to SUBNET_ID|We dautomatically erive this from your SUBNET_ID|
+|JMETER_VERSION||latest|smithmicro/lucy Image tag.  See Docker Hub for [available versions](https://hub.docker.com/r/smithmicro/jmeter/tags/).|
+|INSTANCE_TYPE||t2.micro|To double your memory, pass t2.small|
+|MEM_LIMIT||950m|If you are using t2.small, set MEM_LIMIT to 1995m|
+|MINION_COUNT||2||
+|PEM_PATH||/keys|This must match your Volume map.  See Volume section above.|
+|MINION_CLUSTER_NAME||JMeterMinion|Name that appears in your AWS Cluster UI|
+|GRU_CLUSTER_NAME||JMeterGru|Name that appears in your AWS Cluster UI|
+|GRU_PRIVATE_IP||(blank)|Set to true if you would like to run Lucy within AWS.  See GitHub [Issue 8](https://github.com/smithmicro/jmeter-ecs/issues/8) for details.|
+
 ## Notes
 This Docker image uses the Instance Metadata API documented here:
 * http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html
@@ -138,6 +153,7 @@ To get the instance public hostname within the `entrypoint.sh` script, we call:
 For more information on JMeter Distributed Testing, see:
 * http://jmeter.apache.org/usermanual/remote-test.html
 
+
 ## Inspired by...
 https://en.wikipedia.org/wiki/Despicable_Me_2
 

diff --git a/aws-setup.sh b/aws-setup.sh
@@ -9,9 +9,13 @@ if [ "$CIDR_BLOCK" == '' ]; then
   # create a CIDR block at 10.74, the 74 being ASCII 'J'
   CIDR_BLOCK=10.74.0.0/16
 fi
-if [ "$SUBNET_CIDR_BLOCK" == '' ]; then
+if [ "$SUBNET_CIDR_BLOCK1" == '' ]; then
   # this CIDR limits us to 251 JMeter Minions - protection from a typo trying to create 1000 instances
-  SUBNET_CIDR_BLOCK=10.74.1.0/24
+  SUBNET_CIDR_BLOCK1=10.74.1.0/24
+fi
+if [ "$SUBNET_CIDR_BLOCK2" == '' ]; then
+  # this CIDR limits us to 251 JMeter Minions - protection from a typo trying to create 1000 instances
+  SUBNET_CIDR_BLOCK2=10.74.2.0/24
 fi
 if [ "$OWNER" == '' ]; then
   OWNER=jmeter-ecs
@@ -39,10 +43,12 @@ echo "Created VPC $VPC_ID"
 # enable DNS hostnames
 aws ec2 modify-vpc-attribute --vpc-id $VPC_ID --enable-dns-hostnames --output text
 
-# create a single subnet
-SUBNET_ID=$(aws ec2 create-subnet --vpc-id $VPC_ID --cidr-block $SUBNET_CIDR_BLOCK \
+# create a 2 subnets
+SUBNET_ID1=$(aws ec2 create-subnet --vpc-id $VPC_ID --cidr-block $SUBNET_CIDR_BLOCK1 \
+    --query 'Subnet.[SubnetId]' --output text | tr -d '\n')
+SUBNET_ID2=$(aws ec2 create-subnet --vpc-id $VPC_ID --cidr-block $SUBNET_CIDR_BLOCK2 \
     --query 'Subnet.[SubnetId]' --output text | tr -d '\n')
-echo "Created Subnet $SUBNET_ID"
+echo "Created Subnets $SUBNET_ID1,$SUBNET_ID2"
 
 # Step 2: Make Your Subnet Public
 IGW_ID=$(aws ec2 create-internet-gateway --query 'InternetGateway.[InternetGatewayId]' --output text | tr -d '\n')
@@ -59,12 +65,14 @@ if [ "$CREATE_ROUTE_RESULT" == 'True' ]; then
   echo "Created route for all traffic to the Internet Gateway"
 fi
 
-# make this a public subnet
-RTBASSOC_ID=$(aws ec2 associate-route-table  --subnet-id $SUBNET_ID --route-table-id $RTB_ID --output text | tr -d '\n')
-echo "Created Route Table Association $RTBASSOC_ID"
+# make these public subnet
+RTBASSOC_ID1=$(aws ec2 associate-route-table --subnet-id $SUBNET_ID1 --route-table-id $RTB_ID --output text | tr -d '\n')
+RTBASSOC_ID2=$(aws ec2 associate-route-table --subnet-id $SUBNET_ID2 --route-table-id $RTB_ID --output text | tr -d '\n')
+echo "Created Route Table Associations $RTBASSOC_ID1,$RTBASSOC_ID2"
 
 # we need public IP addresses so instances can register with ECS clusters
-aws ec2 modify-subnet-attribute --subnet-id $SUBNET_ID --map-public-ip-on-launch
+aws ec2 modify-subnet-attribute --subnet-id $SUBNET_ID1 --map-public-ip-on-launch
+aws ec2 modify-subnet-attribute --subnet-id $SUBNET_ID2 --map-public-ip-on-launch
 
 # create a security group for JMeter
 SG_ID=$(aws ec2 create-security-group --group-name "JMeter" --description "JMeter Security Group" --vpc-id $VPC_ID --output text | tr -d '\n')
@@ -74,11 +82,11 @@ JMETER_IP_PERMISSIONS='[{"IpProtocol": "tcp", "FromPort": 22, "ToPort": 22, "IpR
 aws ec2 authorize-security-group-ingress --group-id $SG_ID --ip-permissions "$JMETER_IP_PERMISSIONS"
 
 # tag all created resources
-aws ec2 create-tags --resources $VPC_ID $SUBNET_ID $IGW_ID $RTB_ID $SG_ID --tags $VPC_TAGS --output text
+aws ec2 create-tags --resources $VPC_ID $SUBNET_ID1 $SUBNET_ID2 $IGW_ID $RTB_ID $SG_ID --tags $VPC_TAGS --output text
 
 echo "******** Use these two enviroment variables in 'docker run'"
-echo "  --env SUBNET_ID=$SUBNET_ID'"
-echo "  --env SECURITY_GROUP=$SG_ID'"
+echo "  --env SUBNET_ID=$SUBNET_ID1,$SUBNET_ID2"
+echo "  --env SECURITY_GROUP=$SG_ID"
 echo "********"
 
 # ensure we have the Role name 'ecsInstanceRole' created

diff --git a/lucy/Dockerfile b/lucy/Dockerfile
@@ -12,14 +12,20 @@ ENV AWS_DEFAULT_REGION=
 
 # Install the AWS CLI
 RUN apk add --update --no-cache \
+    ca-certificates \
     openssh-client \
+    openssl \
     python \
     py-pip \
   && pip install \
     awscli
 
-# copy our entrypoint script and Task definition
-COPY lucy.sh minion.json /opt/jmeter/
+# Install the ECS CLI
+RUN wget -O /usr/local/bin/ecs-cli -q https://s3.amazonaws.com/amazon-ecs-cli/ecs-cli-linux-amd64-latest \
+  && chmod +x /usr/local/bin/ecs-cli
+
+# copy our entrypoint script and compose file for the Minions
+COPY lucy.sh lucy.yml /opt/jmeter/
 RUN chmod +x /opt/jmeter/lucy.sh
 
 WORKDIR /logs