diff --git a/README.md b/README.md index 32e68c3..a034999 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ The `smithmicro/lucy` Docker image can be run as-is with a number of required en Prerequisites to use this image: * Create a VPC with at least one subnet as ECS requires the use of VPC ** -* Create a VPC security group that allows ports 22, 1099, 50000 and 51000 (tcp) to the VPC ** +* Create a VPC security group that allows ports 22, 1099, 50000 and 51000 (tcp) and 4445 (udp) to the VPC ** * Create a security key pair and place in the `keys` subdirectory * Have your AWS CLI Access Key ID/Secret Access Key handy * Replace or edit the included `plans/demo.jmx` to run your specific tests @@ -31,7 +31,6 @@ docker run -v :/plans -v :/keys -v :/log --env SUBNET_ID= \ --env KEY_NAME= \ --env MINION_COUNT= \ - --enc INSTANCE_TYPE= \ smithmicro/lucy /plans/demo.jmx ``` For 5 test instances in N. Virginia, `docker run` would look like this, assuming your `jmeter-key.pem` file is located in the `keys` subdirectory: @@ -44,50 +43,44 @@ docker run -v $PWD/plans:/plans -v $PWD/keys:/keys -v $PWD/logs:/logs \ --env SUBNET_ID=subnet-12345678 \ --env KEY_NAME=jmeter-key \ --env MINION_COUNT=5 \ - --enc INSTANCE_TYPE=t2.small \ smithmicro/lucy /plans/demo.jmx ``` ## Architecture -This Docker image replaces the JMeter master/slave nomenclature with *Gru*, *Minion* and *Lucy*. *Gru* manages the *Minions* from within EC2, but *Lucy* orchestrates the entire process. +This Docker image replaces the JMeter master/slave nomenclature with *Gru*, *Minion* and *Lucy*. *Gru* manages the *Minions* from within ECS, but *Lucy* orchestrates the entire process. ``` -+-------------------------------------+ -| EC2 +-----------------+ | -| | ECS | | -| | +--------+ | | -| +---------+ | | +--------+ | | +--------+ -| | |---->| | +--------+ ---------->| | -| | Gru |<----| | | | ---------->| Target | -| | | | +-| | Minion | ---------->| | -| +---------+ | +-| | | | +--------+ -| ^ | | +--------+ | | -| | | +-----------------+ | -+------|-|----------------------------+ - | | - .jmx | | .log/.jtl - | v - +----------+ - | | - | Lucy | - | | - +----------+ ++--------------------------------------+ +| EC2 | +| +--------------------------------+ | +| | ECS | | +| | +--------+ | | +| | +-------+ | +--------+ | | +--------+ +| | | |---->| | +--------+ ---------->| | +| | | Gru |<----| | | | ---------->| Target | +| | | | +-| | Minion | ---------->| | +| | +-------+ +-| | | | +--------+ +| | ^ | +--------+ | | +| +-----|-|------------------------+ | ++--------|-|---------------------------+ + | | + .jmx | | .log/.jtl + | v + +----------+ + | | + | Lucy | + | | + +----------+ ``` *Lucy* runs the `lucy.sh` script to perform the following steps: -* Step 1 - Create an ECS Cluster -* Step 2 - Create all instances and register them with the Cluster -* Step 3 - Create the Minion ECS task -* Step 4 - Wait until the instances are running and registered with the Cluster -* Step 5 - Fetch our Contatiner Instance IDs -* Step 6 - Run a Minion Task with the requested instance count -* Step 7 - Get public IP addresses from Gru and Minions -* Step 8 - Run Gru with the specified JMX - * JMeter does its thing here - * Once complete, copy the jmeter.log and results.jtl files from Gru to Lucy -* Step 9 - Stop all Tasks -* Step 10 - Terminate all instances -* Step 11 - Delete the cluster +* Step 1 - Create 2 ECS Clusters +* Step 2 - Fetch our Contatiner Instance IDs +* Step 3 - Run a Minion Task with the requested instance count +* Step 4 - Get IP addresses from Gru and Minions +* Step 5 - Run Gru with the specified JMX +* Step 6 - Fetch the results +* Step 7 - Delete the clusters ### Volumes The `lucy` container uses 3 volumes: @@ -128,6 +121,28 @@ docker-compose up ``` Using the `docker-compose scale` command does not work as it creates hostnames like `minion_1`. This causes an error in JMeter as it uses the hostname in URL form and sees the underscore as an illegal URL character. +## Notes +The following required and optional environment variables are supported: + +| Variable | Required | Default | Notes | +|---|---|---|---| +|AWS_DEFAULT_REGION|Yes|None|AWS Region (e.g. us-east-1)| +|AWS_ACCESS_KEY_ID|Yes|None|AWS Access Key| +|AWS_SECRET_ACCESS_KEY|Yes|None|AWS Secret Key| +|INPUT_JMX|Yes|None|File path of JMeter Test file to run (.jmx). You can optionally specify this as the first command line option of `docker run`| +|KEY_NAME|Yes|None|AWS Security Key Pair .pem file (do not specify the .pem extension)| +|SECURITY_GROUP|Yes|None|AWS Secuirty group that allows ports 22,1099,50000,51000/tcp and 4445/udp from all ports (e.g. sg-12345678)| +|SUBNET_ID|Yes|None|One or more Subnets that are assigned to your VPC| +|VPC_ID||VPC assigned to SUBNET_ID|We dautomatically erive this from your SUBNET_ID| +|JMETER_VERSION||latest|smithmicro/lucy Image tag. See Docker Hub for [available versions](https://hub.docker.com/r/smithmicro/jmeter/tags/).| +|INSTANCE_TYPE||t2.micro|To double your memory, pass t2.small| +|MEM_LIMIT||950m|If you are using t2.small, set MEM_LIMIT to 1995m| +|MINION_COUNT||2|| +|PEM_PATH||/keys|This must match your Volume map. See Volume section above.| +|MINION_CLUSTER_NAME||JMeterMinion|Name that appears in your AWS Cluster UI| +|GRU_CLUSTER_NAME||JMeterGru|Name that appears in your AWS Cluster UI| +|GRU_PRIVATE_IP||(blank)|Set to true if you would like to run Lucy within AWS. See GitHub [Issue 8](https://github.com/smithmicro/jmeter-ecs/issues/8) for details.| + ## Notes This Docker image uses the Instance Metadata API documented here: * http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html @@ -138,6 +153,7 @@ To get the instance public hostname within the `entrypoint.sh` script, we call: For more information on JMeter Distributed Testing, see: * http://jmeter.apache.org/usermanual/remote-test.html + ## Inspired by... https://en.wikipedia.org/wiki/Despicable_Me_2 diff --git a/aws-setup.sh b/aws-setup.sh index 57e786c..7c686b3 100755 --- a/aws-setup.sh +++ b/aws-setup.sh @@ -9,9 +9,13 @@ if [ "$CIDR_BLOCK" == '' ]; then # create a CIDR block at 10.74, the 74 being ASCII 'J' CIDR_BLOCK=10.74.0.0/16 fi -if [ "$SUBNET_CIDR_BLOCK" == '' ]; then +if [ "$SUBNET_CIDR_BLOCK1" == '' ]; then # this CIDR limits us to 251 JMeter Minions - protection from a typo trying to create 1000 instances - SUBNET_CIDR_BLOCK=10.74.1.0/24 + SUBNET_CIDR_BLOCK1=10.74.1.0/24 +fi +if [ "$SUBNET_CIDR_BLOCK2" == '' ]; then + # this CIDR limits us to 251 JMeter Minions - protection from a typo trying to create 1000 instances + SUBNET_CIDR_BLOCK2=10.74.2.0/24 fi if [ "$OWNER" == '' ]; then OWNER=jmeter-ecs @@ -39,10 +43,12 @@ echo "Created VPC $VPC_ID" # enable DNS hostnames aws ec2 modify-vpc-attribute --vpc-id $VPC_ID --enable-dns-hostnames --output text -# create a single subnet -SUBNET_ID=$(aws ec2 create-subnet --vpc-id $VPC_ID --cidr-block $SUBNET_CIDR_BLOCK \ +# create a 2 subnets +SUBNET_ID1=$(aws ec2 create-subnet --vpc-id $VPC_ID --cidr-block $SUBNET_CIDR_BLOCK1 \ + --query 'Subnet.[SubnetId]' --output text | tr -d '\n') +SUBNET_ID2=$(aws ec2 create-subnet --vpc-id $VPC_ID --cidr-block $SUBNET_CIDR_BLOCK2 \ --query 'Subnet.[SubnetId]' --output text | tr -d '\n') -echo "Created Subnet $SUBNET_ID" +echo "Created Subnets $SUBNET_ID1,$SUBNET_ID2" # Step 2: Make Your Subnet Public IGW_ID=$(aws ec2 create-internet-gateway --query 'InternetGateway.[InternetGatewayId]' --output text | tr -d '\n') @@ -59,12 +65,14 @@ if [ "$CREATE_ROUTE_RESULT" == 'True' ]; then echo "Created route for all traffic to the Internet Gateway" fi -# make this a public subnet -RTBASSOC_ID=$(aws ec2 associate-route-table --subnet-id $SUBNET_ID --route-table-id $RTB_ID --output text | tr -d '\n') -echo "Created Route Table Association $RTBASSOC_ID" +# make these public subnet +RTBASSOC_ID1=$(aws ec2 associate-route-table --subnet-id $SUBNET_ID1 --route-table-id $RTB_ID --output text | tr -d '\n') +RTBASSOC_ID2=$(aws ec2 associate-route-table --subnet-id $SUBNET_ID2 --route-table-id $RTB_ID --output text | tr -d '\n') +echo "Created Route Table Associations $RTBASSOC_ID1,$RTBASSOC_ID2" # we need public IP addresses so instances can register with ECS clusters -aws ec2 modify-subnet-attribute --subnet-id $SUBNET_ID --map-public-ip-on-launch +aws ec2 modify-subnet-attribute --subnet-id $SUBNET_ID1 --map-public-ip-on-launch +aws ec2 modify-subnet-attribute --subnet-id $SUBNET_ID2 --map-public-ip-on-launch # create a security group for JMeter SG_ID=$(aws ec2 create-security-group --group-name "JMeter" --description "JMeter Security Group" --vpc-id $VPC_ID --output text | tr -d '\n') @@ -74,11 +82,11 @@ JMETER_IP_PERMISSIONS='[{"IpProtocol": "tcp", "FromPort": 22, "ToPort": 22, "IpR aws ec2 authorize-security-group-ingress --group-id $SG_ID --ip-permissions "$JMETER_IP_PERMISSIONS" # tag all created resources -aws ec2 create-tags --resources $VPC_ID $SUBNET_ID $IGW_ID $RTB_ID $SG_ID --tags $VPC_TAGS --output text +aws ec2 create-tags --resources $VPC_ID $SUBNET_ID1 $SUBNET_ID2 $IGW_ID $RTB_ID $SG_ID --tags $VPC_TAGS --output text echo "******** Use these two enviroment variables in 'docker run'" -echo " --env SUBNET_ID=$SUBNET_ID'" -echo " --env SECURITY_GROUP=$SG_ID'" +echo " --env SUBNET_ID=$SUBNET_ID1,$SUBNET_ID2" +echo " --env SECURITY_GROUP=$SG_ID" echo "********" # ensure we have the Role name 'ecsInstanceRole' created diff --git a/lucy/Dockerfile b/lucy/Dockerfile index 9804738..1357d23 100644 --- a/lucy/Dockerfile +++ b/lucy/Dockerfile @@ -12,14 +12,20 @@ ENV AWS_DEFAULT_REGION= # Install the AWS CLI RUN apk add --update --no-cache \ + ca-certificates \ openssh-client \ + openssl \ python \ py-pip \ && pip install \ awscli -# copy our entrypoint script and Task definition -COPY lucy.sh minion.json /opt/jmeter/ +# Install the ECS CLI +RUN wget -O /usr/local/bin/ecs-cli -q https://s3.amazonaws.com/amazon-ecs-cli/ecs-cli-linux-amd64-latest \ + && chmod +x /usr/local/bin/ecs-cli + +# copy our entrypoint script and compose file for the Minions +COPY lucy.sh lucy.yml /opt/jmeter/ RUN chmod +x /opt/jmeter/lucy.sh WORKDIR /logs diff --git a/lucy/lucy.sh b/lucy/lucy.sh index e50d19f..d5e9bb9 100755 --- a/lucy/lucy.sh +++ b/lucy/lucy.sh @@ -2,8 +2,9 @@ # # jmeter-ecs Orchestrator, aka 'Lucy' -# Leverages the AWS ECS CLI tool: +# Leverages the AWS CLI tool and the AWS ECS CLI tool: # http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ECS_AWSCLI.html +# http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ECS_CLI.html # if the command line is not a .jmx file, then it's another command the user wants to call if [ "${1}" != '' ]; then @@ -25,7 +26,7 @@ if [ "$KEY_NAME" == '' ]; then exit 2 fi if [ "$SECURITY_GROUP" == '' ]; then - echo "Please set a SECURITY_GROUP that allows ports 22, 1099, 50000, 51000 (tcp) from all ports (e.g. sg-12345678)" + echo "Please set a SECURITY_GROUP that allows ports 22,1099,50000,51000/tcp and 4445/udp from all ports (e.g. sg-12345678)" exit 3 fi if [ "$SUBNET_ID" == '' ]; then @@ -37,8 +38,11 @@ fi if [ "$JMETER_VERSION" == '' ]; then JMETER_VERSION=latest fi -if [ "$AWS_DEFAULT_REGION" == '' ]; then - AWS_DEFAULT_REGION=$(aws configure get region) +if [ "$AWS_REGION" == '' ]; then + AWS_REGION=$AWS_DEFAULT_REGION +fi +if [ "$AWS_REGION" == '' ]; then + AWS_REGION=$(aws configure get region) fi if [ "$AWS_ACCESS_KEY_ID" == '' ]; then AWS_ACCESS_KEY_ID=$(aws configure get aws_access_key_id) @@ -49,176 +53,87 @@ fi if [ "$INSTANCE_TYPE" == '' ]; then INSTANCE_TYPE=t2.micro fi +if [ "$MEM_LIMIT" == '' ]; then + MEM_LIMIT=950m +fi if [ "$MINION_COUNT" == '' ]; then MINION_COUNT=2 fi if [ "$PEM_PATH" == '' ]; then PEM_PATH=/keys fi -if [ "$OWNER" == '' ]; then - OWNER=jmeter-ecs -fi if [ "$MINION_CLUSTER_NAME" == '' ]; then - MINION_CLUSTER_NAME=JMeter + MINION_CLUSTER_NAME=JMeterMinion fi if [ "$GRU_CLUSTER_NAME" == '' ]; then GRU_CLUSTER_NAME=JMeterGru fi -if [ "$MINION_TASK_DEFINITION" == '' ]; then - MINION_TASK_DEFINITION=Minion -fi -if [ "$MINION_TAGS" == '' ]; then - MINION_TAGS=ResourceType=instance,Tags=[{Key=Name,Value=Minion},{Key=Owner,Value=$OWNER},{Key=Stack,Value=JMeter}] +if [ "$VPC_ID" == '' ]; then + VPC_ID=$(aws ec2 describe-security-groups --group-ids $SECURITY_GROUP --query 'SecurityGroups[*].[VpcId]' --output text) fi -if [ "$GRU_TAGS" == '' ]; then - GRU_TAGS=ResourceType=instance,Tags=[{Key=Name,Value=Gru},{Key=Owner,Value=$OWNER},{Key=Stack,Value=JMeter}] -fi - -# derive IMAGE_ID from AWS_DEFAULT_REGION -# see: http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-optimized_AMI_launch_latest.html -case "$AWS_DEFAULT_REGION" in - us-east-1) - IMAGE_ID=ami-275ffe31 ;; - us-east-2) - IMAGE_ID=ami-62745007 ;; - us-west-1) - IMAGE_ID=ami-689bc208 ;; - us-west-2) - IMAGE_ID=ami-62d35c02 ;; - eu-west-1) - IMAGE_ID=ami-95f8d2f3 ;; - eu-west-2) - IMAGE_ID=ami-bf9481db ;; - eu-central-1) - IMAGE_ID=ami-085e8a67 ;; - ap-northeast-1) - IMAGE_ID=ami-f63f6f91 ;; - ap-southeast-1) - IMAGE_ID=ami-b4ae1dd7 ;; - ap-southeast-2) - IMAGE_ID=ami-fbe9eb98 ;; - ca-central-1) - IMAGE_ID=ami-ee58e58a ;; - *) - echo "AWS_DEFAULT_REGION must be set to a valid AWS ECS region" - exit 5 -esac -echo "Using image $IMAGE_ID for $AWS_DEFAULT_REGION" - -# Step 1 - Create an ECS Cluster +# Step 1 - Create 2 ECS Clusters +ecs-cli --version echo "Creating cluster/$MINION_CLUSTER_NAME" -aws ecs create-cluster --cluster-name $MINION_CLUSTER_NAME --query 'cluster.[clusterArn]' --output text - -# create a setup script to configure our Cluster name that we pass to --user-data -MINION_CLUSTER_SCRIPT=$'#!/bin/bash\necho ECS_CLUSTER=' -MINION_CLUSTER_SCRIPT="${MINION_CLUSTER_SCRIPT}$MINION_CLUSTER_NAME >> /etc/ecs/ecs.config" -MINION_CLUSTER_BASE64=$(echo "$MINION_CLUSTER_SCRIPT" | base64 | tr -d '\n') - -# Step 2 - Create all instances and register them with the Cluster -echo "Creating $MINION_COUNT Minion instances and register them to cluster/$MINION_CLUSTER_NAME" -MINION_INSTANCE_IDS=$(aws ec2 run-instances --image-id $IMAGE_ID --count $MINION_COUNT --instance-type $INSTANCE_TYPE \ - --iam-instance-profile Name="ecsInstanceRole" --key-name $KEY_NAME \ - --security-group-ids $SECURITY_GROUP --subnet-id $SUBNET_ID --user-data $MINION_CLUSTER_BASE64 \ - --tag-specifications "$MINION_TAGS" \ - --query 'Instances[*].[InstanceId]' --output text | - tr '\n' ' ') -if [ "$MINION_INSTANCE_IDS" == '' ]; then - echo "Creating Minions failed" - echo "Deleting cluster/$MINION_CLUSTER_NAME" - aws ecs delete-cluster --cluster $MINION_CLUSTER_NAME --query 'cluster.[clusterArn]' --output text - exit 6 -fi -echo "Minion instances started: $MINION_INSTANCE_IDS" - -# create a setup script to configure our Gru Cluster name that we pass to --user-data -GRU_CLUSTER_SCRIPT=$'#!/bin/bash\necho ECS_CLUSTER=' -GRU_CLUSTER_SCRIPT="${GRU_CLUSTER_SCRIPT}$GRU_CLUSTER_NAME >> /etc/ecs/ecs.config" -GRU_CLUSTER_BASE64=$(echo "$GRU_CLUSTER_SCRIPT" | base64 | tr -d '\n') - -echo "Creating Gru instance" -GRU_INSTANCE_ID=$(aws ec2 run-instances --image-id $IMAGE_ID --count 1 --instance-type $INSTANCE_TYPE \ - --iam-instance-profile Name="ecsInstanceRole" --key-name $KEY_NAME \ - --security-group-ids $SECURITY_GROUP --subnet-id $SUBNET_ID --user-data $GRU_CLUSTER_BASE64 \ - --tag-specifications "$GRU_TAGS" \ - --query 'Instances[*].[InstanceId]' --output text | - tr '\n' ' ') -if [ "$GRU_INSTANCE_ID" == '' ]; then - echo "Creating Gru failed - terminating Minion instances" - aws ec2 terminate-instances --instance-ids $MINION_INSTANCE_IDS \ - --query 'TerminatingInstances[*].[InstanceId]' --output text - - echo "Waiting for instances to terminate..." - aws ec2 wait instance-terminated --instance-ids $MINION_INSTANCE_IDS --output text - - echo "Deleting cluster/$MINION_CLUSTER_NAME" - aws ecs delete-cluster --cluster $MINION_CLUSTER_NAME --query 'cluster.[clusterArn]' --output text - exit 7 -fi -echo "Gru instance started: $GRU_INSTANCE_ID" - -# Step 3 - Create the Minion ECS task - -# load the requested Docker image using jmeter:JMETER_VERSION with name MINION_TASK_DEFINITION -sed -i 's/JMeterMinion/'"$MINION_TASK_DEFINITION"'/' /opt/jmeter/minion.json -sed -i 's*smithmicro/jmeter:latest*smithmicro/jmeter:'"$JMETER_VERSION"'*' /opt/jmeter/minion.json - -echo "Register Minion task definition" -MINION_TASK_ARN=$(aws ecs register-task-definition --cli-input-json file:///opt/jmeter/minion.json --query 'taskDefinition.taskDefinitionArn' --output text | tr -d '\n') -echo "Minion task registered: $MINION_TASK_ARN" - -# Step 4 - Wait until the instances are running and registered with the Cluster -echo "Waiting for instances to run..." -aws ec2 wait instance-running --instance-ids $MINION_INSTANCE_IDS $GRU_INSTANCE_ID --output text -echo "All instances running - return code $?" - -while true; do - CONTAINER_INSTANCE_COUNT=$(aws ecs list-container-instances --cluster $MINION_CLUSTER_NAME --output text | grep -c container-instance) - if [[ $CONTAINER_INSTANCE_COUNT == $MINION_COUNT ]]; then - echo "Container instances started: $CONTAINER_INSTANCE_COUNT" - break +ecs-cli up --cluster $MINION_CLUSTER_NAME --size $MINION_COUNT --capability-iam --instance-type $INSTANCE_TYPE --keypair $KEY_NAME \ + --security-group $SECURITY_GROUP --vpc $VPC_ID --subnets $SUBNET_ID --force --verbose +echo "Creating cluster/$GRU_CLUSTER_NAME" +ecs-cli up --cluster $GRU_CLUSTER_NAME --capability-iam --instance-type $INSTANCE_TYPE --keypair $KEY_NAME \ + --security-group $SECURITY_GROUP --vpc $VPC_ID --subnets $SUBNET_ID --force --verbose + +# Step 2 - Fetch our Contatiner Instance IDs +while [ "$MINION_CONTAINER_INSTANCE_IDS" = '' ] +do + MINION_CONTAINER_INSTANCE_IDS=$(aws ecs list-container-instances --cluster $MINION_CLUSTER_NAME --output text | + awk '{print $2}' | tr '\n' ' ') + if [ "$MINION_CONTAINER_INSTANCE_IDS" == '' ]; then + echo "Waiting for Minion container instance IDs.." + sleep 5 fi - echo "Waiting 15 seconds for Container instances to register..." - sleep 15 done +echo "Minion container instances IDs: $MINION_CONTAINER_INSTANCE_IDS" +MINION_INSTANCE_IDS=$(aws ecs describe-container-instances --cluster $MINION_CLUSTER_NAME \ + --container-instances $MINION_CONTAINER_INSTANCE_IDS --query 'containerInstances[*].[ec2InstanceId]' --output text) +echo "Minion instances IDs: $MINION_INSTANCE_IDS" -# Step 5 - Fetch our Contatiner Instance IDs -CONTAINER_INSTANCE_IDS=$(aws ecs list-container-instances --cluster $MINION_CLUSTER_NAME --output text | - awk '{print $2}' | tr '\n' ' ') -echo "Container instances IDs: $CONTAINER_INSTANCE_IDS" - -# Step 6 - Run the Minion task with the requested count -echo "Running task: $MINION_TASK_DEFINITION, instance count: $MINION_COUNT" -ECS_MAX=10 -ECS_REMAINING_COUNT=$MINION_COUNT -while [ $ECS_REMAINING_COUNT -gt 0 ] +while [ "$GRU_CONTAINER_INSTANCE_ID" = '' ] do - ECS_REQUEST_COUNT=$(($ECS_REMAINING_COUNT<$ECS_MAX?$ECS_REMAINING_COUNT:$ECS_MAX)) - MINION_TASK_IDS=$(aws ecs run-task --cluster $MINION_CLUSTER_NAME --task-definition $MINION_TASK_DEFINITION --count $ECS_REQUEST_COUNT \ - --query 'tasks[*].[taskArn]' --output text | tr '\n' ' ') - ECS_REMAINING_COUNT=$((ECS_REMAINING_COUNT-ECS_REQUEST_COUNT)) + GRU_CONTAINER_INSTANCE_ID=$(aws ecs list-container-instances --cluster $GRU_CLUSTER_NAME --output text | + awk '{print $2}' | tr '\n' ' ') + if [ "$GRU_CONTAINER_INSTANCE_ID" == '' ]; then + echo "Waiting for Gru container instance ID..." + sleep 5 + fi done -echo "Waiting for tasks to run: $MINION_TASK_IDS" -aws ecs wait tasks-running --cluster $MINION_CLUSTER_NAME --tasks $MINION_TASK_IDS -if [ "$?" = '0' ]; then - echo "Minion tasks running" +echo "Gru container instances ID: $GRU_CONTAINER_INSTANCE_ID" +GRU_INSTANCE_ID=$(aws ecs describe-container-instances --cluster $GRU_CLUSTER_NAME \ + --container-instances $GRU_CONTAINER_INSTANCE_ID --query 'containerInstances[*].[ec2InstanceId]' --output text) +echo "Gru instances ID: $GRU_INSTANCE_ID" + +# Step 3 - Run the Minion task with the requested JMeter version, instance count and memory +sed -i 's/jmeter:latest/jmeter:'"$JMETER_VERSION"'/' /opt/jmeter/lucy.yml +sed -i 's/950m/'"$MEM_LIMIT"'/' /opt/jmeter/lucy.yml +ecs-cli compose --file /opt/jmeter/lucy.yml up --cluster $MINION_CLUSTER_NAME +ecs-cli compose --file /opt/jmeter/lucy.yml --cluster $MINION_CLUSTER_NAME scale $MINION_COUNT + +# Step 4 - Get IP addresses from Gru (Public or Private) and Minions (always Private) +if [ "$GRU_PRIVATE_IP" = '' ]; then + GRU_HOST=$(aws ec2 describe-instances --instance-ids $GRU_INSTANCE_ID \ + --query 'Reservations[*].Instances[*].[PublicIpAddress]' --output text | tr -d '\n') else - echo "Minion tasks failed to run - return code $?" + cho "Using Gru's Private IP" + GRU_HOST=$(aws ec2 describe-instances --instance-ids $GRU_INSTANCE_ID \ + --query 'Reservations[*].Instances[*].[PrivateIpAddress]' --output text | tr -d '\n') fi - -# Step 7 - Get public IP addresses from Gru and Minions -GRU_HOST=$(aws ec2 describe-instances --instance-ids $GRU_INSTANCE_ID \ - --query 'Reservations[*].Instances[*].[PublicIpAddress]' --output text | tr -d '\n') echo "Gru at $GRU_HOST" MINION_HOSTS=$(aws ec2 describe-instances --instance-ids $MINION_INSTANCE_IDS \ - --query 'Reservations[*].Instances[*].[PublicIpAddress]' --output text | tr '\n' ',') -echo "Minions at at $MINION_HOSTS" - + --query 'Reservations[*].Instances[*].[PrivateIpAddress]' --output text | tr '\n' ',') +echo "Minions at $MINION_HOSTS" # uncomment if you want to pause Lucy to inspect Gru or a Minion #read -p "Press enter to start Gru setup: " -# Step 8 - Run Gru with the specified JMX +# Step 5 - Run Gru with the specified JMX echo "Copying $INPUT_JMX to Gru" scp -i $PEM_PATH/$KEY_NAME.pem -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no $INPUT_JMX ec2-user@${GRU_HOST}:/tmp @@ -227,30 +142,14 @@ JMX_IN_COMTAINER=/plans/$(basename $INPUT_JMX) ssh -i $PEM_PATH/$KEY_NAME.pem -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no ec2-user@${GRU_HOST} \ "docker run -p 1099:1099 -p 51000:51000 -v /tmp:/plans -v /logs:/logs --env MINION_HOSTS=$MINION_HOSTS smithmicro/jmeter:$JMETER_VERSION $JMX_IN_COMTAINER" +# Step 6 - Fetch the results echo "Copying results from Gru" scp -r -i $PEM_PATH/$KEY_NAME.pem -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no ec2-user@${GRU_HOST}:/logs/* /logs -# Step 9 - Stop all tesks -echo "Stopping tasks" -aws ecs list-tasks --cluster $MINION_CLUSTER_NAME --output text | - awk '{print $2}' | - while read line; do - aws ecs stop-task --cluster $MINION_CLUSTER_NAME --task $line --query 'task.[taskArn]' --output text; - done - -# Step 10 - Terminate all instances -echo "Terminating instances: $MINION_INSTANCE_IDS $GRU_INSTANCE_ID" -aws ec2 terminate-instances --instance-ids $MINION_INSTANCE_IDS $GRU_INSTANCE_ID \ - --query 'TerminatingInstances[*].[InstanceId]' --output text - -echo "Waiting for instances to terminate..." -aws ec2 wait instance-terminated --instance-ids $MINION_INSTANCE_IDS $GRU_INSTANCE_ID --output text - -# Step 11 - Final cleanup -echo "Deregister task $MINION_TASK_ARN" -aws ecs deregister-task-definition --task-definition $MINION_TASK_ARN --query 'taskDefinition.[taskDefinitionArn]' --output text - +# Step 7 - Delete the clusters echo "Deleting cluster/$MINION_CLUSTER_NAME" -aws ecs delete-cluster --cluster $MINION_CLUSTER_NAME --query 'cluster.[clusterArn]' --output text +ecs-cli down --cluster $MINION_CLUSTER_NAME --force +echo "Deleting cluster/$GRU_CLUSTER_NAME" +ecs-cli down --cluster $GRU_CLUSTER_NAME --force echo "Complete" diff --git a/lucy/lucy.yml b/lucy/lucy.yml new file mode 100644 index 0000000..60f0dcc --- /dev/null +++ b/lucy/lucy.yml @@ -0,0 +1,11 @@ +version: '2' + +services: + minion: + image: smithmicro/jmeter:latest + mem_limit: 950m + ports: + - 1099:1099 + - 50000:50000 + volumes: + - /logs:/logs diff --git a/lucy/minion.json b/lucy/minion.json deleted file mode 100644 index 00556c9..0000000 --- a/lucy/minion.json +++ /dev/null @@ -1,38 +0,0 @@ -{ - "family": "JMeterMinion", - "networkMode": "bridge", - "containerDefinitions": [ - { - "portMappings": [ - { - "hostPort": 1099, - "containerPort": 1099, - "protocol": "tcp" - }, - { - "hostPort": 50000, - "containerPort": 50000, - "protocol": "tcp" - } - ], - "mountPoints": [ - { - "containerPath": "/logs", - "sourceVolume": "logs", - "readOnly": false - } - ], - "name": "minion", - "image": "smithmicro/jmeter:latest", - "memoryReservation": 950 - } - ], - "volumes": [ - { - "host": { - "sourcePath": "/logs" - }, - "name": "logs" - } - ] -} \ No newline at end of file