|
| 1 | +import Tabs from '@theme/Tabs'; |
| 2 | +import TabItem from '@theme/TabItem'; |
| 3 | + |
| 4 | +# Extending your ECS Deploy Runner |
| 5 | + |
| 6 | +Pipelines can be extended in several ways: |
| 7 | +- Adding repositories to supporting building Docker images for many applications |
| 8 | +- Updating which branches can kick off which jobs |
| 9 | +- Adding additional build scripts that can run in Pipelines |
| 10 | +- Adding permissions to Pipelines |
| 11 | + |
| 12 | + |
| 13 | +## Adding a repository |
| 14 | + |
| 15 | +Pipelines has separate configurations for each type of job that can be performed (e.g., building a docker image, running terraform plan, running terraform apply). An allow-list of repos and branches is defined for each job type, which can be updated to extend your usage of pipelines to additional application repositories. |
| 16 | + |
| 17 | +This portion of the guide focuses on building Docker images for application repos. If you have repositories for which you would like to run `terraform plan` or `terraform apply` jobs, similar steps can be followed, modifying the appropriate task configurations. |
| 18 | + |
| 19 | +<Tabs groupId="deployment-type"> |
| 20 | +<TabItem value="RefArch" label="RefArch" default> |
| 21 | + |
| 22 | +If you’ve deployed Pipelines as a part of your Reference Architecture, we recommend following the guide on [how to deploy your apps into the Reference Architecture](../../refarch/usage/maintain-your-refarch//deploying-your-apps.md) to learn how to define a module for your application. |
| 23 | + |
| 24 | +To allow Pipelines jobs to be started by events in your repository, open `shared/<your region>/mgmt/ecs-deploy-runner/terragrunt.hcl` and update `docker_image_builder_config.allowed_repos` to include the HTTPS Git URL of the application repo for which you would like to deploy Docker images. |
| 25 | + |
| 26 | +Since pipelines [cannot update itself](./updating.md), you must run `terragrunt plan` and `terragrunt apply` manually to deploy the change from your local machine. Run `terragrunt plan` to inspect the changes that will be made to your pipeline. Once the changes have been reviewed, run `terragrunt apply` to deploy the changes. |
| 27 | + |
| 28 | +</TabItem> |
| 29 | +<TabItem value="Standalone" label="Standalone"> |
| 30 | + |
| 31 | +If you’ve deployed Pipelines as a standalone framework using the `ecs-deploy-runner` service in the Service Catalog, you will need to locate the file in which you’ve defined a module block sourcing the `ecs-deploy-runner` service. |
| 32 | + |
| 33 | +Once the `ecs-deploy-runner` module block is located, update the `allowed_repos` list in the `docker_image_builder_config` variable to include the HTTPS Git URL of the application repo for which you would like to deploy Docker images. |
| 34 | + |
| 35 | +Refer to the [Variable Reference](../../reference/services/ci-cd-pipeline/ecs-deploy-runner#reference) section for the service in the Library Reference for full configuration details. |
| 36 | + |
| 37 | +Run `terraform plan` to inspect the changes that will be made to your pipeline. Once the changes have been reviewed, run `terraform apply` to deploy the changes. To deploy the application to ECS or EKS you will need to deploy a task definition (ECS) or Deployment (EKS) that references the newly built image. |
| 38 | +</TabItem> |
| 39 | +</Tabs> |
| 40 | + |
| 41 | +### Adding infrastructure deployer to the new repo |
| 42 | + |
| 43 | +Pipelines can be triggered from GitHub events in many repositories. In order to configure Pipelines for the new repository, you need to add a step in your CI/CD configuration for the repository that uses the `infrastructure-deployer` CLI tool to trigger Docker image builds. |
| 44 | + |
| 45 | +```bash |
| 46 | +export ACCOUNT_ID=$(aws sts get-caller-identity --query 'Account' --output text) |
| 47 | +export DEPLOY_RUNNER_REGION=$(aws configure get region) |
| 48 | +export ECR_REPO_URL="${ACCOUNT_ID}.dkr.ecr.${DEPLOY_RUNNER_REGION}.amazonaws.com" |
| 49 | +export DOCKER_TAG=$(git rev-parse --short HEAD) |
| 50 | +export REPOSITORY_NAME="example" |
| 51 | +export GITHUB_ORG="example-org" |
| 52 | + |
| 53 | +infrastructure-deployer --aws-region "us-east-1" -- docker-image-builder build-docker-image \ |
| 54 | + --repo "https://github.com/${GITHUB_ORG}/${REPOSITORY_NAME}" \ |
| 55 | + --ref "origin/main" \ |
| 56 | + --context-path "path/to/directory/with/dockerfile/" \ |
| 57 | + --docker-image-tag "${ECR_REPO_URL}/${REPOSITORY_NAME}:${DOCKER_TAG}" \ |
| 58 | +``` |
| 59 | + |
| 60 | +## Specifying branches that can be deployed |
| 61 | + |
| 62 | +Pipelines can be configured to only allow jobs to be performed on specific branches. For example, a common configuration is to allow `terraform plan` or `terragrunt plan` jobs for pull requests, and only allow `terraform apply` or `terragrunt apply` to run on merges to the main branch. |
| 63 | + |
| 64 | +Depending on your use case, you may need to modify the `allowed_apply_git_refs` attribute to update the allow-list of branch names that can kick off the `plan` and `apply` jobs. |
| 65 | + |
| 66 | +For example, a common configuration for `apply` jobs is to specify that this job can only run on the `main` branch: |
| 67 | +```tf |
| 68 | +allowed_apply_git_refs = ["main", "origin/main"] |
| 69 | +``` |
| 70 | + |
| 71 | +<Tabs groupId="deployment-type"> |
| 72 | +<TabItem value="RefArch" label="RefArch" default> |
| 73 | + |
| 74 | +If you’ve deployed Pipelines as a part of your Reference Architecture, open `shared/<your region>/mgmt/ecs-deploy-runner/terragrunt.hcl` and update the values in the `allowed_apply_git_refs` attribute for the job configuration you would like to modify (either `terraform_planner_config` or `terraform_applier_config`). |
| 75 | + |
| 76 | +Run `terragrunt plan` to inspect the changes that will be made to your pipeline. Once the changes have been reviewed, run `terragrunt apply` to deploy the changes. |
| 77 | + |
| 78 | +</TabItem> |
| 79 | +<TabItem value="Standalone" label="Standalone"> |
| 80 | + |
| 81 | +If you’ve deployed Pipelines as a standalone framework using the `ecs-deploy-runner` service in the Service Catalog, you will need to locate the file in which you’ve defined a module block sourcing the `ecs-deploy-runner` service. |
| 82 | + |
| 83 | +By default, the `ecs-deploy-runner` service from the Service Catalog allows any git ref to be applied. After you locate the module block for `ecs-deploy-runner`, modify the `allowed_apply_git_refs` attribute for the job configuration that you would like to modify (either `terraform_planner_config` or `terraform_applier_config`). |
| 84 | + |
| 85 | +Run `terraform plan` to inspect the changes that will be made to your pipeline. Once the changes have been reviewed, run `terraform apply` to deploy the changes. |
| 86 | +</TabItem> |
| 87 | +</Tabs> |
| 88 | + |
| 89 | +## Adding a new AWS Service |
| 90 | + |
| 91 | +If you are expanding your usage of AWS to include an AWS service you’ve never used before, you will need to grant each job sufficient permissions to access that service. Pipelines executes in ECS tasks running in your AWS account(s). Each task (terraform planner, applier, docker builder, ami builder) has a distinct execution IAM role with only the permissions each task requires to complete successfully. For example, if you need to create an Amazon DynamoDB Table using Pipelines for the first time, you would want to add (at a minimum) the ability to list and describe tables to the policy for the `planner` IAM role, and all permissions for DynamoDB to the IAM policy for the `terraform-applier` IAM role. |
| 92 | + |
| 93 | +We recommend that the `planner` configuration have read-only access to resources, and the applier be able to read, create, modify, and destroy resources. |
| 94 | + |
| 95 | +<Tabs groupId="deployment-type"> |
| 96 | +<TabItem value="RefArch" label="RefArch" default> |
| 97 | + |
| 98 | +If you’ve deployed Pipelines as a part of your Reference Architecture, the permissions for the `terraform-planner` task are located in `_envcommon/mgmt/read_only_permissions.yml` and the permissions for the `terraform-applier` task are located in `_envcommon/mgmt/deploy_permissions.yml`. Open and add the required permissions to each file. |
| 99 | + |
| 100 | +After you are done updating both files, you will need to run `terragrunt plan`, review the changes, then `terragrunt apply` for each account in your Reference Architecture. |
| 101 | +```bash |
| 102 | +cd logs/$DEPLOY_RUNNER_REGION/mgmt/ecs-deploy-runner |
| 103 | +aws-vault exec <your-logs> -- terragrunt apply --terragrunt-source-update -auto-approve |
| 104 | + |
| 105 | +cd shared/$DEPLOY_RUNNER_REGION/mgmt/ecs-deploy-runner |
| 106 | +aws-vault exec <your-shared> -- terragrunt apply --terragrunt-source-update -auto-approve |
| 107 | + |
| 108 | +cd security/$DEPLOY_RUNNER_REGION/mgmt/ecs-deploy-runner |
| 109 | +aws-vault exec <your-security> -- terragrunt apply --terragrunt-source-update -auto-approve |
| 110 | + |
| 111 | +cd dev/$DEPLOY_RUNNER_REGION/mgmt/ecs-deploy-runner |
| 112 | +aws-vault exec <your-dev> -- terragrunt apply --terragrunt-source-update -auto-approve |
| 113 | + |
| 114 | +cd stage/$DEPLOY_RUNNER_REGION/mgmt/ecs-deploy-runner |
| 115 | +aws-vault exec <your-stage> -- terragrunt apply --terragrunt-source-update -auto-approve |
| 116 | + |
| 117 | +cd prod/$DEPLOY_RUNNER_REGION/mgmt/ecs-deploy-runner |
| 118 | +aws-vault exec <your-prod> -- terragrunt apply --terragrunt-source-update -auto-approve |
| 119 | +``` |
| 120 | +</TabItem> |
| 121 | +<TabItem value="Standalone" label="Standalone"> |
| 122 | + |
| 123 | +If you’ve deployed Pipelines as a standalone framework using the `ecs-deploy-runner` service in the Service Catalog, you will need to locate the file in which you’ve defined a module block sourcing the `ecs-deploy-runner` service. |
| 124 | + |
| 125 | +Modify the AWS IAM policy document being passed into the `iam_policy` variable for the [`terraform_applier_config`](../../reference/services/ci-cd-pipeline/ecs-deploy-runner#terraform_applier_config) and the [`terraform_planner_config`](../../reference/services/ci-cd-pipeline/ecs-deploy-runner#terraform_planner_config) variables. Refer to the [variable reference](../../reference/services/ci-cd-pipeline/ecs-deploy-runner#reference) section for the service in the Library Reference for the full set of configuration details for this service. |
| 126 | + |
| 127 | +After you are done updating the IAM policy documents, run `terraform plan` then review the changes that will be made. Finally, run `terraform apply` to apply the changes. |
| 128 | +</TabItem> |
| 129 | +</Tabs> |
| 130 | + |
| 131 | +## Adding scripts that can be run in Pipelines |
| 132 | + |
| 133 | +The `deploy-runner` Docker image for Pipelines only allows scripts within a single directory to be executed in the ECS task as an additional security measure. |
| 134 | + |
| 135 | +By default, the `deploy-runner` ships with three scripts — one to build HashiCorp Packer images, one to run `terraform plan` and `terraform apply`, and one to automatically update the value of a variable in a Terraform tfvars or Terragrunt HCL file. |
| 136 | + |
| 137 | +If you need to run a custom script in the `deploy-runner`, you must fork the image code, add an additional line to copy your script into directory designated by the `trigger_directory` argument. Then, you will need to rebuild the Docker image, push to ECR, then update your Pipelines deployment following the steps in [Updating your Pipeline](./updating.md). |
0 commit comments