Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .github/workflows/ci-scripts.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,3 +40,15 @@ jobs:

- name: Validate multi-org command generation
run: python scripts/generate_import_commands.py --example multi-org --repo-root . > /tmp/imports-multi.sh

- name: Validate single-org OIDC provider command generation
run: python scripts/generate_import_commands.py --example single-org-oidc-provider --repo-root . > /tmp/imports-single-oidc-provider.sh

- name: Validate multi-org OIDC provider command generation
run: python scripts/generate_import_commands.py --example multi-org-oidc-provider --repo-root . > /tmp/imports-multi-oidc-provider.sh

- name: Validate single-org OIDC roles command generation
run: python scripts/generate_import_commands.py --example single-org-oidc-roles --repo-root . > /tmp/imports-single-oidc-roles.sh

- name: Validate multi-org OIDC roles command generation
run: python scripts/generate_import_commands.py --example multi-org-oidc-roles --repo-root . > /tmp/imports-multi-oidc-roles.sh
14 changes: 10 additions & 4 deletions .github/workflows/ci-terraform.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,12 +38,18 @@ jobs:
fail-fast: false
matrix:
stack:
- modules/aws_github_oidc_provider
- modules/aws_github_actions_oidc_roles
- modules/github_org_settings
- modules/github_repo_platform
- examples/s3-backend-single-org
- examples/s3-backend-single-org-repos
- examples/s3-backend-multi-org
- examples/s3-backend-multi-org-repos
- examples/github_org_settings/single-org
- examples/github_repo_platform/single-org
- examples/aws_github_oidc_provider/single-account
- examples/aws_github_actions_oidc_roles/single-account
- examples/github_org_settings/multi-org
- examples/github_repo_platform/multi-org
- examples/aws_github_oidc_provider/multi-account
- examples/aws_github_actions_oidc_roles/multi-account
steps:
- name: Checkout
uses: actions/checkout@v4
Expand Down
14 changes: 10 additions & 4 deletions .github/workflows/ci-tflint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,18 @@ jobs:
fail-fast: false
matrix:
stack:
- modules/aws_github_oidc_provider
- modules/aws_github_actions_oidc_roles
- modules/github_org_settings
- modules/github_repo_platform
- examples/s3-backend-single-org
- examples/s3-backend-single-org-repos
- examples/s3-backend-multi-org
- examples/s3-backend-multi-org-repos
- examples/github_org_settings/single-org
- examples/github_repo_platform/single-org
- examples/aws_github_oidc_provider/single-account
- examples/aws_github_actions_oidc_roles/single-account
- examples/github_org_settings/multi-org
- examples/github_repo_platform/multi-org
- examples/aws_github_oidc_provider/multi-account
- examples/aws_github_actions_oidc_roles/multi-account
steps:
- name: Checkout
uses: actions/checkout@v4
Expand Down
18 changes: 18 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
repos:
- repo: local
hooks:
- id: terraform-fmt-check
name: terraform fmt -check
entry: scripts/precommit_terraform.sh fmt-check
language: system
pass_filenames: false
- id: terraform-validate
name: terraform validate
entry: scripts/precommit_terraform.sh validate
language: system
pass_filenames: false
- id: terraform-tflint
name: tflint
entry: scripts/precommit_terraform.sh tflint
language: system
pass_filenames: false
64 changes: 52 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
# Platform SCM Terraform Catalog

Terraform modules and example stacks for managing GitHub organizations, repositories, environments, and environment-level secrets/variables with a catalog-driven workflow.
Terraform modules and example stacks for managing GitHub organizations, repositories, environments, GitHub Actions OIDC trust in AWS, and AWS IAM role-based access for GitHub Actions with a catalog-driven workflow.

## What This Repository Manages

- Organization settings through `modules/github_org_settings`.
- Repository lifecycle through `modules/github_repo_platform`.
- Repository environments, environment secrets, and environment variables through `modules/github_repo_platform`.
- AWS IAM OIDC providers through `modules/aws_github_oidc_provider`.
- AWS IAM roles and policies for GitHub Actions through `modules/aws_github_actions_oidc_roles`.
- Catalog-based onboarding, where YAML files define repository and environment intent.

## Resource Coverage
Expand All @@ -16,6 +18,8 @@ Terraform modules and example stacks for managing GitHub organizations, reposito
- Repository environments: `github_repository_environment`.
- Environment secrets: `github_actions_environment_secret`.
- Environment variables: `github_actions_environment_variable`.
- AWS OIDC providers: `aws_iam_openid_connect_provider`.
- AWS IAM role-based access: `aws_iam_role`, `aws_iam_policy`, and `aws_iam_role_policy_attachment`.

Environment configuration model:
- `repositories.<repo>.shared_environment_secrets` and `repositories.<repo>.shared_environment_variables` apply to all environments in a repository.
Expand All @@ -25,6 +29,7 @@ Environment configuration model:
Catalog loading model:
- Example stacks load YAML definitions using `fileset(...)` + `yamldecode(...)`.
- Repositories are onboarded by adding or updating catalog files instead of editing module wiring.
- OIDC providers, repository IAM roles, and repository short names are onboarded the same way.

## Quick Start

Expand All @@ -36,17 +41,17 @@ Prerequisites:
1. Enter an example stack:

```bash
cd examples/s3-backend-single-org-repos
cd examples/github_repo_platform/single-org
# or
cd examples/s3-backend-multi-org-repos
cd examples/github_repo_platform/multi-org
```

Org settings stacks:

```bash
cd examples/s3-backend-single-org
cd examples/github_org_settings/single-org
# or
cd examples/s3-backend-multi-org
cd examples/github_org_settings/multi-org
```

2. Copy example inputs and backend config:
Expand All @@ -65,24 +70,59 @@ terraform validate
terraform plan
```

## Local Pre-Commit Checks

Install and run Terraform checks locally before pushing:

```bash
pip install pre-commit
pre-commit install
pre-commit run --all-files
```

Configured hooks:
- `terraform fmt -check`
- `terraform validate` (across modules and example stacks used in CI)
- `tflint` (across modules and example stacks used in CI)

## Examples

- `examples/s3-backend-single-org-repos`: single-organization repository onboarding.
- `examples/s3-backend-multi-org-repos`: multi-organization repository onboarding with provider aliases.
- `examples/s3-backend-single-org`: single-organization settings management.
- `examples/s3-backend-multi-org`: multi-organization settings management.
- Directory index: `examples/README.md`
- `examples/github_repo_platform/single-org`: single-organization repository onboarding.
- `examples/github_repo_platform/multi-org`: multi-organization repository onboarding with provider aliases.
- `examples/github_org_settings/single-org`: single-organization settings management.
- `examples/github_org_settings/multi-org`: multi-organization settings management.
- `examples/aws_github_oidc_provider/single-account`: single-account GitHub OIDC provider management.
- `examples/aws_github_oidc_provider/multi-account`: multi-account GitHub OIDC provider management.
- `examples/aws_github_actions_oidc_roles/single-account`: single-account repository/environment IAM role management.
- `examples/aws_github_actions_oidc_roles/multi-account`: multi-account repository/environment IAM role management.

## Catalog-Driven Onboarding Workflow

- Single-org catalog path: `examples/s3-backend-single-org-repos/catalog/repositories/`.
- Single-org catalog path: `examples/github_repo_platform/single-org/catalog/repositories/`.
- Multi-org catalog paths:
- `examples/s3-backend-multi-org-repos/catalog/orgs/platform/repositories/`
- `examples/s3-backend-multi-org-repos/catalog/orgs/shared/repositories/`
- `examples/github_repo_platform/multi-org/catalog/orgs/platform/repositories/`
- `examples/github_repo_platform/multi-org/catalog/orgs/shared/repositories/`
- Onboard by adding/updating catalog YAML, then run Terraform plan/apply in the matching stack.

AWS OIDC/IAM catalog paths:
- OIDC providers:
- `examples/aws_github_oidc_provider/single-account/catalog/oidc-connections/`
- `examples/aws_github_oidc_provider/multi-account/catalog/orgs/platform/oidc-connections/`
- `examples/aws_github_oidc_provider/multi-account/catalog/orgs/shared/oidc-connections/`
- OIDC roles:
- `examples/aws_github_actions_oidc_roles/single-account/catalog/repositories/`
- `examples/aws_github_actions_oidc_roles/multi-account/catalog/orgs/platform/repositories/`
- `examples/aws_github_actions_oidc_roles/multi-account/catalog/orgs/shared/repositories/`
- Repository short names:
- `examples/aws_github_actions_oidc_roles/single-account/catalog/repo-short-names.yaml`
- `examples/aws_github_actions_oidc_roles/multi-account/catalog/orgs/platform/repo-short-names.yaml`
- `examples/aws_github_actions_oidc_roles/multi-account/catalog/orgs/shared/repo-short-names.yaml`

## Operations And Migration Playbooks

- Import existing resources into Terraform state: `docs/playbooks/import-existing-github-resources.md`.
- Import existing AWS OIDC/IAM resources into Terraform state: `docs/playbooks/import-existing-aws-oidc-resources.md`.
- GitHub Actions variable/secret setup reference:
`docs/playbooks/import-existing-github-resources.md#github-actions-configuration-reference`.

Expand Down
43 changes: 43 additions & 0 deletions docs/playbooks/import-existing-aws-oidc-resources.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Playbook: Import Existing AWS OIDC/IAM Resources Into Terraform State

This playbook is for adopting existing AWS IAM OIDC providers, IAM roles, and IAM policies used by GitHub Actions into this repo's Terraform state without recreating them.

## Scope

This repository manages these AWS resources via split modules:

- `modules/aws_github_oidc_provider` manages `aws_iam_openid_connect_provider`
- `modules/aws_github_actions_oidc_roles` manages `aws_iam_role`
- `modules/aws_github_actions_oidc_roles` manages `aws_iam_policy`
- `modules/aws_github_actions_oidc_roles` manages `aws_iam_role_policy_attachment`

## Stacks

- OIDC providers:
- `examples/aws_github_oidc_provider/single-account`
- `examples/aws_github_oidc_provider/multi-account`
- OIDC roles/policies:
- `examples/aws_github_actions_oidc_roles/single-account`
- `examples/aws_github_actions_oidc_roles/multi-account`

## Generate Import Command Stubs

```bash
# From repo root
python3 scripts/generate_import_commands.py --example single-org-oidc-provider
python3 scripts/generate_import_commands.py --example multi-org-oidc-provider
python3 scripts/generate_import_commands.py --example single-org-oidc-roles
python3 scripts/generate_import_commands.py --example multi-org-oidc-roles
```

Review output before running commands.

## Notes

- Import IDs for OIDC providers and customer-managed policies use ARNs.
- Import IDs for role-policy attachments follow `role-name/policy-arn`.
- If a resource imports to the wrong address, remove it from state and re-import:

```bash
terraform state rm <address>
```
28 changes: 14 additions & 14 deletions docs/playbooks/import-existing-github-resources.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,11 @@ This repository manages these GitHub resources via split modules:
- Terraform CLI is installed locally.
- Your selected stack is configured:
- Repository onboarding stack:
- `examples/s3-backend-single-org-repos`, or
- `examples/s3-backend-multi-org-repos`
- `examples/github_repo_platform/single-org`, or
- `examples/github_repo_platform/multi-org`
- Org settings stack (if importing org settings):
- `examples/s3-backend-single-org`, or
- `examples/s3-backend-multi-org`
- `examples/github_org_settings/single-org`, or
- `examples/github_org_settings/multi-org`
- Backend config exists (`backend.hcl`) and points to the target state.
- GitHub token(s) have sufficient scopes to read/update org/repo/environment settings.
- Catalog YAML files match the existing GitHub resources you want Terraform to manage.
Expand Down Expand Up @@ -63,31 +63,31 @@ State isolation:
Single-org:

```bash
cd examples/s3-backend-single-org-repos
cd examples/github_repo_platform/single-org
cp backend.hcl.example backend.hcl # if needed
terraform init -backend-config=backend.hcl
```

Multi-org:

```bash
cd examples/s3-backend-multi-org-repos
cd examples/github_repo_platform/multi-org
cp backend.hcl.example backend.hcl # if needed
terraform init -backend-config=backend.hcl
```

Single-org org settings:

```bash
cd examples/s3-backend-single-org
cd examples/github_org_settings/single-org
cp backend.hcl.example backend.hcl # if needed
terraform init -backend-config=backend.hcl
```

Multi-org org settings:

```bash
cd examples/s3-backend-multi-org
cd examples/github_org_settings/multi-org
cp backend.hcl.example backend.hcl # if needed
terraform init -backend-config=backend.hcl
```
Expand All @@ -110,12 +110,12 @@ Address patterns used by the repository module:
Module names:

- Single-org example:
- Settings: `module.org_settings` (in `examples/s3-backend-single-org`)
- Settings: `module.org_settings` (in `examples/github_org_settings/single-org`)
- Repositories: `module.org_repositories`
- Multi-org example:
- Platform settings: `module.platform_org_settings` (in `examples/s3-backend-multi-org`)
- Platform settings: `module.platform_org_settings` (in `examples/github_org_settings/multi-org`)
- Platform repositories: `module.platform_org_repositories`
- Shared settings: `module.shared_org_settings` (in `examples/s3-backend-multi-org`)
- Shared settings: `module.shared_org_settings` (in `examples/github_org_settings/multi-org`)
- Shared repositories: `module.shared_org_repositories`

## 3. Import Order
Expand Down Expand Up @@ -163,7 +163,7 @@ Replace placeholders (org/repo/env/names).

```bash
# 4.1 Optional org settings
# Run this from examples/s3-backend-single-org
# Run this from examples/github_org_settings/single-org
terraform import 'module.org_settings.github_organization_settings.this[0]' '<org_name>'

# 4.2 Repository
Expand Down Expand Up @@ -192,7 +192,7 @@ terraform import 'module.org_repositories.github_actions_environment_secret.this

```bash
# Optional org settings
# Run this from examples/s3-backend-multi-org
# Run this from examples/github_org_settings/multi-org
terraform import 'module.platform_org_settings.github_organization_settings.this[0]' '<platform_org_name>'

# Repository
Expand All @@ -214,7 +214,7 @@ terraform import 'module.platform_org_repositories.github_actions_environment_se

```bash
# Optional org settings (only if enabled in config)
# Run this from examples/s3-backend-multi-org
# Run this from examples/github_org_settings/multi-org
terraform import 'module.shared_org_settings.github_organization_settings.this[0]' '<shared_org_name>'

# Repository
Expand Down
24 changes: 24 additions & 0 deletions examples/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Examples

Examples are grouped by module to make ownership and usage intent explicit.

## Layout

- `github_org_settings/`
- `single-org/`
- `multi-org/`
- `github_repo_platform/`
- `single-org/`
- `multi-org/`
- `aws_github_oidc_provider/`
- `single-account/`
- `multi-account/`
- `aws_github_actions_oidc_roles/`
- `single-account/`
- `multi-account/`

Each stack includes:
- `main.tf`, `providers.tf`, `variables.tf`, `versions.tf`
- `terraform.tfvars.example`
- `backend.hcl.example`
- `catalog/` YAML files for catalog-driven onboarding
22 changes: 22 additions & 0 deletions examples/aws_github_actions_oidc_roles/multi-account/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# S3 Backend Multi-Org OIDC Roles Example

Reference stack for managing repository and environment IAM roles/policies across two AWS accounts.

## Catalog Location

- Platform repo IAM catalog: `catalog/orgs/platform/repositories/*.yaml`
- Shared repo IAM catalog: `catalog/orgs/shared/repositories/*.yaml`
- Platform short names: `catalog/orgs/platform/repo-short-names.yaml`
- Shared short names: `catalog/orgs/shared/repo-short-names.yaml`
- Platform OIDC provider ARNs: `catalog/orgs/platform/oidc-provider-arns.yaml`
- Shared OIDC provider ARNs: `catalog/orgs/shared/oidc-provider-arns.yaml`

## Local Run

```bash
cp terraform.tfvars.example terraform.tfvars
cp backend.hcl.example backend.hcl
terraform init -backend-config=backend.hcl
terraform validate
terraform plan
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
bucket = "my-platform-tf-state"
key = "github-enterprise/aws-oidc-roles/multi-org/terraform.tfstate"
region = "us-east-1"
dynamodb_table = "my-platform-tf-locks"
encrypt = true
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
https://token.actions.githubusercontent.com: arn:aws:iam::111122223333:oidc-provider/token.actions.githubusercontent.com
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
github-com/platform/platform-api: plat-api
Loading