⚠️ ARCHIVED REPOSITORYThis repository is archived and no longer actively maintained. While the code remains available for reference, we recommend considering GitHub Action runners with Azure Private VNET as an excellent alternative for connecting runners to your private infrastructure. This approach provides secure connectivity between GitHub-hosted runners and your private Azure resources without the overhead of managing self-hosted runner infrastructure.
For more information, see the GitHub documentation on using GitHub-hosted runners in Azure private networks.
This project includes all necessary components to spin up the infrastructure for VM based GitHub self-hosted runners in Azure. This project was created with some inspiration from the Philips Lab AWS Solution with some opinionated changes on what our team at Liatrio has seen work well across different enterprises.
- Ephemeral Only
- Runners should only run one job to avoid interference from one workflow run to the next
- Warm Pool by Default
- Keeping idle runners on is a must to ensure quick feedback loops
- Custom Images
- Images should be able to build most apps in organization without additional tool installation (example)
- Including necessary tools in VM Image to reduce startup time for most builds
- Security
- Runner VMs are granted a single use registration token with no additional access to GitHub
- All components utilize Managed Identities for access to other resources, and are granted the least access required to function.
- GitHub Actions event ➜ Event-Handler – A
workflow_job
event is emitted by GitHub Enterprise/Cloud and forwarded by a GitHub App to the Event-Handler Azure Function. - Event-Handler ➜ Service Bus – The function validates the request, filters on runner labels, and publishes a message to an Azure Service Bus queue.
- Service Bus ➜ Runner-Controller – The Runner-Controller Azure App Service continuously listens to the queue. Based on settings stored in Azure App Configuration it decides whether to spin up a new runner or rely on the warm-pool.
- Runner-Controller ➜ Azure VM – When a new runner is required the controller:
• Fetches the latest Packer-built image from an Azure Shared Image Gallery.
• Retrieves a one-time registration token from GitHub, storing it in the Registration Key Vault.
• Creates a spot/on-demand VM, injecting the token via cloud-init so the VM registers itself with GitHub on first boot. - Runner lifecycle – The VM processes exactly one job, then a shutdown hook notifies the controller which de-registers the runner and deletes the VM, keeping the warm-pool at the desired size.
- Secrets & config – GitHub App credentials and webhook secret live in the App Key Vault; operational parameters are stored in Azure App Configuration. All resources use Managed Identities for least-privilege access.
- Provisioning – This entire architecture is provisioned reproducibly by the Terraform module contained in this repository.
This Terraform module generates the infrastructure required to host the applications that will manage the self-hosted runners.
The event-handler will receive messages from the GitHub App during workflow run events. It will act as a filter to ensure they are from GitHub with labels that match what is provided in the module.
This application will act as the controller for the warm pool and ensure that the pool size adheres to the parameters specified in the Terraform module. It consumes Service Bus messages to create or delete VMs so that a healthy number of runners are always ready to process workflow jobs.
A reliable message queue that buffers workflow_job
events between the Event-Handler and Runner-Controller, ensuring no job is lost and enabling smooth scale-out.
Central store for runtime settings such as warm-pool size, VM SKU, image version, and spot/on-demand preferences. Runner-Controller reads these values on startup and on a configurable interval.
Holds long-lived secrets used by the Event-Handler and Runner-Controller (GitHub App private key, webhook secret, etc.). Accessed via Managed Identity with the minimum required permissions.
Stores the short-lived registration tokens generated by Runner-Controller. Each token is valid for a single VM and is deleted once the runner registers with GitHub.
Contains HashiCorp Packer-built VM images pre-loaded with the toolchain your organization needs. Runner-Controller always provisions the latest image version.
- GitHub App for Organization (owner access)
- Azure
- Subscription
- Note: Subscription quota for "Total Regional Spot vCPUs" should be increased to allow multiple spot instances
- Resource Group
- Subnet with internet access
- KeyVault for GitHub App Credential
- Service Principal Roles/Permissions
Microsoft.AppConfiguration/locations/deletedConfigurationStores/read
(Needed until closure of issue #19605 in hashicorp/terraform-provider-azurerm)
- Resource Provider Registration (See here for steps to register the resource provider)
Microsoft.AppConfiguration
- optional - Managed Image accessible by Runner-Controller
- Subscription
For convenience we have provided an image in our public Azure Community Gallery that can be used for quick setup, but you may want to build a custom image tailored to your use case. Referencing the Packer Template repo, create an image and publish it to Azure Compute Gallery that can be created by this Terraform module.
The GitHub App serves as the foundation for sending webhook events to App A and retrieving registration tokens to store in Azure Key Vault.
- Navigate: Settings → Developer Settings → GitHub Apps → New GitHub App
- Configure permissions
- Configure settings, webhook settings to be updated later
- Save App and take note of App ID, Client ID
Permission | Access |
---|---|
Repository: Actions | Read-only |
Repository: Checks | Read-only |
Repository: Metadata | Read-only |
Organization: Self-hosted runners | Read and write |
Required Field | Value |
---|---|
GitHub App Name | {insert-name} |
Homepage URL | {insert-any-url} |
Webhook Active | False |
Webhook URL | |
Subscribe to events* | Workflow job |
Where can this GitHub App be installed? | Only on this account |
*Note: You will need one GitHub App per org. Allowing installation to "Any account" makes it difficult to change access if installed on orgs outside your control. *Note: Initially the webhook is disabled, but will be enabled in the next step. You will only see 'Subscribed to events' after the webhook is enabled.
(optional) Set Key Vault name variable:
export KEYVAULT_NAME=<keyvault-name>
Runner Password:
az keyvault secret set --name azure-runner-default-password --value $(uuidgen) --vault-name $KEYVAULT_NAME
GitHub Client Secret:
az keyvault secret set --name github-client-secret --vault-name $KEYVAULT_NAME --value <secret-value>
GitHub Private Key:
az keyvault secret set --name github-private-key --encoding utf-8 --vault-name $KEYVAULT_NAME --file <location/pem>
Webhook Secret:
az keyvault secret set --name github-webhook-secret --value $(uuidgen) --vault-name $KEYVAULT_NAME
Note: The private key must be added via the AZ CLI, all other secrets can be added manually via the portal if you choose to do so.
Key Vault authorization models
• If your Key Vault was created with RBAC authorization (
enableRbacAuthorization = true
oraz keyvault create --enable-rbac-authorization true
), you must grant the Key Vault Secrets Officer (or Administrator) role to both the Event-Handler Function and the Runner-Controller App Service managed identities. Terraform cannot assign this role unless you opt into the upcoming RBAC support variable.• If your vault uses the older access-policy model (
enableRbacAuthorization = false
, the module default), Terraform will automatically create an access policy that lets the Function and Controller read secrets.Default behaviour: The Azure portal and recent versions of the Azure CLI (az ≥ 2.51) create Key Vaults with RBAC enabled by default. In contrast, the Terraform
azurerm_key_vault
resource keeps RBAC disabled by default—you must setenable_rbac_authorization = true
to opt-in. Keep this in mind when mixing manual and IaC-provisioned environments.
Below is a minimal working example you can copy-paste into a new main.tf
. Replace the placeholder values with your own IDs.
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = ">= 3.100"
}
azuread = {
source = "hashicorp/azuread"
version = ">= 2.45"
}
}
}
provider "azurerm" {
features {}
tenant_id = "00000000-0000-0000-0000-000000000000" # Azure AD tenant
subscription_id = "11111111-1111-1111-1111-111111111111" # Azure subscription
}
provider "azuread" {}
module "github_runners" {
source = "git::https://github.com/liatrio/terraform-azure-github-runner.git?ref=vX.Y.Z" # pin a release tag
# ---- Azure settings ----
azure_tenant_id = "00000000-0000-0000-0000-000000000000"
azure_subscription_id = "11111111-1111-1111-1111-111111111111"
azure_resource_group_name = "rg-github-runners"
azure_resource_group_location = "eastus"
azure_subnet_id = "/subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.Network/virtualNetworks/vnet-github-runners/subnets/subnet-runners"
# ---- GitHub App details ----
github_organization = "my-org"
github_app_id = "123456"
github_client_id = "Iv1.abcdef123456"
github_installation_id = "7891011"
# ---- Key Vault secret IDs (Key Vault *ID* not value) ----
azure_secrets_key_vault_resource_id = "/subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.KeyVault/vaults/kv-gh-secrets"
azure_runner_default_password_key_vault_id = "/.../secrets/azure-runner-default-password"
github_client_secret_key_vault_id = "/.../secrets/github-client-secret"
github_webhook_secret_key_vault_id = "/.../secrets/github-webhook-secret"
github_private_key_key_vault_id = "/.../secrets/github-private-key"
# ---- Misc ----
owners = ["00000000-0000-0000-0000-000000000000"] # Azure AD object IDs
}
output "function_webhook_url" {
value = module.github_runners.function_webhook_url
description = "URL to paste into the GitHub App webhook settings"
sensitive = true
}
Run Terraform:
terraform init
terraform plan -out tfplan
terraform apply tfplan
After terraform apply
completes, run:
terraform output -raw function_webhook_url
This returns the full Azure Function URL, including a ?code=
query-string. Copy the entire string – the code
value is the Function host key and must stay in the URL.
You will also need the GitHub webhook secret that you stored in Key Vault earlier:
az keyvault secret show \
--vault-name <your-kv-name> \
--name github-webhook-secret \
--query value -o tsv
Keep both values handy for the next step.
This terraform module is set up by default to use the latest version of both apps and deploy them on terraform apply
. Specific versions found in our public GitHub Packages and set in the terraform module inputs. If you choose to publish your own images, functionality to do so will be implemented soon™.
In the GitHub UI navigate to Settings → Developer settings → GitHub Apps → Your App and:
- Activate the webhook toggle.
- Webhook URL – paste the Function URL you copied from
terraform output
(it already contains thecode
host key). - Webhook secret – paste the value of
github-webhook-secret
you retrieved from Key Vault. - Save.
- Open Install App, select the gear icon next to your organization, under 'Repository access' select 'All repositories' and click Save.
The system is now wired together: GitHub sends signed workflow-job events to your Function, the Function enqueues work, and the Runner Controller spins up VMs on demand.
Below are the required inputs required to get started with this module. Some may be marked with an asterisk (*) which indicates we recommend you pull this from a data source. Examples of usage are coming soon.
Name | Description | Type |
---|---|---|
azure_tenant_id | Azure tenant ID | string |
azure_subscription_id | Azure subscription ID | string |
azure_resource_group_name | Resource Group that the components and runners will be created within | string |
azure_subnet_id | Azure subnet ID | string |
name_suffix | Identifying suffix that will be appended to all components created by this module (default: null ) |
string |
github_organization | GitHub organization | string |
github_app_id | GitHub App ID | string |
github_client_id | GitHub Client ID | string |
github_installation_id | GitHub App installation ID | string |
azure_secrets_key_vault_resource_id | Key Vault ID where GitHub secrets are stored | string |
*azure_runner_default_password_key_vault_id | Key Vault ID for Azure runner default password (data source) | string |
*github_client_secret_key_vault_id | Keyvault Vault ID for GitHub App client secret (data source) | string |
*github_webhook_secret_key_vault_id | Keyvault Vault ID for GitHub App webhook secret (data source) | string |
*github_private_key_key_vault_id | Keyvault Vault ID for GitHub App private key (data source) | string |
*owners | The list of owners that will be assigned to all components (data source) | list(string) |
One goal of this module is to minimize the number of customizations needed in order to run autoscaling self-hosted runners. With this being said, this list of optional inputs will grow but hopefully not so much that it becomes difficult to manage and get started with this solution.
Name | Description | Type | Default |
---|---|---|---|
log_level | Log level used across applications | string |
Information |
azure_gallery_image_id | Azure Compute Gallery image ID to be used in runner creation, leave default to use latest Liatrio public image |
string |
/communityGalleries/liatrio-4e8ffc8d-5950-4137-b02c-df028384cdcd /images/ubuntu_gh_runner /versions/latest |
azure_gallery_image_type | Azure Compute Gallery image type to be used in runner creation. Available options: 'community', 'direct-shared', 'rbac' | string |
community |
event_handler_image_tag | Event-Handler image tag to use from GitHub Packages | string |
latest |
runner_controller_image_tag | Runner-Controller image tag to use from GitHub Packages | string |
latest |
github_runner_group | Runner Group to register runners to | string |
Default |
tags | Map of tags that will be added to created resources | map(string) |
{} |
azure_location | Azure location in which to create resources | string |
location of the resource group |