Merge pull request #2710 from jeromevdl/feat/stepfunctions-eventbridge-onpremise-tf

New serverless pattern : step functions to on-premises API (terraform)

Author: julianwood

.gitignore

@@ -206,8 +206,8 @@ crash.log
 crash.*.log
 
 # Exclude all .tfvars files, which are likely to contain sensitive data, such as
-# password, private keys, and other secrets. These should not be part of version 
-# control as they are data points which are potentially sensitive and subject 
+# password, private keys, and other secrets. These should not be part of version
+# control as they are data points which are potentially sensitive and subject
 # to change depending on the environment.
 *.tfvars
 *.tfvars.json

stepfunctions-eventbridge-onpremise-tf/README.md

@@ -0,0 +1,90 @@
+# AWS Step Functions to on-premises API (Terraform)
+
+This pattern demonstrate how to call an on-premises API from a Step Functions state machine, leveraging Amazon EventBridge connection and VPC Lattice resource gateway and resource configuration.
+
+Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example.
+
+## Requirements
+
+* [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources.
+* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured
+* [Git Installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
+* [Terraform](https://learn.hashicorp.com/tutorials/terraform/install-cli?in=terraform/aws-get-started) installed
+
+## Deployment Instructions
+
+### Pre-requisites
+
+This example assumes you already have a VPC with a connection to your datacenter (through VPN or Direct Connect) and an API is exposed on-premises and accessible from this VPC.
+The VPC and connection to your datacenter are not provided by this example. Refer to this [documentation](https://docs.aws.amazon.com/whitepapers/latest/aws-vpc-connectivity-options/network-to-amazon-vpc-connectivity-options.html) to set up such connectivity.
+
+### Deployment
+
+1. Create a new directory, navigate to that directory in a terminal and clone the GitHub repository:
+    ``` 
+    git clone https://github.com/aws-samples/serverless-patterns
+    ```
+2. Change directory to the pattern directory:
+    ```
+    cd stepfunctions-eventbridge-onpremise-tf
+    ```
+3. Create a `.tfvars` file with the following variables (use your custom values)
+
+   ```
+   api_domain_name = "api.internal.mycompany.com"
+	api_key_secret_arn = "arn:aws:secretsmanager:us-east-1:123456789012:secret:07a4e645-fc95-4a10-853a-410b1b1eca5b-012nZO"
+	vpc_id = "vpc-0e03d4ab114e951be"
+	private_subnet_ids = ["subnet-05d53fa850148290e","subnet-070324fd8bc5885a5"]
+	on_premises_cidr = "172.32.0.0/20"
+	```
+	
+4. From the command line, use Terraform to deploy the AWS resources:
+    ```
+    terraform init
+    terraform apply -var-file=your-variables.tfvars
+    ```
+
+   When prompted "Do you want to deploy the infrastructure", type ```yes``` and press enter.
+
+5. Note the outputs from the terraform deployment process. These contain the resource ARNs which are used for testing.
+
+## How it works
+
+![Architecture](architecture.png)
+
+1. The HTTP task in Step Functions is leveraging an EventBridge Connection. It defines the target endpoint (e.g. https://my-internal-api.company.com/customer) and HTTP method (e.g. GET) as well as eventual HTTP headers.
+2. The EventBridge Connection defines the authentication mechanism (OAuth, Basic or API Key in this case) for the target endpoint as well as the resource configuration to use for a private/internal endpoint.
+3. The resource configuration defines the target endpoint itself, generally an on-premise IP address or DNS name (e.g. my-internal-api.company.com). Resource configuration is associated to a resource gateway.
+4. The resource gateway "opens a door" to the VPC and allow ingress. It is linked to the chosen subnets (generally private) and is also protected by a security group to further protect your backend API. Note: You could stop here at the VPC level, with a private API deployed in a private subnet.
+5. The site-to-site VPN or Direct Connect connection establishes the connection between the AWS cloud (generally with a VPN Gateway or a Transit Gateway) and your datacenter (through a Customer Gateway).
+6. Finally, the internal API that resides in your datacenter can be accessed via this "route".
+
+You can get more details in this [blog post](https://community.aws/content/2oExiwtkpK7go3wzAVzzF05ysqu).
+
+## Testing
+
+1. First make sure the EventBridge connection is active. Use the command `aws events describe-connection --name on-premise-connection --query ConnectionState` and verify it is `ACTIVE`. Otherwise, wait for an additional minute and verify again.
+2. Go to the AWS Step Functions console and open the state machine deployed by the example (`state-machine-call-onprem`).
+3. Click on `Start Execution` on the top right and again in the popup (no input is required for this example).
+4. Observe the result. Your on-premise API should be called by the state machine and an eventual result returned to the task.
+
+You can also use the AWS CLI with the following command (make sure to use the output of the terraform script):
+
+```shell
+ aws stepfunctions start-execution --state-machine-arn arn:aws:states:us-east-1:123456789012:stateMachine:state-machine-call-onprem
+```
+
+## Cleanup
+**To avoid incurring future charges, delete the resources created by the Terraform script.**
+1.	Return to the directory where you deployed your terraform script.
+2.	To destroy the infrastructure in AWS, run the command
+
+```bash
+    terraform destroy
+```
+   When prompted do you want to destroy the infrastructure, type ```yes``` and press enter.
+
+----
+Copyright 2025 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+
+SPDX-License-Identifier: MIT-0

stepfunctions-eventbridge-onpremise-tf/architecture.png

@@ -0,0 +1,59 @@
+{
+  "title": "Step Functions to on-premises API (Terraform)",
+  "description": "Step Functions performing HTTP call to on-premises API",
+  "language": "",
+  "level": "200",
+  "framework": "Terraform",
+  "introBox": {
+    "headline": "How it works",
+    "text": [
+      "This sample project demonstrates how to use an AWS Step Functions state machine to call an on-premises API without using an intermediary Lambda function. This pattern is leveraging EventBridge connection to connect to an HTTP endpoint and VPC Lattice to access private resources (in a VPC or on premises).",
+      "This pattern deploys one Step Functions, one EventBridge connection, a VPC Lattice resource configuration and resource gateway. You need to have a connection between a VPC and a datacenter (using VPN or Direct Connect)."
+    ]
+  },
+  "gitHub": {
+    "template": {
+      "repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/stepfunctions-eventbridge-onpremise-tf",
+      "templateURL": "serverless-patterns/stepfunctions-eventbridge-onpremise-tf",
+      "projectFolder": "stepfunctions-eventbridge-onpremise-tf",
+      "templateFile": "main.tf"
+    }
+  },
+  "resources": {
+    "bullets": [
+      {
+        "text": "Connect to private APIs using EventBridge connections",
+        "link": "https://docs.aws.amazon.com/eventbridge/latest/userguide/connection-private.html"
+      },
+      {
+        "text": "Call HTTPs endpoints using Step Functions HTTP Task",
+        "link": "https://docs.aws.amazon.com/step-functions/latest/dg/call-https-apis.html"
+      }
+    ]
+  },
+  "deploy": {
+    "text": [
+      "terraform init",
+      "terraform apply"
+    ]
+  },
+  "testing": {
+    "text": [
+      "See the GitHub repo for detailed testing instructions."
+    ]
+  },
+  "cleanup": {
+    "text": [
+      "terraform destroy"
+    ]
+  },
+  "authors": [
+    {
+      "name": "Jerome Van Der Linden",
+      "image": "https://serverlessland.com/assets/images/resources/contributors/jerome-van-der-linden.jpg",
+      "bio": "Jerome is a Solutions Architect Builder at AWS. Passionate about building stuff using the AWS services, and especially the serverless ones.",
+      "linkedin": "jeromevdl",
+      "twitter": "jeromevdl"
+    }
+  ]
+}

stepfunctions-eventbridge-onpremise-tf/example-pattern.json

@@ -0,0 +1,33 @@
+terraform {
+  required_version = ">= 1.0.0" # Ensure that the Terraform version is 1.0.0 or higher
+
+  required_providers {
+    aws = {
+      source = "hashicorp/aws" # Specify the source of the AWS provider
+      version = ">= 5.0.0"        # Use a version of the AWS provider that is compatible with version
+    }
+  }
+}
+
+provider "aws" {
+  region = "us-east-1"
+}
+
+module "eventbridge_connection" {
+  source = "./modules/eventbridge_connection"
+
+  vpc_id       = var.vpc_id
+  on_premises_cidr    = var.on_premises_cidr
+  api_domain_name    = var.api_domain_name
+  private_subnet_ids = var.private_subnet_ids
+  api_key_secret_arn         = var.api_key_secret_arn
+}
+
+module "state_machine" {
+  source = "./modules/state_machine"
+
+  connection_arn        = module.eventbridge_connection.connection_arn
+  connection_secret_arn = module.eventbridge_connection.connection_secret_arn
+  api_domain_name       = var.api_domain_name
+  log_retention_days    = 30
+}

stepfunctions-eventbridge-onpremise-tf/main.tf

@@ -0,0 +1,81 @@
+resource "aws_security_group" "resource_gateway_sg" {
+  name_prefix = "resource-gateway-sg"
+  description = "Security group for resource gateway"
+  vpc_id      = var.vpc_id
+
+  egress {
+    from_port   = 443
+    to_port     = 443
+    protocol    = "tcp"
+    cidr_blocks = [var.on_premises_cidr]
+    description = "Allow HTTPS traffic to on-premises"
+  }
+
+  egress {
+    from_port   = 80
+    to_port     = 80
+    protocol    = "tcp"
+    cidr_blocks = [var.on_premises_cidr]
+    description = "Allow HTTP traffic to on-premises"
+  }
+}
+
+resource "aws_vpclattice_resource_gateway" "on_premise_resource_gateway" {
+  name                = "resource-gateway"
+  ip_address_type     = "IPV4"
+  vpc_id              = var.vpc_id
+  security_group_ids  = [aws_security_group.resource_gateway_sg.id]
+  subnet_ids          = var.private_subnet_ids
+}
+
+resource "aws_vpclattice_resource_configuration" "on_premise_resource_configuration" {
+  name                          = "resource-config"
+  port_ranges                   = ["80", "443"]
+  protocol                      = "TCP"
+  resource_gateway_identifier   = aws_vpclattice_resource_gateway.on_premise_resource_gateway.id
+  type                          = "SINGLE"
+
+  resource_configuration_definition {
+    # uncomment if using ip address
+    # ip_resource {
+    #   ip_address = var.api_ip_address
+    # }
+
+    # remove if using ip address
+    dns_resource {
+      domain_name      = var.api_domain_name
+      ip_address_type  = "IPV4"
+    }
+  }
+}
+
+data "aws_secretsmanager_secret_version" "api_key" {
+  secret_id = var.api_key_secret_arn
+}
+
+resource "aws_cloudwatch_event_connection" "on_premise_connection" {
+  name               = "on-premise-connection"
+  description        = "Connection to on premises API"
+  authorization_type = "API_KEY"
+
+  auth_parameters {
+    # configure basic or oauth instead of api_key, depending on your authentication method
+    api_key {
+      key   = "x-api-key"
+      value = data.aws_secretsmanager_secret_version.api_key.secret_string
+    }
+    # eventually add http parameters (header, body or query string) to the connection
+    invocation_http_parameters {
+      header {
+        key = "x-origin"
+        value = "aws-state-machine"
+      }
+    }
+  }
+
+  invocation_connectivity_parameters {
+    resource_parameters {
+      resource_configuration_arn = aws_vpclattice_resource_configuration.on_premise_resource_configuration.arn
+    }
+  }
+}

stepfunctions-eventbridge-onpremise-tf/modules/eventbridge_connection/main.tf

@@ -0,0 +1,10 @@
+output "connection_arn" {
+  description = "ARN of the EventBridge connection"
+  value       = aws_cloudwatch_event_connection.on_premise_connection.arn
+}
+
+output "connection_secret_arn" {
+  description = "ARN of the secret for EventBridge connection"
+  value       = aws_cloudwatch_event_connection.on_premise_connection.secret_arn
+}
+

stepfunctions-eventbridge-onpremise-tf/modules/eventbridge_connection/outputs.tf

@@ -0,0 +1,31 @@
+variable "vpc_id" {
+  type        = string
+  description = "ID of the VPC linked to on-premises network"
+}
+
+variable "private_subnet_ids" {
+  type        = list(string)
+  description = "List of private subnet IDs in the VPC"
+}
+
+variable "on_premises_cidr" {
+  type        = string
+  description = "CIDR block of the on-premises network"
+}
+
+# Choose either a domain name if you have one configured for your API, or and IP address
+variable "api_domain_name" {
+  type        = string
+  description = "Domain name of the on-premises API"
+}
+
+# variable "api_ip_address" {
+#   type        = string
+#   description = "IP address of the on-premises API"
+# }
+
+# If using Basic or OAuth, change the authentication mechanism
+variable "api_key_secret_arn" {
+  type        = string
+  description = "ARN of the existing secret containing the API key"
+}