Skip to content

adding the new config for aws lambda and defaults #11509

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

adding the new config for aws lambda and defaults #11509

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
5cfa68e
adding the new config for aws lambda and defaults
AntonEliatra Nov 10, 2025
db8e3ac
removing invocation_type #9872
AntonEliatra Nov 10, 2025
7fd3853
addressing PR comments
AntonEliatra Nov 12, 2025
9f29fdb
Merge branch 'main' into updating-AWS-lambda-data-prepper
AntonEliatra Nov 18, 2025
f13a26d
Update aws-lambda.md
AntonEliatra Nov 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 11 additions & 13 deletions _data-prepper/pipelines/configuration/processors/aws-lambda.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,7 @@ You can configure the processor using the following configuration options.

Field | Type | Required | Description
-------------------- | ------- | -------- | ----------------------------------------------------------------------------
`function_name` | String | Required | The name of the AWS Lambda function to invoke. Must be 3--500 characters.
`invocation_type` | String | Optional | Specifies the invocation type, either `request-response` or `event`. Default is `request-response`.
`function_name` | String | Required | The name of the AWS Lambda function to invoke. Must be 3--500 characters.
`aws.region` | String | Required | The AWS Region in which the Lambda function is located.
`aws.sts_role_arn` | String | Optional | The Amazon Resource Name (ARN) of the role to assume before invoking the Lambda function. Must be 20--2048 characters.
`aws.sts_external_id` | String | Optional | An external ID for STS role assumption. Must be 2--1224 characters.
Expand All @@ -45,31 +44,30 @@ Field | Type | Required | Description
`circuit_breaker_retries` | Integer | Optional | The maximum number of circuit breaker checks before proceeding. Default is `0`.
`circuit_breaker_wait_interval` | Long | Optional | The amount of time, in milliseconds, between circuit breaker checks. Default is `1000ms`.

#### Example configuration
The following is example configuration:

```yaml
processors:
- aws_lambda:
function_name: "my-lambda-function"
invocation_type: "request-response"
function_name: my-lambda-function
response_events_match: false
response_mode: "replace"
response_mode: replace
aws:
region: "us-east-1"
sts_role_arn: "arn:aws:iam::123456789012:role/my-lambda-role"
region: us-east-1
sts_role_arn: arn:aws:iam::123456789012:role/my-lambda-role
client:
max_retries: 3
api_call_timeout: "PT60S"
connection_timeout: "PT60S"
api_call_timeout: PT60S
connection_timeout: PT60S
max_concurrency: 200
base_delay: "PT0.1S"
max_backoff: "PT20S"
batch:
key_name: "events"
key_name: events
threshold:
event_count: 100
maximum_size: "5mb"
event_collect_timeout: "PT10S"
maximum_size: 5mb
event_collect_timeout: PT10S
lambda_when: "/some_key == null"
keys: ["key1", "key2"]
cache:
Expand Down