http: new body transform filter with the substitution formatter #41612
+2,467
−20
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| # DO NOT EDIT. This file is generated by tools/proto_format/proto_sync.py. | ||
|
|
||
| load("@envoy_api//bazel:api_build_system.bzl", "api_proto_package") | ||
|
|
||
| licenses(["notice"]) # Apache 2 | ||
|
|
||
| api_proto_package( | ||
| deps = [ | ||
| "//envoy/config/common/mutation_rules/v3:pkg", | ||
| "//envoy/config/core/v3:pkg", | ||
| "@com_github_cncf_xds//udpa/annotations:pkg", | ||
| ], | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,99 @@ | ||
| syntax = "proto3"; | ||
|
|
||
| package envoy.extensions.filters.http.transform.v3; | ||
|
|
||
| import "envoy/config/common/mutation_rules/v3/mutation_rules.proto"; | ||
| import "envoy/config/core/v3/substitution_format_string.proto"; | ||
|
|
||
| import "udpa/annotations/status.proto"; | ||
| import "validate/validate.proto"; | ||
|
|
||
| option java_package = "io.envoyproxy.envoy.extensions.filters.http.transform.v3"; | ||
| option java_outer_classname = "TransformProto"; | ||
| option java_multiple_files = true; | ||
| option go_package = "github.com/envoyproxy/go-control-plane/envoy/extensions/filters/http/transform/v3;transformv3"; | ||
| option (udpa.annotations.file_status).package_version_status = ACTIVE; | ||
|
|
||
| // [#protodoc-title: Transform filter configuration] | ||
| // Transform filter :ref:`configuration overview <config_http_filters_transform>` to perform | ||
| // HTTP header and body transformations. | ||
| // [#extension: envoy.filters.http.transform] | ||
|
|
||
| // Configuration for the transform filter. The filter may buffer the request/response until the | ||
| // entire body is received, and then mutate the headers and body according to the contents | ||
| // of the request/response. The request and response transformations are independent and could | ||
| // be configured separately. | ||
| // Only JSON body transformation is supported for now. | ||
| message TransformConfig { | ||
wbpcode marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| // Configuration for transforming request. | ||
| // | ||
| // .. note:: | ||
| // | ||
| // If set then the entire request headers and body will always be buffered on a JSON request | ||
| // even if only headers are transformed. | ||
| Transformation request_transformation = 1; | ||
|
|
||
| // Configuration for transforming response. | ||
| // | ||
| // .. note:: | ||
| // | ||
| // If set then the entire response headers and body will always be buffered on a JSON response | ||
| // even if only headers are transformed. | ||
| Transformation response_transformation = 2; | ||
|
|
||
| // If true and the request headers are transformed, Envoy will re-evaluate the target | ||
| // cluster in the same route. Please ensure the cluster specifier in the route supports | ||
| // dynamic evaluation or this flag will have no effect, e.g. | ||
| // :ref:`matcher cluster specifier | ||
| // <envoy_v3_api_msg_extensions.router.cluster_specifiers.matcher.v3.MatcherClusterSpecifier>`. | ||
| // | ||
| // Only one of ``clear_cluster_cache`` and ``clear_route_cache`` can be true. | ||
| bool clear_cluster_cache = 3; | ||
|
|
||
| // If true and the request headers are transformed, Envoy will clear the route cache for | ||
| // the current request and force re-evaluation of the route. This has performance penalty and | ||
| // should only be used when the route match criteria depends on the transformed headers. | ||
| // | ||
| // Only one of ``clear_cluster_cache`` and ``clear_route_cache`` can be true. | ||
| bool clear_route_cache = 4; | ||
| } | ||
|
|
||
| message Transformation { | ||
| // The header mutations to perform. | ||
| // The :ref:`substitution format specifier <config_access_log_format>` could be applied here. | ||
| // In addition to the commonly used format specifiers, this filter introduces additional format specifiers: | ||
| // | ||
| // * ``%REQUEST_BODY(KEY*)%``: the request body. And ``Key`` KEY is an optional | ||
| // lookup key in the namespace with the option of specifying nested keys separated by ':'. | ||
| // * ``%RESPONSE_BODY(KEY*)%``: the response body. And ``Key`` KEY is an optional | ||
| // lookup key in the namespace with the option of specifying nested keys separated by ':'. | ||
| repeated config.common.mutation_rules.v3.HeaderMutation headers_mutations = 1; | ||
|
Contributor
There was a problem hiding this comment. Will the headers be blocked until the body is buffered even if
Member
Author
There was a problem hiding this comment. For now, yes for simplification of the code. But this is good point that could be optimized. Hmmm, it will require the decodeData to handle the continue or stop of decodeHeaders correctly. Seems it's not deserved to optimize because the additional complexity 🤔 The filter will not be performant anyway because the body processing.
Contributor
There was a problem hiding this comment. I understand that this is an implementation issue, but I think it should be spelled clearly in a note above (example: buffering of the entire request headers and body will always happen on a request even if only headers are transformed). |
||
|
|
||
| // The body transformation configuration. If not set, no body transformation will be performed. | ||
| BodyTransformation body_transformation = 2; | ||
|
Contributor
There was a problem hiding this comment. Do you see a case for multiple body transformations?
Member
Author
There was a problem hiding this comment. Nope. This is a one time logic. We will generate a new body based on the
Contributor
There was a problem hiding this comment. I guess the same can be said about headers, but I agree that headers are more likely to have multiple transformations (one for each header key). |
||
| } | ||
|
|
||
| message BodyTransformation { | ||
| enum TransformAction { | ||
| // Merge the transformed body with the original body. This is the default action. | ||
| MERGE = 0; | ||
|
|
||
| // Replace the original body with the transformed body. | ||
| REPLACE = 1; | ||
| } | ||
|
|
||
| // Body transformation configuration. The substitution format string is used as the template | ||
| // to generate the transformed new body content. | ||
| // The :ref:`substitution format specifier <config_access_log_format>` could be applied here. | ||
| // And except the commonly used format specifiers, the additional format specifiers | ||
| // ``%REQUEST_BODY(KEY*)%`` and ``%RESPONSE_BODY(KEY*)%`` could also be used here. | ||
| config.core.v3.SubstitutionFormatString body_format = 1 | ||
| [(validate.rules).message = {required: true}]; | ||
|
|
||
| // The action to perform for new body content and original body content. | ||
| // For example, if ``MERGE`` is used, then the new body content generated from the ``body_format`` | ||
| // will be merged into the original body content. | ||
| // | ||
| // Default is ``MERGE``. | ||
| TransformAction action = 2; | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| static_resources: | ||
| listeners: | ||
| - name: listener_0 | ||
| address: | ||
| socket_address: | ||
| protocol: TCP | ||
| address: 0.0.0.0 | ||
| port_value: 10000 | ||
| filter_chains: | ||
| - filters: | ||
| - name: envoy.filters.network.http_connection_manager | ||
| typed_config: | ||
| "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager | ||
| stat_prefix: ingress_http | ||
| route_config: | ||
| name: local_route | ||
| virtual_hosts: | ||
| - name: local_service | ||
| domains: | ||
| - "*" | ||
| routes: | ||
| - match: | ||
| prefix: "/route/override" | ||
| route: | ||
| host_rewrite_literal: upstream.com | ||
| cluster: upstream_com | ||
| typed_per_filter_config: | ||
| transform: | ||
| "@type": type.googleapis.com/envoy.extensions.filters.http.transform.v3.TransformConfig | ||
| request_transformation: | ||
| headers_mutations: | ||
| - append: | ||
| header: | ||
| key: "model-header" | ||
| value: "%REQUEST_BODY(model)%" | ||
| append_action: OVERWRITE_IF_EXISTS_OR_ADD | ||
| - match: | ||
| prefix: "/" | ||
| route: | ||
| host_rewrite_literal: upstream.com | ||
| cluster: upstream_com | ||
| http_filters: | ||
| - name: transform | ||
| typed_config: | ||
| "@type": type.googleapis.com/envoy.extensions.filters.http.transform.v3.TransformConfig | ||
| request_transformation: | ||
| headers_mutations: | ||
| - append: | ||
| header: | ||
| key: "model-header" | ||
| value: "%REQUEST_BODY(model)%" | ||
| append_action: OVERWRITE_IF_EXISTS_OR_ADD | ||
| body_transformation: | ||
| body_format: | ||
| json_format: | ||
| model: "new-model" | ||
| action: MERGE | ||
| response_transformation: | ||
| headers_mutations: | ||
| - append: | ||
| header: | ||
| key: "prompt-tokens" | ||
| value: "%RESPONSE_BODY(usage:prompt_tokens)%" | ||
| append_action: OVERWRITE_IF_EXISTS_OR_ADD | ||
| - append: | ||
| header: | ||
| key: "completion-tokens" | ||
| value: "%RESPONSE_BODY(usage:completion_tokens)%" | ||
| append_action: OVERWRITE_IF_EXISTS_OR_ADD | ||
| - name: envoy.filters.http.router | ||
| typed_config: | ||
| "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router | ||
|
|
||
| clusters: | ||
| - name: upstream_com | ||
| type: LOGICAL_DNS | ||
| # Comment out the following line to test on v6 networks | ||
| dns_lookup_family: V4_ONLY | ||
| lb_policy: ROUND_ROBIN | ||
| load_assignment: | ||
| cluster_name: service_upstream_com | ||
| endpoints: | ||
| - lb_endpoints: | ||
| - endpoint: | ||
| address: | ||
| socket_address: | ||
| address: upstream.com | ||
| port_value: 443 | ||
| transport_socket: | ||
| name: envoy.transport_sockets.tls | ||
| typed_config: | ||
| "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext | ||
| sni: upstream.com |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -69,3 +69,4 @@ HTTP filters | |
| thrift_to_metadata_filter | ||
| upstream_codec_filter | ||
| wasm_filter | ||
| transform_filter | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,60 @@ | ||
| .. _config_http_filters_transform: | ||
|
|
||
| Transform | ||
| ========= | ||
|
|
||
| * This filter should be configured with the type URL ``type.googleapis.com/envoy.extensions.filters.http.transform.v3.TransformConfig``. | ||
| * :ref:`v3 API reference <envoy_v3_api_msg_extensions.filters.http.transform.v3.TransformConfig>` | ||
|
|
||
| This filter can be used to transform HTTP requests and responses with | ||
| :ref:`substitution format string <envoy_v3_api_msg_config.core.v3.SubstitutionFormatString>`. For example, it can be used to: | ||
|
|
||
| * Modify request or response headers based on values in the body and refresh the routes accordingly. | ||
| * Modify JSON request or response bodies. | ||
|
|
||
| Configuration | ||
| ------------- | ||
|
|
||
| The following example configuration will extract the ``model`` field from a JSON request body | ||
| and add it as a request header ``model-header`` before forwarding the request to the upstream. | ||
| At the same time, it will also rewrite the ``model`` field in the JSON response body as ``new-model``. | ||
|
|
||
| At the response path, the filter is configured to extract the ``completion_tokens`` and ``prompt_tokens`` | ||
| fields from the JSON response body and add them as response headers. | ||
|
|
||
| .. literalinclude:: _include/transform_filter.yaml | ||
| :language: yaml | ||
| :lines: 42-69 | ||
| :lineno-start: 42 | ||
| :linenos: | ||
| :caption: :download:`transform_filter.yaml <_include/transform_filter.yaml>` | ||
|
|
||
| Per-route configuration | ||
| ----------------------- | ||
|
|
||
| Per-route overrides may be supplied via the same protobuf API in the ``typed_per_filter_config`` | ||
| field of route configuration. | ||
|
|
||
| The following example configuration will override the global filter configuration to keep | ||
| only the request headers transformation. | ||
|
|
||
| .. literalinclude:: _include/transform_filter.yaml | ||
| :language: yaml | ||
| :lines: 26-35 | ||
| :lineno-start: 26 | ||
| :linenos: | ||
| :caption: :download:`transform_filter.yaml <_include/transform_filter.yaml>` | ||
|
|
||
| Enhanced substitution format | ||
| ---------------------------- | ||
|
|
||
| The :ref:`substitution format specifier <config_access_log_format>` could be used for both | ||
| headers and body transformations. | ||
|
|
||
| And except the commonly used format specifiers, there are some additional format specifiers | ||
| provided by the transform filter: | ||
|
|
||
| * ``%REQUEST_BODY(KEY*)%``: the request body. And ``Key`` KEY is an optional | ||
| lookup key in the namespace with the option of specifying nested keys separated by ':'. | ||
| * ``%RESPONSE_BODY(KEY*)%``: the response body. And ``Key`` KEY is an optional | ||
| lookup key in the namespace with the option of specifying nested keys separated by ':'. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder if there should just be a JsonTransform filter here, instead of something generic. The reason I bring this up is because the transformations seem to be highly coupled with JSON bodies. (this is a discussion and not a a call for action).
WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's actually depends the feedback from the users and community, I personally hope this filter could handle various content type like plain text or url form by extend other fields in this filter.
JSON is supported first is because that the JSON is used widely for now in open API and AI related things. In addition, for the response, we need to support event stream to support AI traffic.