Skip to content

Commit 4d7a357

Browse files
Add documentation page for the EDOT Collector attributes processor (#11331)
* Add attributeprocessor * Add link to toc * Apply comment * merge sections and add examples * Update to clarify resource attribute modifications and add new action type 'truncate'. * remove duplicate from toc
1 parent 4664a08 commit 4d7a357

File tree

2 files changed

+221
-0
lines changed

2 files changed

+221
-0
lines changed
Lines changed: 220 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,220 @@
1+
---
2+
navigation_title: Attributes processor
3+
description: The attributes processor is an OpenTelemetry Collector component that modifies resource attributes and span, metric, or log attributes before they are exported.
4+
applies_to:
5+
stack:
6+
serverless:
7+
observability:
8+
product:
9+
edot_collector:
10+
products:
11+
- id: elastic-agent
12+
- id: observability
13+
- id: edot-collector
14+
---
15+
16+
# Attributes processor
17+
18+
The attributes processor modifies telemetry attributes as they pass through the {{edot}} Collector pipeline. It can add, update, rename, hash, or delete attributes on spans, metrics, and logs before they reach downstream processors or exporters.
19+
20+
This processor is part of the core {{edot}} Collector distribution. It is useful when you need to normalize attribute names, remove sensitive fields, or enrich telemetry with additional context.
21+
22+
For full contrib details, refer to the [OpenTelemetry `attributesprocessor` documentation](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/attributesprocessor).
23+
24+
## How it works
25+
26+
The attributes processor applies a list of **actions** to matching telemetry. Each action defines:
27+
28+
* Action type (such as `insert`, `update`, `delete`, `hash`, or `rename`)
29+
* Target attribute key
30+
* Optional value (for inserts/updates)
31+
* Optional include/exclude match criteria
32+
33+
Actions run in order. Matching rules give fine-grained control over which spans, metrics, or logs the processor modifies.
34+
35+
## Typical use cases
36+
37+
The attributes processor is commonly used in {{product.observability}} pipelines to:
38+
39+
* Remove or transform sensitive attributes. For example, deleting user email fields or hashing IP addresses before they are exported.
40+
41+
* Normalize attribute naming across services. For example, converting custom key names (such as `"userId"`) to standard semantic conventions (`"user.id"`).
42+
43+
* Add static attributes, such as environment or cluster identifiers when they cannot be set at the source.
44+
45+
* Clean up noisy or irrelevant attributes. For example, removing temporary or dynamically generated labels that would otherwise create high-cardinality fields in {{es}}.
46+
47+
* Override or enrich resource attributes for standardizing deployment names, namespaces, or service metadata.
48+
49+
## Key configuration options
50+
51+
The following are the most important settings when configuring the attributes processor:
52+
53+
| Option | Description |
54+
|--------|-------------|
55+
| `actions` | A list of attribute modifications to apply. Each action uses the upstream-supported action types: `insert`, `update`, `upsert`, `delete`, `hash`, `extract`, `convert`, `rename`, `truncate`. |
56+
| `include` / `exclude` | Rules for matching telemetry based on service name, span name, resource attributes, log severity, or metric name. |
57+
| `match_type` | How attribute comparison is evaluated (`strict`, `regexp`, `expr`). |
58+
59+
For the complete list of configuration options and action-specific settings, refer to the [contrib `attributesprocessor` documentation](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/attributesprocessor).
60+
61+
## Example configurations
62+
63+
The following example deletes a sensitive attribute, renames an attribute, inserts a cluster ID, and hashes an IP address before export:
64+
65+
```yaml
66+
processors:
67+
attributes:
68+
actions:
69+
- key: user.email
70+
action: delete
71+
72+
- key: userId
73+
action: rename
74+
new_key: user.id
75+
76+
- key: cluster.id
77+
action: insert
78+
value: my-observability-cluster
79+
80+
- key: client.ip
81+
action: hash
82+
```
83+
84+
To enable the processor in a {{edot}} Collector pipeline:
85+
86+
```yaml
87+
service:
88+
pipelines:
89+
traces:
90+
receivers: [otlp]
91+
processors: [attributes]
92+
exporters: [elastic]
93+
```
94+
95+
### Remove or obfuscate sensitive information from logs
96+
97+
Remove sensitive data from log attributes before export to comply with privacy regulations or security policies:
98+
99+
```yaml
100+
processors:
101+
attributes:
102+
actions:
103+
# Delete sensitive fields - illustrative example (not real values)
104+
- key: user.password
105+
action: delete
106+
- key: credit_card.number
107+
action: delete
108+
- key: ssn
109+
action: delete
110+
111+
# Hash IP addresses for privacy
112+
- key: client.ip
113+
action: hash
114+
- key: server.ip
115+
action: hash
116+
117+
# Remove authorization headers from HTTP spans
118+
- key: http.request.header.authorization
119+
action: delete
120+
- key: http.request.header.cookie
121+
action: delete
122+
```
123+
124+
### Normalize attribute names across multiple services
125+
126+
Standardize attribute naming across different services that use inconsistent conventions:
127+
128+
```yaml
129+
processors:
130+
attributes:
131+
actions:
132+
# Normalize user ID attributes
133+
- key: userId
134+
action: rename
135+
new_key: user.id
136+
- key: user_id
137+
action: rename
138+
new_key: user.id
139+
- key: UserID
140+
action: rename
141+
new_key: user.id
142+
143+
# Normalize environment attributes
144+
- key: env
145+
action: rename
146+
new_key: deployment.environment
147+
- key: environment
148+
action: rename
149+
new_key: deployment.environment
150+
151+
# Convert custom service names to semantic conventions
152+
- key: app.name
153+
action: rename
154+
new_key: service.name
155+
```
156+
157+
### Add environment metadata conditionally
158+
159+
Enrich telemetry with environment-specific attributes based on service or resource attributes:
160+
161+
```yaml
162+
processors:
163+
attributes:
164+
# Add cluster ID to all telemetry from production namespace
165+
include:
166+
match_type: strict
167+
resource_attributes:
168+
- key: k8s.namespace.name
169+
value: production
170+
actions:
171+
- key: cluster.id
172+
action: insert
173+
value: prod-cluster-01
174+
- key: environment
175+
action: insert
176+
value: production
177+
- key: region
178+
action: insert
179+
value: us-east-1
180+
```
181+
182+
## Matching and filtering
183+
184+
You can limit attribute modifications to only specific spans, logs, or metrics using `include` and `exclude` blocks:
185+
186+
```yaml
187+
processors:
188+
attributes:
189+
include:
190+
match_type: strict
191+
services: ["checkout-service"]
192+
actions:
193+
- key: http.request.header.authorization
194+
action: delete
195+
```
196+
197+
This example removes an authorization header only from telemetry produced by the `checkout-service`.
198+
199+
:::{note}
200+
Deleting keys like `http.request.header.authorization` only works if the key exists as a flattened attribute. Some SDKs store headers inside structured maps, which are not supported by the attributes processor.
201+
:::
202+
203+
## Best practices and caveats
204+
205+
When using the attributes processor, keep these recommendations and constraints in mind:
206+
207+
* Plan action order carefully. Actions run sequentially, so a `delete` followed by an `update` on the same key will behave differently than the reverse order. Consider the sequence of operations when designing your attribute transformations.
208+
209+
* Use `include` and `exclude` filters to target specific telemetry. Instead of applying actions to all telemetry, limit modifications to specific services, spans, or metrics. This reduces processing overhead and prevents unintended side effects.
210+
211+
* Be cautious with dynamic attribute insertion. Inserting dynamically changing attributes can significantly increase cardinality in {{es}}, leading to performance issues and increased storage costs. Use static values or carefully controlled dynamic values to avoid high-cardinality issues.
212+
213+
* Consider downstream processor dependencies when renaming attributes. If later processors in your pipeline rely on specific attribute keys (for example, `service.name`), renaming those attributes can cause failures or unexpected behavior. Review your pipeline configuration to understand attribute dependencies.
214+
215+
* Understand that hashing is irreversible. Once hashed, attribute values cannot be retrieved or correlated using their original value. Use hashing for sensitive data that doesn't need to be queried or correlated later.
216+
217+
## Resources
218+
219+
* [Contrib component: attributesprocessor](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/attributesprocessor)
220+
* [OpenTelemetry semantic conventions](https://opentelemetry.io/docs/specs/semconv/)

docs/reference/edot-collector/toc.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ toc:
1515
- file: components.md
1616
children:
1717
- file: components/apmconfigextension.md
18+
- file: components/attributesprocessor.md
1819
- file: components/elasticapmconnector.md
1920
- file: components/elasticapmintakereceiver.md
2021
- file: components/elasticapmprocessor.md

0 commit comments

Comments
 (0)