Skip to content

Commit 45a9ff7

Browse files
Merge pull request #1600 from fluent/conditional-processing-fixes
Conditional processing doc fixes
2 parents 67c6a3c + 00b2cf2 commit 45a9ff7

File tree

2 files changed

+40
-27
lines changed

2 files changed

+40
-27
lines changed

pipeline/processors/README.md

+13-9
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,28 @@
11
# Processors
22

3-
Processors are components that modify, transform, or enhance data as it flows through Fluent Bit.
4-
Unlike [filters](../filters/README.md), processors are tightly coupled to inputs, which means they
5-
execute immediately and avoid creating a performance bottleneck.
3+
Processors are components that modify, transform, or enhance data as it flows
4+
through Fluent Bit. Unlike [filters](../filters/README.md), processors are
5+
tightly coupled to inputs, which means they execute immediately and avoid
6+
creating a performance bottleneck.
67

7-
Additionally, filters can be implemented in a way that mimics the behavior of processors, but
8-
processors can't be implemented in a way that mimics filters.
8+
Additionally, filters can be implemented in a way that mimics the behavior of
9+
processors, but processors can't be implemented in a way that mimics filters.
910

1011
## Available processors
1112

1213
Fluent Bit offers the following processors:
1314

14-
- [Content Modifier](content-modifier.md): Manipulate the content, metadata, and attributes of logs and traces.
15+
- [Content Modifier](content-modifier.md): Manipulate the content, metadata, and
16+
attributes of logs and traces.
1517
- [Labels](labels.md): Add, update, or delete metric labels.
1618
- [Metrics Selector](metrics-selector.md): Choose which metrics to keep or discard.
17-
- [OpenTelemetry Envelope](opentelemetry-envelope.md): Transform logs into an OpenTelemetry-compatible format.
19+
- [OpenTelemetry Envelope](opentelemetry-envelope.md): Transform logs into an
20+
OpenTelemetry-compatible format.
1821
- [SQL](sql.md): Use SQL queries to extract log content.
1922

2023
## Features
2124

22-
All available processors include the following features:
25+
Compatible processors include the following features:
2326

24-
- [Conditional Processing](conditional-processing.md): Apply processors selectively to records that meet specific criteria.
27+
- [Conditional Processing](conditional-processing.md): Selectively apply processors
28+
to logs based on the value of fields that those logs contain.

pipeline/processors/conditional-processing.md

+27-18
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,19 @@
11
# Conditional processing
22

3-
Conditional processing lets you selectively apply [processors](README.md) to logs based on
4-
the value of fields that those logs contain. This feature lets you create processing pipelines
5-
that only process records that meet certain criteria, and ignore the rest.
3+
Conditional processing lets you selectively apply [processors](README.md) to
4+
logs based on the value of fields that those logs contain. This feature lets you
5+
create processing pipelines that only process records that meet certain
6+
criteria, and ignore the rest.
7+
8+
Conditional processing is available in Fluent Bit version 4.0 and greater.
69

710
## Configuration
811

9-
You can turn a standard processor into a conditional processor by adding a `condition` block to the
10-
processor's YAML configuration settings.
12+
You can turn a standard processor into a conditional processor by adding a
13+
`condition` block to the processor's YAML configuration settings.
1114

1215
{% hint style="info" %}
13-
Conditional processing is only available for
14-
[YAML configuration files](../../administration/configuring-fluent-bit/yaml/README.md),
15-
not [classic configuration files](../../administration/configuring-fluent-bit/classic-mode/README.md).
16+
Conditional processing is only available for [YAML configuration files](../../administration/configuring-fluent-bit/yaml/README.md), not [classic configuration files](../../administration/configuring-fluent-bit/classic-mode/README.md).
1617
{% endhint %}
1718

1819

@@ -38,15 +39,19 @@ pipeline:
3839
<...>
3940
```
4041

41-
Each processor can only have a single `condition` block, but can have multiple rules within that condition.
42-
These rules are stored as items in the `condition.rules` array.
42+
Each processor can only have a single `condition` block, but that condition can
43+
include multiple rules. These rules are stored as items in the `condition.rules`
44+
array.
4345

4446
### Condition evaluation
4547

46-
The `condition.op` parameter specifies the condition's evaluation logic. It has two possible values:
48+
The `condition.op` parameter specifies the condition's evaluation logic. It has
49+
two possible values:
4750

48-
- `and`: All rules in the `condition.rules` array must evaluate to `true` for the condition to be met.
49-
- `or`: One or more rules in the `conditions.rules` array must evaluate to `true` for the condition to be met.
51+
- `and`: A log entry meets this condition when all of the rules in the `condition.rules`
52+
are [truthy](https://developer.mozilla.org/en-US/docs/Glossary/Truthy).
53+
- `or`: A log entry meets this condition when one or more rules in the `condition.rules`
54+
array are [truthy](https://developer.mozilla.org/en-US/docs/Glossary/Truthy).
5055

5156
### Rules
5257

@@ -56,7 +61,7 @@ Each item in the `condition.rules` array must include values for the following p
5661
| --- | --- |
5762
| `field` | The field within your logs to evaluate. The value of this parameter must use [the correct syntax](#field-access) to access the fields inside logs. |
5863
| `op` | The [comparison operator](#comparison-operators) to evaluate whether the rule is true. This parameter (`condition.rules.op`) is distinct from the `condition.op` parameter and has different possible values. |
59-
| `value` | The value of the specified log field to use in your comparison. Optionally, you can provide [an array that contains multiple values](#array-of-values). |
64+
| `value` | The value of the specified log field to use in your comparison. Optionally, you can provide [an array that contains multiple values](#array-of-values). |
6065

6166
Rules are evaluated against each log that passes through your data pipeline. For example, given a rule with these parameters:
6267

@@ -93,7 +98,8 @@ The `conditions.rules.op` parameter has the following possible values:
9398

9499
### Basic condition
95100

96-
This example applies a condition that only processes logs that contain the string `{"request": {"method": "POST"`:
101+
This example applies a condition that only processes logs that contain the
102+
string `{"request": {"method": "POST"`:
97103

98104
```yaml
99105
pipeline:
@@ -117,7 +123,8 @@ pipeline:
117123
118124
### Multiple conditions with `and`
119125

120-
This example applies a condition that only processes logs when all of the specified rules are met:
126+
This example applies a condition that only processes logs when all of the
127+
specified rules are met:
121128

122129
```yaml
123130
pipeline:
@@ -144,7 +151,8 @@ pipeline:
144151

145152
### Multiple conditions with `or`
146153

147-
This example applies a condition that only processes logs when one or more of the specified rules are met:
154+
This example applies a condition that only processes logs when one or more of
155+
the specified rules are met:
148156

149157
```yaml
150158
pipeline:
@@ -234,4 +242,5 @@ pipeline:
234242
value: ["error", "fatal"]
235243
```
236244

237-
This configuration would add the `alert` field to error logs from critical services, and add the `paging_required` field to errors containing specific critical patterns.
245+
This configuration adds an `alert` field to error logs from critical services,
246+
and adds a `paging_required` field to errors that contain specific critical patterns.

0 commit comments

Comments
 (0)