Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Remove references to the mutate filter #12084

Merged
merged 2 commits into from
Nov 24, 2023
Merged

docs: Remove references to the mutate filter #12084

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0815388
remove references to mutate filter
colleenmcginnis Nov 20, 2023
1249fab
Merge branch 'main' into issue-11917
simitt Nov 24, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 1 addition & 59 deletions docs/configure/outputs/logstash.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ include::../../shared-kibana-endpoint.asciidoc[tag=shared-kibana-config]
=== {ls} configuration pipeline

Finally, you must create a {ls} configuration pipeline that listens for incoming
APM Server connections, dedots the `data_stream.*` fields, and indexes received events into {es}.
APM Server connections and indexes received events into {es}.

. Use the {logstash-ref}/plugins-inputs-elastic_agent.html[Elastic Agent input plugin] to configure
{ls} to receive events from the APM Server. A minimal `input` config might look like this:
Expand All @@ -68,54 +68,6 @@ input {
}
----

. Use the {logstash-ref}/plugins-filters-mutate.html[Mutate filter plugin] to set up <<apm-data-streams,data streams>>.
Because the {ls} {es} output doesn't understand dotted field notation, you must use this filter to
dedot the default `data_stream.*` fields sent from APM Server to {ls}.
+
[source,conf]
----
filter {
mutate {
rename => {
"[data_stream.type]" => "[data_stream][type]"
"[data_stream.dataset]" => "[data_stream][dataset]"
"[data_stream.namespace]" => "[data_stream][namespace]"
}
}
}
----
+
.Expand to learn more
[%collapsible]
====
****
APM Server sends data stream information to {ls} in the following format:
[source,json]
----
{
"data_stream.dataset": "apm",
"data_stream.type": "traces",
"data_stream.namespace": "default"
}
----
{es} expects to receive data stream information in the following format:
[source,json]
----
"data_stream" {
"dataset": "apm",
"type": "traces",
"dataset": "default"
}
----
The mutation defined above transforms what APM Server sends to {ls} into a data format that {es} understands.
This allows you to automatically route APM data to the appropriate data streams.
****
====

. Use the {logstash-ref}/plugins-outputs-elasticsearch.html[{es} output plugin] to send
events to {es} for indexing. A minimal `output` config might look like this:
+
Expand All @@ -142,16 +94,6 @@ input {
}
}
filter {
mutate {
rename => {
"[data_stream.type]" => "[data_stream][type]"
"[data_stream.dataset]" => "[data_stream][dataset]"
"[data_stream.namespace]" => "[data_stream][namespace]"
}
}
}
output {
elasticsearch {
data_stream => "true"
Expand Down