diff --git a/docs/apm-data-security.asciidoc b/docs/apm-data-security.asciidoc index 994937402bc..071bc69e2b2 100644 --- a/docs/apm-data-security.asciidoc +++ b/docs/apm-data-security.asciidoc @@ -483,18 +483,6 @@ TIP: If you prefer using a GUI, you can instead open {kib} and navigate to **Stack Management** -> **Ingest Pipelines** -> **Create pipeline**. Use the same naming convention explained previously to ensure your new pipeline matches the correct APM data stream. -**Roll over the data stream (optional)** - -Pipeline changes are not applied retroactively to existing indices. -For changes to take effect immediately, you must create a new write index for the data stream. -This can be done with the {es} {ref}/indices-rollover-index.html[Rollover API]. -For example, to roll over the default application traces data stream, with a `namespace` of "default", run: - -[source,console] ----- -POST /traces-apm-default/_rollover/ ----- - That's it! Passwords will now be redacted from your APM HTTP body data. To learn more about ingest pipelines, see <>. diff --git a/docs/command-reference.asciidoc b/docs/command-reference.asciidoc index e8f7df12f42..f3c030ffb4c 100644 --- a/docs/command-reference.asciidoc +++ b/docs/command-reference.asciidoc @@ -34,6 +34,7 @@ ifdef::serverless[] endif::serverless[] :help-command-short-desc: Shows help for any command +:keystore-command-short-desc: Manages the <> :modules-command-short-desc: Manages configured modules :package-command-short-desc: Packages the configuration and executable into a zip file :remove-command-short-desc: Removes the specified function from your serverless environment @@ -106,6 +107,9 @@ ifdef::apm-server[] endif::[] |<> |{export-command-short-desc}. |<> |{help-command-short-desc}. +ifndef::serverless[] +|<> |{keystore-command-short-desc}. +endif::[] ifeval::["{beatname_lc}"=="functionbeat"] |<> |{package-command-short-desc}. |<> |{remove-command-short-desc}. @@ -430,6 +434,65 @@ Specifies the name of the command to show help for. {beatname_lc} help export ----- +ifndef::serverless[] +[float] +[[keystore-command]] +==== `keystore` command + +{keystore-command-short-desc}. + +*SYNOPSIS* + +["source","sh",subs="attributes"] +---- +{beatname_lc} keystore SUBCOMMAND [FLAGS] +---- + +*`SUBCOMMAND`* + +*`add KEY`*:: +Adds the specified key to the keystore. Use the `--force` flag to overwrite an +existing key. Use the `--stdin` flag to pass the value through `stdin`. + +*`create`*:: +Creates a keystore to hold secrets. Use the `--force` flag to overwrite the +existing keystore. + +*`list`*:: +Lists the keys in the keystore. + +*`remove KEY`*:: +Removes the specified key from the keystore. + +*FLAGS* + +*`--force`*:: +Valid with the `add` and `create` subcommands. When used with `add`, overwrites +the specified key. When used with `create`, overwrites the keystore. + +*`--stdin`*:: +When used with `add`, uses the stdin as the source of the key's value. + +*`-h, --help`*:: +Shows help for the `keystore` command. + + +{global-flags} + +*EXAMPLES* + +["source","sh",subs="attributes"] +----- +{beatname_lc} keystore create +{beatname_lc} keystore add ES_PWD +{beatname_lc} keystore remove ES_PWD +{beatname_lc} keystore list +----- + +See <> for more examples. + +endif::[] + ifeval::["{beatname_lc}"=="functionbeat"] [float] [[package-command]] diff --git a/docs/custom-index-template.asciidoc b/docs/custom-index-template.asciidoc index 42699aede2d..aa18b1f16dc 100644 --- a/docs/custom-index-template.asciidoc +++ b/docs/custom-index-template.asciidoc @@ -15,7 +15,7 @@ These index templates are composed of multiple component templates--reusable bui that configure index mappings, settings, and aliases. The default APM index templates can be viewed in {kib}. -Navigate to **{stack-manage-app}** > **Index Management** > **Index Templates**, and search for `apm`. +Navigate to **{stack-manage-app}** → **Index Management** → **Index Templates**, and search for `apm`. Select any of the APM index templates to view their relevant component templates. [discrete] @@ -30,21 +30,27 @@ When you install the APM integration, {fleet} creates a default `@custom` compon You can edit this `@custom` component template to customize your {es} indices. First, determine which <> you'd like to edit. -Then, open {kib} and navigate to **{stack-manage-app}** > **Index Management** > **Component Templates**. +Then, open {kib} and navigate to **{stack-manage-app}** → **Index Management** → **Component Templates**. Custom component templates are named following this pattern: `@custom`. Search for the name of the data stream, like `traces-apm`, and select its custom component template. In this example, that'd be, `traces-apm@custom`. -Then click **Manage** > **Edit**. +Then click **Manage** → **Edit**. -Add any custom index settings, metadata, or mappings. -For example, you may want to... +Add any custom metadata, index settings, or mappings. + +[discrete] +[[custom-index-template-index-settings]] +==== Index settings + +In the **Index settings** step, you can specify custom {ref}/index-modules.html#index-modules-settings[index settings]. +For example, you could: * Customize the index lifecycle policy applied to a data stream. See <> for a walk-through. * Change the number of {ref}/scalability.html[shards] per index. -Specify the number of primary shards in the **index settings**: +Specify the number of primary shards: + [source,json] ---- @@ -56,7 +62,7 @@ Specify the number of primary shards in the **index settings**: ---- * Change the number of {ref}/docs-replication.html[replicas] per index. -Specify the number of replica shards in the **index settings**: +Specify the number of replica shards: + [source,json] ---- @@ -67,6 +73,29 @@ Specify the number of replica shards in the **index settings**: } ---- +[discrete] +[[custom-index-template-mappings]] +==== Mappings + +{ref}/mapping.html[Mapping] is the process of defining how a document, and the fields it contains, are stored and indexed. +In the *Mappings* step, you can add custom field mappings. +For example, you could: + +* Add custom field mappings that you can index on and search. +In the *Mapped fields* tab, add a new field including the {ref}/mapping-types.html[field type]: ++ +image::images/custom-index-template-mapped-fields.png[Editing a component template to add a new mapped field] + +* Add a {ref}/runtime.html[runtime field] that is evaluated at query time. +In the *Runtime fields* tab, click *Create runtime field* and provide a field name, +type, and optionally a script: ++ +image::images/custom-index-template-runtime-fields.png[Editing a component template to add a new runtime field] + +[discrete] +[[custom-index-template-rollover]] +=== Roll over the data stream + Changes to component templates are not applied retroactively to existing indices. For changes to take effect, you must create a new write index for the data stream. This can be done with the {es} {ref}/indices-rollover-index.html[Rollover API]. diff --git a/docs/images/custom-index-template-mapped-fields.png b/docs/images/custom-index-template-mapped-fields.png new file mode 100644 index 00000000000..cfd92d21744 Binary files /dev/null and b/docs/images/custom-index-template-mapped-fields.png differ diff --git a/docs/images/custom-index-template-runtime-fields.png b/docs/images/custom-index-template-runtime-fields.png new file mode 100644 index 00000000000..a792bcc164b Binary files /dev/null and b/docs/images/custom-index-template-runtime-fields.png differ diff --git a/docs/ingest-pipelines.asciidoc b/docs/ingest-pipelines.asciidoc index e5488fe1110..05cfcac9d26 100644 --- a/docs/ingest-pipelines.asciidoc +++ b/docs/ingest-pipelines.asciidoc @@ -60,7 +60,6 @@ The process for creating a custom ingest pipeline is as follows: * Create a pipeline with processors specific to your use case * Add the newly created pipeline to an `@custom` pipeline that matches an APM data stream -* Roll over your data stream If you prefer more guidance, see one of these tutorials: diff --git a/docs/keystore.asciidoc b/docs/keystore.asciidoc new file mode 100644 index 00000000000..06822a4af1a --- /dev/null +++ b/docs/keystore.asciidoc @@ -0,0 +1,109 @@ +[[keystore]] +=== Secrets keystore for secure settings + +++++ +Secrets keystore +++++ + +IMPORTANT: The APM Server keystore only applies to the APM Server binary installation method. + +When you configure APM Server, you might need to specify sensitive settings, +such as passwords. Rather than relying on file system permissions to protect +these values, you can use the APM Server keystore to securely store secret +values for use in configuration settings. + +After adding a key and its secret value to the keystore, you can use the key in +place of the secret value when you configure sensitive settings. + +The syntax for referencing keys is identical to the syntax for environment +variables: + +`${KEY}` + +Where KEY is the name of the key. + +For example, imagine that the keystore contains a key called `ES_PWD` with the +value `yourelasticsearchpassword`: + +* In the configuration file, use `output.elasticsearch.password: "${ES_PWD}"` +* On the command line, use: `-E "output.elasticsearch.password=\${ES_PWD}"` + +When APM Server unpacks the configuration, it resolves keys before resolving +environment variables and other variables. + +Notice that the APM Server keystore differs from the {es} keystore. +Whereas the {es} keystore lets you store `elasticsearch.yml` values by +name, the APM Server keystore lets you specify arbitrary names that you can +reference in the APM Server configuration. + +To create and manage keys, use the `keystore` command. +See the <> for the full command syntax, +including optional flags. + +NOTE: The `keystore` command must be run by the same user who will run +APM Server. + +[discrete] +[[creating-keystore]] +=== Create a keystore + +To create a secrets keystore, use: + +[source,sh] +----- +apm-server keystore create +----- + +APM Server creates the keystore in the directory defined by the `path.data` +configuration setting. + +[discrete] +[[add-keys-to-keystore]] +=== Add keys + +To store sensitive values, such as authentication credentials for {es}, +use the `keystore add` command: + +[source,sh] +----- +apm-server keystore add ES_PWD +----- + +When prompted, enter a value for the key. + +To overwrite an existing key's value, use the `--force` flag: + +[source,sh] +----- +apm-server keystore add ES_PWD --force +----- + +To pass the value through stdin, use the `--stdin` flag. You can also use +`--force`: + +[source,sh] +----- +cat /file/containing/setting/value | apm-server keystore add ES_PWD --stdin --force +----- + +[discrete] +[[list-settings]] +=== List keys + +To list the keys defined in the keystore, use: + +[source,sh] +----- +apm-server keystore list +----- + +[discrete] +[[remove-settings]] +=== Remove keys + +To remove a key from the keystore, use: + +[source,sh] +----- +apm-server keystore remove ES_PWD +----- diff --git a/docs/setting-up-and-running.asciidoc b/docs/setting-up-and-running.asciidoc index 796cda351ed..18c44af495f 100644 --- a/docs/setting-up-and-running.asciidoc +++ b/docs/setting-up-and-running.asciidoc @@ -12,12 +12,16 @@ for basic installation and running instructions. This section includes additional information on how to set up and run APM Server, including: * <> +* <> * <> +* <> * <> * <> include::{docdir}/shared-directory-layout.asciidoc[] +include::{docdir}/keystore.asciidoc[] + include::{docdir}/command-reference.asciidoc[] include::{docdir}/data-ingestion.asciidoc[]