[elastic_agent] Fix metrics dashboard to include Elastic Defend

[AndersonQ]

Proposed commit message

[elastic_agent] Fix Elastic Agent metrics dashboard to include Elastic Defend

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• [ ] I have verified that Kibana version constraints are current according to guidelines.
• [ ] I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

• spin up 8.15+ elastic stack on cloud
  I suggest using a cloud stack because you'll need to install Elastic Defend and it's not possible on a Elastic computer. Thus you'll need a VM and it's easier with a cloud stack than with elastic-package
• have a elastic agent 8.15+ with Elastic Defend in the policy
  • go to the beats repo and spin up a VM

cd ../beats
vagrant up beats
vagrant ssh beats
# install elastic-agent

• setup the following environment variables:

export ELASTIC_PACKAGE_ELASTICSEARCH_HOST=https://ES.elastic-cloud.com
export ELASTIC_PACKAGE_ELASTICSEARCH_KIBANA_HOST=https://KIBANA.elastic-cloud.com
export ELASTIC_PACKAGE_KIBANA_HOST=https://KIBANA.elastic-cloud.com
export ELASTIC_PACKAGE_KIBANA_USERNAME=elastic
export ELASTIC_PACKAGE_KIBANA_PASSWORD=SECRET
export ELASTIC_PACKAGE_ELASTICSEARCH_USERNAME=elastic
export ELASTIC_PACKAGE_ELASTICSEARCH_PASSWORD=SECRET

• install the integration

cd packages/elastic_agent
elastic-package install

• check the dashboards show data about endpoint. See screenshots below

Related issues

• Closes [Elastic Agent]: Include endpoint-security in Agent CPU and Memory graphs #14272

For the reviewer

When editing the dashboards there was no way I could make elastic-package to export the dashboard without adding "typeMigrationVersion": "10.2.0" , what breaks the build on CI saying this version of kibana in unknown. So I manually edited it back to its original value

Screenshots

• to check the aggregated CPU and Memory maps include the endpoint data, edit the integration, then go to edit in lens (as shown below). There, choose "table" for the chart type. Then you'll see all the data like in the images below.

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[Copilot]

Pull Request Overview

This PR fixes the Elastic Agent metrics dashboard to include metrics from Elastic Defend (endpoint security) processes. The dashboard was previously only showing metrics from core Elastic Agent processes, but now includes CPU and memory metrics from both data streams.

• Added Elastic Defend endpoint security dataset to dashboard filters and visualizations
• Enhanced CPU calculation formulas to handle both time-based and percentage-based metrics
• Updated field definitions to include the new total.pct CPU metric field

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
packages/elastic_agent/manifest.yml	Version bump to 2.3.1 for the bug fix
packages/elastic_agent/kibana/dashboard/elastic_agent-f47f18cc-9c7d-4278-b2ea-a6dee816d395.json	Updated dashboard configuration to include endpoint security data and enhanced CPU calculations
packages/elastic_agent/data_stream/endpoint_security_metrics/fields/fields.yml	Added total.pct CPU field definition and reorganized existing CPU fields
packages/elastic_agent/changelog.yml	Added changelog entry for the dashboard fix

[Copilot]

The new field total.pct is being added in the middle of existing fields, but the system.ticks field that was previously at this location is moved to line 61. This creates a breaking change in field ordering that could affect existing queries or mappings. Consider adding new fields at the end of the section to maintain backward compatibility.

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 4c57c1e

History

• 💔 Build #28682 failed 3e1d4b0
• 💔 Build #28650 failed 056f0b6

cc @AndersonQ

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
96.4% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube