From 08eadcd8208892261cae782c240a7a1093aff5a5 Mon Sep 17 00:00:00 2001 From: Borja Gomez Date: Tue, 25 Nov 2025 10:53:20 +0100 Subject: [PATCH 1/6] Add job failure analysis for Azure Pipelines --- .../guides/use_ci_jobs_failure_analysis.md | 2 ++ .../pipelines/_index.md | 4 +-- .../continuous_integration/pipelines/azure.md | 27 +++++++++++++------ 3 files changed, 23 insertions(+), 10 deletions(-) diff --git a/content/en/continuous_integration/guides/use_ci_jobs_failure_analysis.md b/content/en/continuous_integration/guides/use_ci_jobs_failure_analysis.md index 55d8a19523b..783ea76fae5 100644 --- a/content/en/continuous_integration/guides/use_ci_jobs_failure_analysis.md +++ b/content/en/continuous_integration/guides/use_ci_jobs_failure_analysis.md @@ -94,6 +94,7 @@ CI jobs failure analysis is available for the following CI providers: * [GitHub Actions][1] * [GitLab][2] +* [Azure Pipeline][8] **Note:** You must enable CI job logs collection, and the logs need to be indexed. To set up CI job logs collection, select your CI provider on [Pipeline Visibility][6] and follow the instructions to collect job logs. @@ -147,3 +148,4 @@ For PR Comments to be posted, your repositories need to be integrated with Datad [5]:/dashboards/ [6]:/continuous_integration/pipelines/#setup [7]:/integrations/guide/source-code-integration/#connect-your-git-repositories-to-datadog +[8]:/continuous_integration/pipelines/azure/ diff --git a/content/en/continuous_integration/pipelines/_index.md b/content/en/continuous_integration/pipelines/_index.md index fdc10a3b2db..31ae1c30995 100644 --- a/content/en/continuous_integration/pipelines/_index.md +++ b/content/en/continuous_integration/pipelines/_index.md @@ -139,7 +139,7 @@ If your CI provider is not supported, you can try setting up Pipeline Visibility | | Jenkins | GitLab | CircleCI | Buildkite | GitHub Actions | Azure Pipelines | Codefresh | TeamCity | AWS CodePipeline | Other CI Providers | |---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:| | {{< ci-details title="Pipeline trace visualization" >}}Visualization of pipeline executions with associated tracing.{{< /ci-details >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | -| {{< ci-details title="Job failure analysis" >}}Analysis and categorization of job failures using LLM models based on relevant logs. More info.{{< /ci-details >}} | | {{< X >}} | | | {{< X >}} | | | | | | +| {{< ci-details title="Job failure analysis" >}}Analysis and categorization of job failures using LLM models based on relevant logs. More info.{{< /ci-details >}} | | {{< X >}} | | | {{< X >}} | {{< X >}} | | | | | | {{< ci-details title="Running pipelines" >}}Identification of pipelines executions that are running with associated tracing.{{< /ci-details >}} | {{< X >}} | {{< X >}} | | | {{< X >}} | | | | {{< X >}} | {{< X >}} | | {{< ci-details title="Critical path analysis" >}}Identification of CI jobs that are on the critical path of the pipeline. More info{{< /ci-details >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | | {{< ci-details title="Partial retries" >}}Identification of partial retries (for example, when only a subset of jobs were retried).{{< /ci-details >}} | | {{< X >}} | | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | @@ -176,4 +176,4 @@ You can export your search query to a [CI Pipeline monitor][12] on the [**Execut [11]: /notebooks [12]: /monitors/types/ci [13]: https://app.datadoghq.com/ci/test-runs -[14]: /continuous_integration/guides/use_ci_jobs_failure_analysis/ \ No newline at end of file +[14]: /continuous_integration/guides/use_ci_jobs_failure_analysis/ diff --git a/content/en/continuous_integration/pipelines/azure.md b/content/en/continuous_integration/pipelines/azure.md index 14bb7415519..0c5e6055c5c 100644 --- a/content/en/continuous_integration/pipelines/azure.md +++ b/content/en/continuous_integration/pipelines/azure.md @@ -26,12 +26,13 @@ Set up tracing on Azure Pipelines to gain real time insights into your CI/CD wor ### Compatibility -| Pipeline Visibility | Platform | Definition | -|---|---|---| +| Pipeline Visibility | Platform | Definition | +|-------------------------------------------------|---|---| +| [CI jobs failure analysis][21] | CI jobs failure analysis | Analysis of the root causes of failed CI jobs based on relevant logs using LLM models. | | [Custom tags][10] [and measures at runtime][11] | Custom tags and measures at runtime | Configure [custom tags and measures][6] at runtime. | -| [Custom spans][15] | Custom spans | Configure custom spans for your pipelines. | -| [Filter CI Jobs on the critical path][19] | Filter CI Jobs on the critical path | Filter by jobs on the critical path. | -| [Execution time][20] | Execution time | View the amount of time pipelines have been running jobs. | +| [Custom spans][15] | Custom spans | Configure custom spans for your pipelines. | +| [Filter CI Jobs on the critical path][19] | Filter CI Jobs on the critical path | Filter by jobs on the critical path. | +| [Execution time][20] | Execution time | View the amount of time pipelines have been running jobs. | ## Configure the Datadog integration @@ -119,15 +120,23 @@ To enable job log collection: 1. Install a Datadog app registration on your Azure console. Follow the steps in the [Azure integration tile][14]. -2. Add the Datadog app registration to your Azure DevOps organization: - a. Navigate to **Organization settings** in your DevOps console. - b. Click **Users** from the left side panel, then click on **Add Users**. +2. Add the Datadog app registration to your Azure DevOps organization: + a. Navigate to **Organization settings** in your DevOps console. + b. Click **Users** from the left side panel, then click on **Add Users**. **Note**: If you don't see the **Add Users** button, you may not have the necessary permissions. To enable log collection, add your app registration as a user with Basic Access Level to each project. Alternatively, you can click **Add to all projects** to configure all projects in bulk. Logs are billed separately from CI Visibility. Log retention, exclusion, and indexes are configured in [Log Management][18]. Logs for Azure jobs can be identified by the `datadog.product:cipipeline` and `source:azurepipelines` tags. +### CI jobs failure analysis + +If job logs collection is enabled, CI Visibility computes analysis using LLM models for failed CI jobs based on relevant logs coming from Azure Pipelines. + +You can also add job failure analysis to a PR comment. See the guide on [using PR comments][22]. + +For a full explanation, see the guide on [using CI jobs failure analysis][21]. + ## Visualize pipeline data in Datadog The [**CI Pipeline List**][4] and [**Executions**][5] pages populate with data after the workflows finish. @@ -157,3 +166,5 @@ The **CI Pipeline List** page shows data for only the default branch of each rep [18]: /logs/guide/best-practices-for-log-management/ [19]: /continuous_integration/guides/identify_highest_impact_jobs_with_critical_path/ [20]: /glossary/#pipeline-execution-time +[21]: /continuous_integration/guides/use_ci_jobs_failure_analysis/ +[22]: /continuous_integration/guides/use_ci_jobs_failure_analysis/#using-pr-comments From 07447c496edff7805243662e9537340bf60064c2 Mon Sep 17 00:00:00 2001 From: Borja Gomez Date: Wed, 26 Nov 2025 13:54:09 +0100 Subject: [PATCH 2/6] Apply suggestions --- .../pipelines/_index.md | 2 +- .../continuous_integration/pipelines/azure.md | 21 +++++++++---------- .../pipelines/github.md | 4 ++-- .../pipelines/gitlab.md | 4 ++-- 4 files changed, 15 insertions(+), 16 deletions(-) diff --git a/content/en/continuous_integration/pipelines/_index.md b/content/en/continuous_integration/pipelines/_index.md index 31ae1c30995..fb1a3aa6ec9 100644 --- a/content/en/continuous_integration/pipelines/_index.md +++ b/content/en/continuous_integration/pipelines/_index.md @@ -139,7 +139,7 @@ If your CI provider is not supported, you can try setting up Pipeline Visibility | | Jenkins | GitLab | CircleCI | Buildkite | GitHub Actions | Azure Pipelines | Codefresh | TeamCity | AWS CodePipeline | Other CI Providers | |---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:| | {{< ci-details title="Pipeline trace visualization" >}}Visualization of pipeline executions with associated tracing.{{< /ci-details >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | -| {{< ci-details title="Job failure analysis" >}}Analysis and categorization of job failures using LLM models based on relevant logs. More info.{{< /ci-details >}} | | {{< X >}} | | | {{< X >}} | {{< X >}} | | | | | +| {{< ci-details title="Job failure analysis" >}}Uses LLM models on relevant logs to analyze and categorize job failures. More info.{{< /ci-details >}} | | {{< X >}} | | | {{< X >}} | {{< X >}} | | | | | | {{< ci-details title="Running pipelines" >}}Identification of pipelines executions that are running with associated tracing.{{< /ci-details >}} | {{< X >}} | {{< X >}} | | | {{< X >}} | | | | {{< X >}} | {{< X >}} | | {{< ci-details title="Critical path analysis" >}}Identification of CI jobs that are on the critical path of the pipeline. More info{{< /ci-details >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | | {{< ci-details title="Partial retries" >}}Identification of partial retries (for example, when only a subset of jobs were retried).{{< /ci-details >}} | | {{< X >}} | | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | {{< X >}} | diff --git a/content/en/continuous_integration/pipelines/azure.md b/content/en/continuous_integration/pipelines/azure.md index 0c5e6055c5c..aaf4a933456 100644 --- a/content/en/continuous_integration/pipelines/azure.md +++ b/content/en/continuous_integration/pipelines/azure.md @@ -26,13 +26,12 @@ Set up tracing on Azure Pipelines to gain real time insights into your CI/CD wor ### Compatibility -| Pipeline Visibility | Platform | Definition | -|-------------------------------------------------|---|---| -| [CI jobs failure analysis][21] | CI jobs failure analysis | Analysis of the root causes of failed CI jobs based on relevant logs using LLM models. | -| [Custom tags][10] [and measures at runtime][11] | Custom tags and measures at runtime | Configure [custom tags and measures][6] at runtime. | -| [Custom spans][15] | Custom spans | Configure custom spans for your pipelines. | -| [Filter CI Jobs on the critical path][19] | Filter CI Jobs on the critical path | Filter by jobs on the critical path. | -| [Execution time][20] | Execution time | View the amount of time pipelines have been running jobs. | +| Pipeline Visibility | Platform | Definition | +|-------------------------------------------------|-------------------------------------|-----------------------------------------------------------| +| [Custom tags][10] [and measures at runtime][11] | Custom tags and measures at runtime | Configure [custom tags and measures][6] at runtime. | +| [Custom spans][15] | Custom spans | Configure custom spans for your pipelines. | +| [Filter CI Jobs on the critical path][19] | Filter CI Jobs on the critical path | Filter by jobs on the critical path. | +| [Execution time][20] | Execution time | View the amount of time pipelines have been running jobs. | ## Configure the Datadog integration @@ -121,9 +120,9 @@ To enable job log collection: 1. Install a Datadog app registration on your Azure console. Follow the steps in the [Azure integration tile][14]. 2. Add the Datadog app registration to your Azure DevOps organization: - a. Navigate to **Organization settings** in your DevOps console. - b. Click **Users** from the left side panel, then click on **Add Users**. - **Note**: If you don't see the **Add Users** button, you may not have the necessary permissions. + a. Navigate to **Organization settings** in your DevOps console. + b. Click **Users** from the left side panel, then click on **Add Users**. + **Note**: If you don't see the **Add Users** button, you may not have the necessary permissions. To enable log collection, add your app registration as a user with Basic Access Level to each project. Alternatively, you can click **Add to all projects** to configure all projects in bulk. @@ -131,7 +130,7 @@ Logs are billed separately from CI Visibility. Log retention, exclusion, and ind ### CI jobs failure analysis -If job logs collection is enabled, CI Visibility computes analysis using LLM models for failed CI jobs based on relevant logs coming from Azure Pipelines. +If job logs collection is enabled, CI Visibility uses LLM models to compute the analysis for failed CI jobs based on relevant logs coming from Azure Pipelines. You can also add job failure analysis to a PR comment. See the guide on [using PR comments][22]. diff --git a/content/en/continuous_integration/pipelines/github.md b/content/en/continuous_integration/pipelines/github.md index a40b1d88dde..0f10f122db1 100644 --- a/content/en/continuous_integration/pipelines/github.md +++ b/content/en/continuous_integration/pipelines/github.md @@ -28,7 +28,7 @@ Set up CI Visibility in GitHub Actions to track the execution of your workflows, | Pipeline Visibility | Platform | Definition | |---|---|---| | [Running pipelines][2] | Running pipelines | View pipeline executions that are running. Queued or waiting pipelines show with status "Running" on Datadog. | -| [CI jobs failure analysis][23] | CI jobs failure analysis | Analysis of the root causes of failed CI jobs based on relevant logs using LLM models. | +| [CI jobs failure analysis][23] | CI jobs failure analysis | Uses LLM models on relevant logs to analyze the root cause of failed CI jobs. | | [Partial retries][3] | Partial pipelines | View partially retried pipeline executions. | | Logs correlation | Logs correlation | Correlate pipeline and job spans to logs and enable [job log collection](#collect-job-logs). | | Infrastructure metric correlation | Infrastructure metric correlation | Correlate jobs to [infrastructure host metrics][4] for GitHub jobs. | @@ -106,7 +106,7 @@ To see the metrics, click on a job span in the trace view. A window opens with a ### CI jobs failure analysis -If job logs collection is enabled, CI Visibility computes analysis using LLM models for failed CI jobs based on relevant logs coming from GitHub Actions. +If job logs collection is enabled, CI Visibility uses LLM models to compute the analysis for failed CI jobs based on relevant logs coming from Azure Pipelines. You can also add job failure analysis to a PR comment. See the guide on [using PR comments][26]. diff --git a/content/en/continuous_integration/pipelines/gitlab.md b/content/en/continuous_integration/pipelines/gitlab.md index d489816bac6..ad2623406bf 100644 --- a/content/en/continuous_integration/pipelines/gitlab.md +++ b/content/en/continuous_integration/pipelines/gitlab.md @@ -25,7 +25,7 @@ Set up tracing in GitLab to collect data on your pipeline executions, analyze pe | Pipeline Visibility | Platform | Definition | |---|---|---| | [Running pipelines][24] | Running pipelines | View pipeline executions that are running. Queued or waiting pipelines show with status "Running" on Datadog. | -| [CI jobs failure analysis][28] | CI jobs failure analysis | Analysis of the root causes of failed CI jobs based on relevant logs using LLM models. | +| [CI jobs failure analysis][28] | CI jobs failure analysis | Uses LLM models on relevant logs to analyze the root cause of failed CI jobs. | | [Filter CI Jobs on the critical path][29] | Filter CI Jobs on the critical path | Filter by jobs on the critical path. | | [Partial retries][19] | Partial pipelines | View partially retried pipeline executions. | | [Manual steps][20] | Manual steps | View manually triggered pipelines. | @@ -303,7 +303,7 @@ For failed GitLab pipeline executions, each error under the `Errors` tab within #### CI jobs failure analysis -If job logs collection is enabled, CI Visibility computes analysis using LLM models for failed CI jobs based on relevant logs coming from GitLab. +If job logs collection is enabled, CI Visibility uses LLM models to compute the analysis for failed CI jobs based on relevant logs coming from Azure Pipelines. You can also add job failure analysis to a PR comment. See the guide on [using PR comments][30]. From d473e7b624ce8a9ef6151003c3b14be12aa00960 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Borja=20G=C3=B3mez?= Date: Wed, 26 Nov 2025 16:23:42 +0100 Subject: [PATCH 3/6] Update content/en/continuous_integration/pipelines/github.md Co-authored-by: May Lee --- content/en/continuous_integration/pipelines/github.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/continuous_integration/pipelines/github.md b/content/en/continuous_integration/pipelines/github.md index 0f10f122db1..79eb2e3b43e 100644 --- a/content/en/continuous_integration/pipelines/github.md +++ b/content/en/continuous_integration/pipelines/github.md @@ -106,7 +106,7 @@ To see the metrics, click on a job span in the trace view. A window opens with a ### CI jobs failure analysis -If job logs collection is enabled, CI Visibility uses LLM models to compute the analysis for failed CI jobs based on relevant logs coming from Azure Pipelines. +If job logs collection is enabled, CI Visibility uses LLM models to analyze failed CI jobs based on relevant logs coming from Azure Pipelines. You can also add job failure analysis to a PR comment. See the guide on [using PR comments][26]. From 37a073342c3c7b7a3f5566521341463dd257ec86 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Borja=20G=C3=B3mez?= Date: Wed, 26 Nov 2025 16:23:54 +0100 Subject: [PATCH 4/6] Update content/en/continuous_integration/pipelines/gitlab.md Co-authored-by: May Lee --- content/en/continuous_integration/pipelines/gitlab.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/continuous_integration/pipelines/gitlab.md b/content/en/continuous_integration/pipelines/gitlab.md index ad2623406bf..db6dfdf215f 100644 --- a/content/en/continuous_integration/pipelines/gitlab.md +++ b/content/en/continuous_integration/pipelines/gitlab.md @@ -303,7 +303,7 @@ For failed GitLab pipeline executions, each error under the `Errors` tab within #### CI jobs failure analysis -If job logs collection is enabled, CI Visibility uses LLM models to compute the analysis for failed CI jobs based on relevant logs coming from Azure Pipelines. +If job logs collection is enabled, CI Visibility uses LLM models to analyze failed CI jobs based on relevant logs coming from Azure Pipelines. You can also add job failure analysis to a PR comment. See the guide on [using PR comments][30]. From a4f1fde807f44da1e4cc670d6c16adc1f3c6aa0f Mon Sep 17 00:00:00 2001 From: Borja Gomez Date: Wed, 26 Nov 2025 16:27:58 +0100 Subject: [PATCH 5/6] Fix incorrect provider name # Conflicts: # content/en/continuous_integration/pipelines/github.md # content/en/continuous_integration/pipelines/gitlab.md --- content/en/continuous_integration/pipelines/github.md | 2 +- content/en/continuous_integration/pipelines/gitlab.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/content/en/continuous_integration/pipelines/github.md b/content/en/continuous_integration/pipelines/github.md index 79eb2e3b43e..1291e900dcd 100644 --- a/content/en/continuous_integration/pipelines/github.md +++ b/content/en/continuous_integration/pipelines/github.md @@ -106,7 +106,7 @@ To see the metrics, click on a job span in the trace view. A window opens with a ### CI jobs failure analysis -If job logs collection is enabled, CI Visibility uses LLM models to analyze failed CI jobs based on relevant logs coming from Azure Pipelines. +If job logs collection is enabled, CI Visibility uses LLM models to analyze failed CI jobs based on relevant logs coming from GitHub Actions. You can also add job failure analysis to a PR comment. See the guide on [using PR comments][26]. diff --git a/content/en/continuous_integration/pipelines/gitlab.md b/content/en/continuous_integration/pipelines/gitlab.md index db6dfdf215f..283e6cba766 100644 --- a/content/en/continuous_integration/pipelines/gitlab.md +++ b/content/en/continuous_integration/pipelines/gitlab.md @@ -303,7 +303,7 @@ For failed GitLab pipeline executions, each error under the `Errors` tab within #### CI jobs failure analysis -If job logs collection is enabled, CI Visibility uses LLM models to analyze failed CI jobs based on relevant logs coming from Azure Pipelines. +If job logs collection is enabled, CI Visibility uses LLM models to analyze failed CI jobs based on relevant logs coming from GitLab. You can also add job failure analysis to a PR comment. See the guide on [using PR comments][30]. From 6b8c808597d8761049a5a3505bba6543ca3364b4 Mon Sep 17 00:00:00 2001 From: May Lee Date: Wed, 26 Nov 2025 10:33:55 -0500 Subject: [PATCH 6/6] fix indentation --- content/en/continuous_integration/pipelines/azure.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/content/en/continuous_integration/pipelines/azure.md b/content/en/continuous_integration/pipelines/azure.md index aaf4a933456..4df0e4f98af 100644 --- a/content/en/continuous_integration/pipelines/azure.md +++ b/content/en/continuous_integration/pipelines/azure.md @@ -120,9 +120,8 @@ To enable job log collection: 1. Install a Datadog app registration on your Azure console. Follow the steps in the [Azure integration tile][14]. 2. Add the Datadog app registration to your Azure DevOps organization: - a. Navigate to **Organization settings** in your DevOps console. - b. Click **Users** from the left side panel, then click on **Add Users**. - **Note**: If you don't see the **Add Users** button, you may not have the necessary permissions. +
a. Navigate to **Organization settings** in your DevOps console. +
b. Click **Users** from the left side panel, then click **Add Users**.
**Note**: If you don't see the **Add Users** button, you may not have the necessary permissions. To enable log collection, add your app registration as a user with Basic Access Level to each project. Alternatively, you can click **Add to all projects** to configure all projects in bulk.