diff --git a/.gitignore b/.gitignore index f972f48..baf1a8e 100644 --- a/.gitignore +++ b/.gitignore @@ -6,3 +6,4 @@ /.gradle build/ +token.txt diff --git a/CHANGELOG.md b/CHANGELOG.md index 92c9160..6aec11c 100755 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,14 @@ All changes to the log analytics integration will be documented in this file. +## [1.1.0] - August 2025 + +* Upgrade application versions +* Create new dashboards for Artifactory and Xray +* Remove Artifactory HA example. It's not the recommended chart to install Artifactory +* Fix breaking configurations in Artifactory and Xray Helm installations +* Improve instructions + ## [1.0.12] - March 18, 2025 * FluentD sidecar image version bumped to 4.15, to upgrade base image to bitnami/fluentd 1.18 diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0c5037c..e4b6b00 100755 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -40,23 +40,24 @@ In the interest of fostering an open and welcoming environment, we as contributo ### Our Standards Examples of behavior that contributes to creating a positive environment include: - ```` +``` Using welcoming and inclusive language Being respectful of differing viewpoints and experiences Gracefully accepting constructive criticism Focusing on what is best for the company Showing empathy towards other colleagues - ```` +``` Examples of unacceptable behavior by participants include: - ```` +``` The use of sexualized language or imagery and unwelcome sexual attention or advances Trolling, insulting/derogatory comments, and personal or political attacks Public or private harassment Publishing others' private information, such as a physical or electronic address, without explicit permission Other conduct which could reasonably be considered inappropriate in a professional setting - ```` +``` + ### Our Responsibilities Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior. diff --git a/README.md b/README.md index 267cf39..c5e29ff 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# Prometheus + Loki + Grafana based Log Analytics and Metrics for JFrog Artifactory, Xray +# Prometheus, Loki and Grafana Log Collection and Metrics for JFrog Artifactory and Xray The JFrog Log Analytics and Metrics solution using Prometheus consists of three segments, @@ -8,39 +8,39 @@ The JFrog Log Analytics and Metrics solution using Prometheus consists of three ## Pre-Requisites -1. Working and configured Kubernetes Cluster - Amazon EKS / Google GKE / Azure AKS / Docker Desktop / Minikube - +1. A Kubernetes Cluster - Amazon EKS / Google GKE / Azure AKS / Docker Desktop / Minikube 1. Recommended Kubernetes Version 1.25.2 and above 2. For Google GKE, refer [GKE Guide](https://cloud.google.com/kubernetes-engine/docs/how-to) 3. For Amazon EKS, refer [EKS Guide](https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html) 4. For Azure AKS, refer [AKS Guide](https://docs.microsoft.com/en-us/azure/aks/) - 5. For Docker Desktop and Kubernetes, refer [DOCKER Guide](https://docs.docker.com/desktop/kubernetes/) -2. 'kubectl' utility on the workstation which is capable of connecting to the Kubernetes cluster - - 1. For Installation and usage refer [KUBECTL Guide](https://kubernetes.io/docs/tasks/tools/) -3. HELM v3 Installed - - 1. For Installation and usage refer [HELM Guide](https://helm.sh/docs/intro/install/) -4. Versions supported and Tested: + 5. For Docker Desktop and Kubernetes, refer [Docker Guide](https://docs.docker.com/desktop/kubernetes/) - Artifactory: 7.104.7 +2. `kubectl` configured to the Kubernetes cluster + 1. For Installation and usage refer [kubectl setup](https://kubernetes.io/docs/tasks/tools/#kubectl) - Xray: 3.103.6 +3. `helm` v3 + 1. For Installation and usage refer [helm setup](https://helm.sh/docs/intro/install/) - Prometheus: 2.51.0 +4. Versions supported and Tested: + 1. Artifactory: 7.117.x + 2. Xray: 3.124.x + 3. Prometheus: 3.5.x + 4. Grafana: 12.0.x + 5. Loki: 3.5.x - Grafana: 10.4.0 +## Known Limitations - Loki: 2.9.6 +Some known limitations we are aware of +- The stack does not install well on a [GKE Autopilot](https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview) due to permissions needed -## Read me before installing +## Read This Before Installing -### Important Note: This version replaces all previous implementations. This version is not an in-place upgrade to the existing solution from JFrog but is a full reinstall. Any dashboard customizations done on previous versions will need to be redone after this install. +### Important Note: This version replaces all previous implementations. This version is not an in-place upgrade to the existing solution from JFrog but is a full reinstall. Any dashboard customizations done on previous versions will need to be redone. -```html -This guide assumes the implementer is performing new setup, Changes to handle install in an existing setup will be highlighted where applicable. - if prometheus is already installed and configured, we recommend to have the existing prometheus release name handy. - If Loki is already installed and configured, we recommend to have its service URL handy. +``` +This guide assumes the implementer is performing new setup. Changes to handle install in an existing setup will be highlighted where applicable. +If Prometheus is already installed and configured, we recommend to have the existing Prometheus release name handy. +If Loki is already installed and configured, we recommend to have its service URL handy. ``` If Prometheus and Loki are already available you can skip the installation section and proceed to [Configuration Section](#Configuration). @@ -51,291 +51,228 @@ If Prometheus and Loki are already available you can skip the installation secti # Installation -## Installing Prometheus, Loki and Grafana on Kubernetes +## Installing Prometheus, Grafana and Loki The Prometheus Community [kube-prometheus-stack](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack) helm chart allows the creation of Prometheus instances and includes Grafana. The Grafana Community [grafana](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack) helm chart allows the creation of Loki instances and includes Grafana which can link to prometheus. Once the Pre-Requisites are met, to install Prometheus Kubernetes stack: -#### Create the namespace required for the kubernetes deployment - -```bash -export INST_NAMESPACE=jfrog-plg -``` +1. Create the namespaces required for the Kubernetes deployments + 1. We use the `jfrog` namespace for the JFrog applications + 2. We use the `monitoring` namespace for the observability tools -We will use `jfrog-plg` as the namespace throughout this document. That said, you can use a different or existing namespace instead by setting the above variable. +```shell +export JFROG_NAMESPACE=jfrog +kubectl create namespace ${JFROG_NAMESPACE} -```bash -kubectl create namespace $INST_NAMESPACE -kubectl config set-context --current --namespace=$INST_NAMESPACE +export OBS_NAMESPACE=monitoring +kubectl create namespace ${OBS_NAMESPACE} ``` -#### Install the Prometheus chart - -Note: This installation comes with a Grafana installation +Note: The `monitoring` namespace is also used in the Loki configuration in [artifactory-values.yaml](helm/artifactory-values.yaml) and [xray-values.yaml](helm/xray-values.yaml). If you decide to change it, make sure to update these files (the `LOKI_URL` variable). -Add the required Helm Repositories: +2. Install Prometheus and Grafana -```bash +```shell +# Add the required Helm repository helm repo add prometheus-community https://prometheus-community.github.io/helm-charts helm repo update ``` -Install the chart: +```shell +# Install the kube-prometheus-stack chart +helm upgrade --install prometheus --values helm/prometheus-grafana-values.yaml prometheus-community/kube-prometheus-stack -n ${OBS_NAMESPACE} -```bash -helm upgrade --install "prometheus" prometheus-community/kube-prometheus-stack -n $INST_NAMESPACE +# Might need to add --set prometheus.prometheusSpec.maximumStartupDurationSeconds=600 to avoid an error (bug?) ``` -```yaml -* "prometheus" here is the value that needs to be used against the value for "release_name" in the configuration section -``` +3. For Docker Desktop -For Docker Desktop, run this additional command to correct the mount path propagation for prometheus node-exporter component, -An error event will be appearing as follows "Error: failed to start container "node-exporter": Error response from daemon: path / is mounted on / but it is not a shared or slave mount" + Run this additional command to correct the mount path propagation for prometheus node-exporter component. -```bash -kubectl patch ds prometheus-prometheus-node-exporter --type "json" -p '[{"op": "remove", "path" : "/spec/template/spec/containers/0/volumeMounts/2/mountPropagation"}]' -n $INST_NAMESPACE -``` + An error event will be appearing as follows "Error: failed to start container "node-exporter": Error response from daemon: path / is mounted on / but it is not a shared or slave mount" -#### Install the Loki chart +```shell +kubectl patch ds prometheus-prometheus-node-exporter --type json -p '[{"op": "remove", "path" : "/spec/template/spec/containers/0/volumeMounts/2/mountPropagation"}]' -n ${OBS_NAMESPACE} +``` -Add the required Helm Repositories: +4. Install Loki -```bash +```shell +# Add the required Helm repository helm repo add grafana https://grafana.github.io/helm-charts helm repo update ``` -Install the chart: - -```bash -helm upgrade --install "loki" --values helm/loki-values.yaml grafana/loki --version 5.48.0 -n $INST_NAMESPACE +```shell +# Install the Loki chart +helm upgrade --install loki --values helm/loki-values.yaml grafana/loki -n ${OBS_NAMESPACE} ``` -:bulb: The above helm command is hard-coding the `loki` chart version to 5.48.0, since we only tested it with `loki` 2.9.6. `loki` 3.x charts (v6.0.x and up) have a breaking change, so if you would like to install `loki` 3.x please visit the [loki's official docs](https://grafana.com/docs/loki/latest/setup/install/helm/) and provide your own `loki-values.yaml` +## Install Artifactory with Open Metrics -```yaml -* "loki" will be the service name, the url to access loki as a datasource can be visualised as http://.: - ex: http://loki.$INST_NAMESPACE:3100 will be the "loki_url" value +### Artifactory +Installing Artifactory using the official [Helm Chart](https://github.com/jfrog/charts/tree/master/stable/artifactory) -* version 2.9.6 is the most recent loki version at the time of writing the document - if there is a need to deploy this exact version, change the version value in "--set loki.image.tag=my_desired_version" to your desired version. +1. Before starting the Artifactory installation generate a join and master keys -``` - -## Artifactory / Artifactory HA + Metrics via Helm ⎈ - -Before starting Artifactory or Artifactory HA installtion generate join and master keys for the installation: - -```bash +```shell export JOIN_KEY=$(openssl rand -hex 32) export MASTER_KEY=$(openssl rand -hex 32) ``` -Then helm install the Artifactory or Artifactory HA charts: - -Artifactory ⎈: +2. Install Artifactory (using the generated join and master keys) -1. helm install `artifactory` chart (using the above generated join and master keys). - -```bash +```shell +# Install Artifactory helm upgrade --install artifactory jfrog/artifactory \ - --set artifactory.masterKey=$MASTER_KEY \ - --set artifactory.joinKey=$JOIN_KEY \ - -n $INST_NAMESPACE + --set artifactory.masterKey=${MASTER_KEY} \ + --set artifactory.joinKey=${JOIN_KEY} \ + --set artifactory.metrics.enabled=true \ + -n ${JFROG_NAMESPACE} ``` -**If you are installing in the same cluster with the deprecated solution, use the same namespace as the previous one instead of jfrog-plg above.** +:bulb: Open Metrics is disabled by default in Artifactory. It's enabled by setting `artifactory.metrics.enabled=true`. -:bulb: Metrics collection is disabled by default in Artifactory. Please make sure that you are enabling them in Artifactory by setting `artifactory.metrics.enabled` to `true` in your [helm values file](helm/artifactory-values.yaml). For Artifactory versions <=7.86.x, please instead set the flag `artifactory.openMetrics.enabled` to `true +3. Follow the instructions how to get your new Artifactory URL from the helm install output -2. Follow the instructions how to get your new Artifactory URL from the helm install output +```shell +export SERVICE_IP=$(kubectl get svc --namespace ${JFROG_NAMESPACE} artifactory-artifactory-nginx -o jsonpath='{.status.loadBalancer.ingress[0].ip}') -```bash -export SERVICE_IP=$(kubectl get svc --namespace $INST_NAMESPACE artifactory-artifactory-nginx -o jsonpath='{.status.loadBalancer.ingress[0].ip}') -echo http://$SERVICE_IP/ +echo ${SERVICE_IP} +echo "http://${SERVICE_IP}/" ``` + OR +```shell +export SERVICE_IP=$(kubectl get svc --namespace ${JFROG_NAMESPACE} artifactory-artifactory-nginx -o jsonpath='{.status.loadBalancer.ingress[0].hostname}') -3. Using the Artifactory UI generate JFrog's admin [Access Token](https://jfrog.com/help/r/how-to-generate-an-access-token-video/artifactory-creating-access-tokens-in-artifactory). Using that fetched token, create a kubernetes generic secret for JFrog's admin token - using any of the following methods - -```bash -kubectl create secret generic jfrog-admin-token --from-file=token= +echo ${SERVICE_IP} +echo "http://${SERVICE_IP}/" ``` -OR - -```bash -kubectl create secret generic jfrog-admin-token --from-literal=token= - -``` - -4. Postgres password is required to upgrade Artifactory. Run the following command to get the current Postgres password - - ```bash - POSTGRES_PASSWORD=$(kubectl get secret artifactory-postgresql -o jsonpath="{.data.postgresql-password}" | base64 --decode) - ``` -5. Upgrade Artifactory and apply Helm chart to create additional kubernetes resources, which are required for Prometheus service discovery process: - - ```bash - helm upgrade --install artifactory jfrog/artifactory \ - --set artifactory.joinKey=$JOIN_KEY \ - --set databaseUpgradeReady=true --set postgresql.postgresqlPassword=$POSTGRES_PASSWORD \ - -f helm/artifactory-values.yaml \ - -n $INST_NAMESPACE - ``` +4. Browse to the URL above and login to Artifactory with the default credentials: `admin`/`password` + 1. Follow initial setup wizard + 2. You will need to enter a valid Artifactory license. If needed, get a free trial license from [here](https://jfrog.com/start-free/) -**💡Note: You need to be at the root of this repository folder to have `helm/artifactory-values.yaml` file available for the following command** +5. In the Artifactory UI, go to "Administration" -> "User Management" -> "Access Tokens" and generate an [admin access token](https://jfrog.com/help/r/how-to-generate-an-access-token-video/artifactory-creating-access-tokens-in-artifactory). Using the generated token, create a Kubernetes generic secret for the token - using one of the following methods -This will complete the necessary configuration for Artifactory and expose new service monitors `servicemonitor-artifactory` and `servicemonitor-observability` to expose metrics to Prometheus - -Artifactory-HA ⎈: - -1. helm install `artifactory-ha` chart (using the above generated join and master keys). - -```bash -helm upgrade --install artifactory jfrog/artifactory-ha \ - --set artifactory.masterKey=$MASTER_KEY \ - --set artifactory.joinKey=$JOIN_KEY \ - -n $INST_NAMESPACE +```shell +kubectl create secret generic jfrog-admin-token --from-file=token= -n ${JFROG_NAMESPACE} ``` - -**If you are installing in the same cluster with the deprecated solution, use the same namespace as the previous one instead of jfrog-plg above.** - -:bulb: Metrics collection is disabled by default in Artifactory-HA. Please make sure that you are enabling them in Artifactory-HA by setting `artifactory.metrics.enabled` to `true` in your [helm values file](helm/artifactory-ha-values.yaml). For Artifactory versions <=7.86.x, please instead set the flag `artifactory.openMetrics.enabled` to `true - -2. Follow the instructions how to get your new Artifactory URL from the helm install output - -```bash -export SERVICE_IP=$(kubectl get svc --namespace $INST_NAMESPACE artifactory-artifactory-nginx -o jsonpath='{.status.loadBalancer.ingress[0].ip}') -echo http://$SERVICE_IP/ +OR +```shell +kubectl create secret generic jfrog-admin-token --from-literal=token= -n ${JFROG_NAMESPACE} ``` -3. Using the Artifactory UI generate JFrog's admin [Access Token](https://jfrog.com/help/r/how-to-generate-an-access-token-video/artifactory-creating-access-tokens-in-artifactory). Using that fetched token, create a kubernetes generic secret for JFrog's admin token - using any of the following methods - -```bash -kubectl create secret generic jfrog-admin-token --from-file=token= +6. The PostgreSQL password is required for Artifactory upgrade. Run the following command to get the current PostgreSQL password +```shell +export POSTGRES_PASSWORD=$(kubectl get secret -n ${JFROG_NAMESPACE} artifactory-postgresql -o jsonpath="{.data.postgres-password}" | base64 --decode) +echo ${POSTGRES_PASSWORD} ``` -OR - -```bash -kubectl create secret generic jfrog-admin-token --from-literal=token= +7. Upgrade Artifactory with the custom values in [helm/artifactory-values.yaml](helm/artifactory-values.yaml) to create additional Kubernetes resources, which are required for the Prometheus service discovery process. +```shell +# Upgrade Artifactory +helm upgrade --install artifactory jfrog/artifactory \ + --set artifactory.joinKey=${JOIN_KEY} \ + --set databaseUpgradeReady=true --set postgresql.auth.password=${POSTGRES_PASSWORD} \ + -f helm/artifactory-values.yaml \ + -n ${JFROG_NAMESPACE} ``` -4. Postgres password is required to upgrade Artifactory. Run the following command to get the current Postgres password - - ```bash - POSTGRES_PASSWORD=$(kubectl get secret artifactory-postgresql -o jsonpath="{.data.postgresql-password}" | base64 --decode) - ``` -5. Upgrade Artifactory and helm upgrade the `artifactory-ha` chart to create additional kubernetes resources, which are required for Prometheus service discovery process: - - You need to be at the root of this repository folder to have `helm/artifactory-ha-values.yaml` file available for the following command: - - ```bash - helm upgrade --install artifactory jfrog/artifactory-ha \ - --set artifactory.joinKey=$JOIN_KEY \ - --set databaseUpgradeReady=true --set postgresql.postgresqlPassword=$POSTGRES_PASSWORD \ - -f helm/artifactory-ha-values.yaml \ - -n $INST_NAMESPACE - ``` - -**💡Note: The above examples are only references you will need additional parameters to configure TLS, binary blob storage, or other common Artifactory features.** - -This will complete the necessary configuration for Artifactory-HA and expose new service monitors `servicemonitor-artifactory-ha` and `servicemonitor-observability` to expose metrics to Prometheus +This will complete the necessary configuration for Artifactory and expose new service monitors `servicemonitor-artifactory` and `servicemonitor-observability` to expose metrics to Prometheus -## Xray + Metrics via Helm ⎈ +## Install Xray with Open Metrics To configure and install Xray with Prometheus metrics being exposed use our file `helm/xray-values.yaml` to expose a metrics and new service monitor to Prometheus. -Xray ⎈: +### Xray -Generate master keys for the Xray installation: +1. Generate a master key for the Xray installation: -````bash +```shell export XRAY_MASTER_KEY=$(openssl rand -hex 32) -```` +``` -Use the same `JOIN_KEY` from the Artifactory installation, in order to connect Xray to Artifactory. You'll also be using the `jfrog-admin-token` kubernetes secret, that was created early as part of Artifactory/Artifactory-HA installation +2. Use the same `JOIN_KEY` from the Artifactory installation, in order to connect Xray to Artifactory. You'll also be using the `jfrog-admin-token` kubernetes secret, that was created earlier as part of Artifactory installation -**💡Note: You need to be at the root of this repository folder to have `helm/xray-values.yaml` file available for the following command** + Getting the Artifactory URL: -```bash -# getting Artifactory URL -export JFROG_JPD=$(kubectl get svc -n $INST_NAMESPACE artifactory-artifactory-nginx -o jsonpath='{.status.loadBalancer.ingress[0].ip}') +```shell +export JFROG_URL=$(kubectl get svc -n ${JFROG_NAMESPACE} artifactory-artifactory-nginx -o jsonpath='{.status.loadBalancer.ingress[0].ip}') -# helm install xray -helm upgrade --install xray jfrog/xray --set xray.jfrogUrl=http://$JFROG_JPD \ - --set xray.masterKey=$XRAY_MASTER_KEY \ - --set xray.joinKey=$JOIN_KEY \ - -f helm/xray-values.yaml \ - -n $INST_NAMESPACE +echo "http://${JFROG_URL}" ``` -**If you are installing in the same cluster with the deprecated solution, Use the same namespace as the previous one instead of jfrog-plg above.** + OR +```shell +export JFROG_URL=$(kubectl get svc -n ${JFROG_NAMESPACE} artifactory-artifactory-nginx -o jsonpath='{.status.loadBalancer.ingress[0].hostname}') -# Configuration +echo "http://${JFROG_URL}" +``` -## To Assess the setup for Prometheus + Install Xray +```shell +helm upgrade --install xray jfrog/xray --set xray.jfrogUrl=http://${JFROG_URL} \ + --set xray.masterKey=${XRAY_MASTER_KEY} \ + --set xray.joinKey=${JOIN_KEY} \ + -f helm/xray-values.yaml \ + -n ${JFROG_NAMESPACE} +``` -Use 'kubectl port forward' as mentioned below in a separate terminal window +# Configuration -```bash - kubectl port-forward service/prometheus-operated 9090:9090 -n $INST_NAMESPACE -``` +## Access the Prometheus UI -Go to the web UI of the Prometheus instance "http://localhost:9090" and verify "Status -> Service Discovery", the list shows the new ServiceMonitor for Artifactory or Xray or both, as shown here:![targets](images/ServiceDiscovery.jpeg) +Use `kubectl port-forward` as mentioned below in a separate terminal window -```plaintext -Deafult user/password for Prometheus is -> UNAME/PASSWD +```shell +kubectl port-forward service/prometheus-operated 9090:9090 -n ${OBS_NAMESPACE} ``` -## To Assess the setup for Grafana +Go to the web UI of the Prometheus instance http://localhost:9090 and verify "Status -> Service Discovery", the list shows all the `serviceMonitor`s. -use 'kubectl port forward' as mentioned below in a separate terminal window +Search for `servicemonitor-artifactory` and `servicemonitor-xray` to confirm they are successfully picked up by Prometheus. -```bash - kubectl port-forward service/prometheus-grafana 3000:80 -n $INST_NAMESPACE +## Complete the Grafana Setup -``` +Use `kubectl port-forward` as mentioned below in a separate terminal window -1. Open your Grafana on a browser at "http://localhost:3000" (grafana default credentials are "admin" / "prom-operator"). -2. Go to "Configuratoin" -> "Data Sources" on the left menu: +```shell +kubectl port-forward service/prometheus-grafana 3000:80 -n ${OBS_NAMESPACE} +``` - ![datasource](images/DataSourceAddition-01.jpeg) -3. Add your Prometheus instance and Loki Instance as datasources: +1. Open your Grafana on a browser at http://localhost:3000. Grafana default credentials are `admin/prom-operator` (set in [prometheus-grafana-values.yaml](helm/prometheus-grafana-values.yaml)). - * When adding Loki data source, specify url value as `http://loki:3100` - * Prometheus data dource might be added from config automatically. If not, add Prometheus data dource and specify url value as `http://prometheus-kube-prometheus-prometheus:9090/` - ![datasource](images/DataSourceAddition-04.jpeg) -4. While specifying datasource url for `Loki` and `Prometheus`, please test and confirm that the connection is successful using the `Save & Test` button at the bottom of the adding data source page: +2. Go to "Data sources" on the sidebar menu - ![datasource](images/DataSourceAddition-03.jpeg) +3. Click `Add new data source` + 1. Add your Prometheus as datasources (if not already configured): Set "Prometheus server URL" to `http://prometheus-kube-prometheus-prometheus:9090/` + 2. Add your Loki as datasources: Set "URL" to `http://loki:3100` - After adding both `Loki` and `Prometheus` Data Sources your "Configuration" -> "Data Sources" page should look like the following: - ![datasource](images/DataSourceAddition-02.jpeg) +4. When adding the `Loki` and `Prometheus` datasources, click `Save & Test` button at the bottom to validate connection to services is successful -## Grafana Dashboard +## Artifactory and Xray Grafana Dashboards -Example dashboards are included in the [grafana directory](grafana). These dashboards needs to be imported to the grafana. These include: +Example dashboards are included in the [grafana](grafana) directory. These dashboards need to be imported to Grafana. These include: -- Artifactory Metrics and Log Analytics Dashboard [Download Here](grafana/ArtifactoryLogAnalyticsAndSystemMetrics.json) -- Xray Metrics and Log Analytics Dashboard [Download Here](grafana/XrayLogAnalyticsAndSystemMetrics.json) +- Artifactory Application Metrics (Open Metrics) Dashboard [Download Here](grafana/ArtifactoryMetrics.json) +- Xray Application Metrics (Open Metrics) Dashboard [Download Here](grafana/XrayMetrics.json) -After downlowding the dashboards go to "Dashboards" -> "Import": +1. After downloading the dashboards go to "Dashboards" -> "New" -> "Import" -![dashboards](images/DashboardAddition-01.jpeg) +2. Pick `Upload dashboard JSON file` and upload Artifactory and Xray dashboards files that you downloaded in the previous step -Pick `Uplaod JSON file` and upload Artifactory and Xray dashboards files that you downloaded in the previous step. +## Artifactory and Xray Logs in Grafana -Import the Dashboards and select the appropriate sources (Loki and Prometheus) +If you have Loki configured as a Grafana datasource, you will see a `Logs` link on the sidebar menu. Click it and you should see the Artifactory and Xray services with a snippet of their logs. -![dashboards](images/DashboardAddition-02.jpeg) +Click the `Show logs` on any of these services, and you can now see all the service (Artifactory or Xray) logs in Grafana and start searching and filtering through them. ## References * [Grafana Dashboards](https://grafana.com/docs/grafana/latest/features/dashboard/dashboards/) -* [Grafana Queries](https://prometheus.io/docs/prometheus/latest/querying/basics/) +* [Prometheus Queries](https://prometheus.io/docs/prometheus/latest/querying/basics/) * [Loki Queries](https://grafana.com/docs/loki/latest/query/) diff --git a/grafana/ArtifactoryMetrics.json b/grafana/ArtifactoryMetrics.json new file mode 100644 index 0000000..d1d5f16 --- /dev/null +++ b/grafana/ArtifactoryMetrics.json @@ -0,0 +1,3973 @@ +{ + "annotations": { + "list": [ + { + "builtIn": 1, + "datasource": { + "type": "datasource", + "uid": "grafana" + }, + "enable": true, + "hide": true, + "iconColor": "rgba(0, 211, 255, 1)", + "name": "Annotations & Alerts", + "target": { + "limit": 100, + "matchAny": false, + "tags": [], + "type": "dashboard" + }, + "type": "dashboard" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "enable": true, + "hide": false, + "iconColor": "red", + "name": "Annotate restarts", + "tagKeys": "restart", + "target": { + "expr": "process_start_time_seconds{application=\"Artifactory\", namespace=\"$namespace\", pod=~\"$pod\"} * 1000", + "interval": "", + "refId": "Anno" + }, + "textFormat": "Artifactory Restart", + "titleFormat": "Restart {{pod}}", + "useValueForTime": "on" + } + ] + }, + "editable": true, + "fiscalYearStartMonth": 0, + "graphTooltip": 0, + "id": 29, + "links": [ + { + "asDropdown": false, + "icon": "external link", + "includeVars": false, + "keepTime": false, + "tags": [], + "targetBlank": false, + "title": "System Metrics", + "tooltip": "", + "type": "link", + "url": "/d/6581e46e4e5c7ba40a07646395ef7b23/kubernetes-compute-resources-pod" + } + ], + "panels": [ + { + "collapsed": false, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 0 + }, + "id": 2, + "panels": [], + "title": "General Information", + "type": "row" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 4, + "w": 6, + "x": 0, + "y": 1 + }, + "id": 181, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "percentChangeColorMode": "standard", + "reduceOptions": { + "calcs": [ + "mean" + ], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "name", + "wideLayout": true + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "editorMode": "code", + "exemplar": false, + "expr": "topk(1,label_replace(kube_pod_container_info{namespace=\"$namespace\",container=\"artifactory\"}, \"version\", \"$1\", \"image\", \".+:(.+)\"))", + "instant": true, + "legendFormat": "{{version}}", + "range": false, + "refId": "A" + } + ], + "title": "Artifactory Version", + "type": "stat" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "Shows the total number of artifacts stored in all repositories.", + "fieldConfig": { + "defaults": { + "color": { + "fixedColor": "blue", + "mode": "fixed" + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "gridPos": { + "h": 2, + "w": 5, + "x": 6, + "y": 1 + }, + "id": 24, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "percentChangeColorMode": "standard", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "auto", + "wideLayout": true + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "exemplar": false, + "expr": "max(jfrt_stg_summary_files_total{namespace=\"$namespace\", pod=~\"$pod\"})", + "instant": true, + "legendFormat": "__auto", + "range": false, + "refId": "A" + } + ], + "title": "Artifacts Count", + "type": "stat" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "Shows the total size of all artifacts stored in Artifactory.\n\nNote: The information about storage is updated infrequently (once in 6 hours), so the number that you see here is just an approximation.", + "fieldConfig": { + "defaults": { + "color": { + "fixedColor": "blue", + "mode": "fixed" + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "bytes" + }, + "overrides": [] + }, + "gridPos": { + "h": 2, + "w": 5, + "x": 11, + "y": 1 + }, + "id": 26, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "percentChangeColorMode": "standard", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "auto", + "wideLayout": true + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "exemplar": false, + "expr": "max(sum(jfrt_stg_summary_size_bytes{namespace=\"$namespace\", pod=~\"$pod\"}) by (pod))", + "instant": true, + "legendFormat": "__auto", + "range": false, + "refId": "A" + } + ], + "title": "Total Artifacts Size", + "type": "stat" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "fixedColor": "blue", + "mode": "thresholds" + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "#EAB839", + "value": 10000 + }, + { + "color": "orange", + "value": 100000 + }, + { + "color": "red", + "value": 1000000 + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "gridPos": { + "h": 2, + "w": 5, + "x": 16, + "y": 1 + }, + "id": 30, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "percentChangeColorMode": "standard", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "auto", + "wideLayout": true + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "exemplar": false, + "expr": "max(sum(jfrt_stg_summary_repo_files_total{namespace=\"$namespace\", pod=~\"$pod\", type=\"NA\"}) by (pod))", + "instant": true, + "legendFormat": "__auto", + "range": false, + "refId": "A" + } + ], + "title": "Files in Trash", + "type": "stat" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "Shows the total number of binaries stored in Artifactory (data from the last checkpoint).", + "fieldConfig": { + "defaults": { + "color": { + "fixedColor": "blue", + "mode": "fixed" + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "gridPos": { + "h": 2, + "w": 5, + "x": 6, + "y": 3 + }, + "id": 20, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "percentChangeColorMode": "standard", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "auto", + "wideLayout": true + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "exemplar": false, + "expr": "max(jfrt_stg_summary_checkpoint_binaries_total{namespace=\"$namespace\", pod=~\"$pod\"})", + "instant": true, + "legendFormat": "__auto", + "range": false, + "refId": "A" + } + ], + "title": "Total Number of Binaries", + "type": "stat" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "Shows the total number of bytes used by Artifactory storage (information is from the last checkpoint)", + "fieldConfig": { + "defaults": { + "color": { + "fixedColor": "blue", + "mode": "fixed" + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + } + ] + }, + "unit": "bytes" + }, + "overrides": [] + }, + "gridPos": { + "h": 2, + "w": 5, + "x": 11, + "y": 3 + }, + "id": 22, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "percentChangeColorMode": "standard", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "auto", + "wideLayout": true + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "exemplar": false, + "expr": "max(jfrt_stg_summary_checkpoint_binaries_size_bytes{namespace=\"$namespace\", pod=~\"$pod\"})", + "instant": true, + "legendFormat": "__auto", + "range": false, + "refId": "A" + } + ], + "title": "Total Used Storage", + "type": "stat" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "Shows the total size of all artifacts in the trash", + "fieldConfig": { + "defaults": { + "color": { + "fixedColor": "blue", + "mode": "thresholds" + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "yellow", + "value": 10000000000 + }, + { + "color": "orange", + "value": 100000000000 + }, + { + "color": "red", + "value": 1000000000000 + } + ] + }, + "unit": "bytes" + }, + "overrides": [] + }, + "gridPos": { + "h": 2, + "w": 5, + "x": 16, + "y": 3 + }, + "id": 28, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "percentChangeColorMode": "standard", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "auto", + "wideLayout": true + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "exemplar": false, + "expr": "max(sum(jfrt_stg_summary_repo_size_bytes{namespace=\"$namespace\", pod=~\"$pod\", type=\"NA\"}) by (pod))", + "instant": true, + "legendFormat": "__auto", + "range": false, + "refId": "A" + } + ], + "title": "Total Trash Size", + "type": "stat" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "The uptime of the JVM - pod $pod", + "fieldConfig": { + "defaults": { + "color": { + "fixedColor": "blue", + "mode": "thresholds" + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "red" + }, + { + "color": "yellow", + "value": 0 + }, + { + "color": "blue", + "value": 3600 + } + ] + }, + "unit": "s" + }, + "overrides": [] + }, + "gridPos": { + "h": 2, + "w": 6, + "x": 0, + "y": 5 + }, + "id": 40, + "options": { + "colorMode": "value", + "graphMode": "none", + "justifyMode": "auto", + "orientation": "auto", + "percentChangeColorMode": "standard", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "auto", + "wideLayout": true + }, + "pluginVersion": "12.0.2", + "repeat": "pod", + "repeatDirection": "v", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "exemplar": false, + "expr": "process_uptime_seconds{application=\"Artifactory\", namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\"}", + "instant": true, + "legendFormat": "{{pod}}", + "range": false, + "refId": "A" + } + ], + "title": "Uptime - $pod", + "type": "stat" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "Shows the total number of repositories in Artifactory", + "fieldConfig": { + "defaults": { + "color": { + "fixedColor": "blue", + "mode": "fixed" + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + } + ] + }, + "unit": "locale" + }, + "overrides": [] + }, + "gridPos": { + "h": 2, + "w": 5, + "x": 6, + "y": 5 + }, + "id": 16, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "percentChangeColorMode": "standard", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "auto", + "wideLayout": true + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "exemplar": false, + "expr": "max(sum(jfrt_repo_by_type_total{namespace=\"$namespace\", pod=~\"$pod\"}) by (pod))", + "instant": true, + "legendFormat": "__auto", + "range": false, + "refId": "A" + } + ], + "title": "Repositories", + "type": "stat" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "Shows the total number of package types that are used in repositories of all customers", + "fieldConfig": { + "defaults": { + "color": { + "fixedColor": "blue", + "mode": "fixed" + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 2, + "w": 5, + "x": 11, + "y": 5 + }, + "id": 18, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "percentChangeColorMode": "standard", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "auto", + "wideLayout": true + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "exemplar": false, + "expr": "count(max(jfrt_repo_by_type_total{namespace=\"$namespace\", pod=~\"$pod\"}) by (package))", + "instant": true, + "legendFormat": "__auto", + "range": false, + "refId": "A" + } + ], + "title": "Package Types", + "type": "stat" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "Shows the total number of federated repositories in the selected JPD", + "fieldConfig": { + "defaults": { + "color": { + "fixedColor": "blue", + "mode": "fixed" + }, + "mappings": [], + "noValue": "0", + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + } + ] + }, + "unit": "locale" + }, + "overrides": [] + }, + "gridPos": { + "h": 2, + "w": 5, + "x": 16, + "y": 5 + }, + "id": 72, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "percentChangeColorMode": "standard", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "auto", + "wideLayout": true + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "exemplar": false, + "expr": "max(sum(jfrt_repo_by_type_total{namespace=\"$namespace\", pod=~\"$pod\", type=\"Federated\"}) by (pod))", + "instant": true, + "legendFormat": "__auto", + "range": false, + "refId": "A" + } + ], + "title": "Federated Repositories", + "type": "stat" + }, + { + "collapsed": true, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 7 + }, + "id": 46, + "panels": [ + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "Artifactory container CPU usage", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "dashed+area" + } + }, + "links": [], + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "transparent" + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 9, + "w": 12, + "x": 0, + "y": 8 + }, + "id": 49, + "options": { + "legend": { + "calcs": [ + "max", + "mean", + "last" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "sum(system_cpu_usage{application=\"Artifactory\", namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\"}) by (pod) + sum(process_cpu_usage{application=\"Artifactory\", namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\"}) by (pod)", + "interval": "$min_step", + "legendFormat": "{{pod}} - artifactory", + "range": true, + "refId": "system_cpu_usage" + } + ], + "title": "Artifactory CPU Usage", + "type": "timeseries" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "The memory utilization ratio: Displays the ratio of used /committed JVM memory to the maximum available JVM memory", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "dashed+area" + } + }, + "links": [ + { + "targetBlank": true, + "title": "View more application metrics for this pod", + "url": "/d/spring-service-per-pod/spring-service-per-pod?var-datasource=GlobalView-Prometheus&var-service=Artifactory&var-deployment=${namespace}&${pod:queryparam}&${__url_time_range}" + } + ], + "mappings": [], + "max": 1, + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "transparent" + }, + { + "color": "yellow", + "value": 0.85 + }, + { + "color": "red", + "value": 0.95 + } + ] + }, + "unit": "percentunit" + }, + "overrides": [ + { + "matcher": { + "id": "byFrameRefID", + "options": "used" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "blue", + "mode": "fixed" + } + } + ] + }, + { + "matcher": { + "id": "byFrameRefID", + "options": "committed" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "purple", + "mode": "fixed" + } + } + ] + } + ] + }, + "gridPos": { + "h": 9, + "w": 12, + "x": 12, + "y": 8 + }, + "id": 48, + "options": { + "legend": { + "calcs": [ + "max", + "mean", + "last" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "sum(jvm_memory_used_bytes{application=\"Artifactory\", namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\"}) by (pod)\n/\nsum(jvm_memory_max_bytes{application=\"Artifactory\", namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\"}) by (pod)", + "interval": "", + "legendFormat": "Used Memory - {{pod}}", + "range": true, + "refId": "used" + }, + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "sum(jvm_memory_committed_bytes{application=\"Artifactory\", namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\"}) by (pod)\n/\nsum(jvm_memory_max_bytes{application=\"Artifactory\", namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\"}) by (pod)", + "hide": false, + "interval": "", + "legendFormat": "Committed Memory - {{pod}}", + "range": true, + "refId": "committed" + } + ], + "title": "JVM Memory Usage (% of Max Memory)", + "type": "timeseries" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "The number of threads in various states within the JVM.\n\nThread States:\n\n- **RUNNABLE**: Threads that are actively executing Java code.\n- **BLOCKED**: Threads that are waiting to acquire a monitor lock to enter a synchronized block or method.\n- **WAITING**: Threads that are waiting indefinitely for another thread to notify or interrupt them.\n- **TIMED_WAITING**: Threads that are waiting for a specific period, such as through a Thread.sleep() or similar operation.\n- **TERMINATED**: Threads that have completed their execution and terminated.", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + } + ] + }, + "unit": "none" + }, + "overrides": [] + }, + "gridPos": { + "h": 9, + "w": 12, + "x": 0, + "y": 17 + }, + "id": 101, + "interval": "$min_step", + "options": { + "legend": { + "calcs": [ + "max", + "mean", + "last" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true, + "sortBy": "Last", + "sortDesc": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "max(jvm_threads_states_threads{application=\"Artifactory\", namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\"}) by (pod, state) > 0", + "legendFormat": "threads-{{state}} - {{pod}}", + "range": true, + "refId": "A" + } + ], + "title": "JVM - Threads By State", + "type": "timeseries" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "The amount of memory that JVM uses:\n\n- **Committed Memory** is the amount of memory that JVM allocated from the operating system\n- **Used Memory** is the amount of memory that JVM actually uses\n- **Max Memory** is the maximal amount of memory available to JVM\n", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + } + ] + }, + "unit": "bytes" + }, + "overrides": [ + { + "matcher": { + "id": "byFrameRefID", + "options": "max_memory" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "red", + "mode": "fixed" + } + }, + { + "id": "custom.lineStyle", + "value": { + "dash": [ + 10, + 10 + ], + "fill": "dash" + } + } + ] + }, + { + "matcher": { + "id": "byFrameRefID", + "options": "committed" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "dark-purple", + "mode": "fixed" + } + }, + { + "id": "custom.lineStyle", + "value": { + "dash": [ + 10, + 10 + ], + "fill": "dash" + } + } + ] + } + ] + }, + "gridPos": { + "h": 9, + "w": 12, + "x": 12, + "y": 17 + }, + "id": 103, + "options": { + "legend": { + "calcs": [ + "max", + "min", + "mean", + "last" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "sum(jvm_memory_used_bytes{application=\"Artifactory\", namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\"}) by (pod)", + "legendFormat": "Used Memory - {{pod}}", + "range": true, + "refId": "used" + }, + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "sum(jvm_memory_committed_bytes{application=\"Artifactory\", namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\"}) by (pod)", + "hide": false, + "legendFormat": "Committed Memory - {{pod}}", + "range": true, + "refId": "committed" + }, + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "sum(jvm_memory_max_bytes{application=\"Artifactory\", namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\"}) by (pod)", + "hide": false, + "legendFormat": "Max Memory - {{pod}}", + "range": true, + "refId": "max_memory" + } + ], + "title": "JVM Memory Usage (Bytes)", + "type": "timeseries" + } + ], + "title": "CPU and JVM Memory Usage", + "type": "row" + }, + { + "collapsed": true, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 8 + }, + "id": 121, + "panels": [ + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "- **Busy Threads** - The current number of threads in Tomcat's thread pool that are actively processing requests.\n- **Idle Threads** - The current number of threads in Tomcat's thread pool that are waiting for incoming requests.\n- **Max Threads** - The maximum number of threads configured for Tomcat's thread pool.", + "fieldConfig": { + "defaults": { + "color": { + "mode": "fixed" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + } + ] + }, + "unit": "none" + }, + "overrides": [ + { + "matcher": { + "id": "byFrameRefID", + "options": "busy_threads" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "yellow", + "mode": "fixed" + } + } + ] + }, + { + "matcher": { + "id": "byFrameRefID", + "options": "idle_threads" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "green", + "mode": "fixed" + } + } + ] + }, + { + "matcher": { + "id": "byFrameRefID", + "options": "max_threads" + }, + "properties": [ + { + "id": "custom.lineStyle", + "value": { + "dash": [ + 10, + 10 + ], + "fill": "dash" + } + }, + { + "id": "color", + "value": { + "fixedColor": "red", + "mode": "fixed" + } + } + ] + } + ] + }, + "gridPos": { + "h": 9, + "w": 12, + "x": 0, + "y": 79 + }, + "id": 123, + "interval": "$min_step", + "options": { + "legend": { + "calcs": [ + "max", + "mean", + "last" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "12.0.2", + "repeat": "tomcat_app", + "repeatDirection": "v", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "max(tomcat_threads_busy_threads{application=\"Artifactory\", namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\", name=~\"$tomcat_app\"}) by (name, pod)", + "legendFormat": "Busy Threads {{name}} / {{pod}}", + "range": true, + "refId": "busy_threads" + }, + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "max(tomcat_threads_current_threads{application=\"Artifactory\", namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\", name=~\"$tomcat_app\"} \n -\ntomcat_threads_busy_threads{application=\"Artifactory\", namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\", name=~\"$tomcat_app\"}\n) by (name, pod)", + "hide": false, + "legendFormat": "Idle Threads {{name}} / {{pod}}", + "range": true, + "refId": "idle_threads" + }, + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "max(tomcat_threads_config_max_threads{application=\"Artifactory\", namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\", name=~\"$tomcat_app\"}) by (name, pod)", + "hide": false, + "legendFormat": "Max Threads {{name}} / {{pod}}", + "range": true, + "refId": "max_threads" + } + ], + "title": "Tomcat - Threads ($tomcat_app)", + "type": "timeseries" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "mode": "fixed" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + } + ] + }, + "unit": "none" + }, + "overrides": [ + { + "matcher": { + "id": "byFrameRefID", + "options": "active_connections" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "yellow", + "mode": "fixed" + } + } + ] + }, + { + "matcher": { + "id": "byFrameRefID", + "options": "max_connections" + }, + "properties": [ + { + "id": "custom.lineStyle", + "value": { + "dash": [ + 10, + 10 + ], + "fill": "dash" + } + }, + { + "id": "color", + "value": { + "fixedColor": "red", + "mode": "fixed" + } + } + ] + } + ] + }, + "gridPos": { + "h": 9, + "w": 12, + "x": 12, + "y": 79 + }, + "id": 124, + "interval": "$min_step", + "options": { + "legend": { + "calcs": [ + "max", + "mean", + "last" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "12.0.2", + "repeat": "tomcat_app", + "repeatDirection": "v", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "max(tomcat_connections_current_connections{application=\"Artifactory\", namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\", name=~\"$tomcat_app\"}) by (pod)", + "legendFormat": "Active Connections - {{pod}}", + "range": true, + "refId": "active_connections" + }, + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "max(tomcat_connections_config_max_connections{application=\"Artifactory\", namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\", name=~\"$tomcat_app\"}) by (pod)", + "hide": false, + "legendFormat": "Max Connections - {{pod}}", + "range": true, + "refId": "max_connections" + } + ], + "title": "Tomcat - Connections ($tomcat_app)", + "type": "timeseries" + } + ], + "title": "Tomcat Metrics", + "type": "row" + }, + { + "collapsed": true, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 9 + }, + "id": 116, + "panels": [ + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "fieldConfig": { + "defaults": { + "color": { + "fixedColor": "blue", + "mode": "fixed" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "transparent" + } + ] + }, + "unit": "short" + }, + "overrides": [ + { + "matcher": { + "id": "byFrameRefID", + "options": "idle_connections" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "green", + "mode": "fixed" + } + } + ] + }, + { + "matcher": { + "id": "byFrameRefID", + "options": "active_connections" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "yellow", + "mode": "fixed" + } + } + ] + } + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 204 + }, + "id": 119, + "options": { + "legend": { + "calcs": [ + "max", + "min", + "mean" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "max(jfrt_db_connections_active_total{namespace=\"$namespace\", pod=~\"$pod\"}) by (pod)", + "legendFormat": "Active Connections - {{pod}}", + "range": true, + "refId": "active_connections" + }, + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "max(jfrt_db_connections_idle_total{namespace=\"$namespace\", pod=~\"$pod\"}) by (pod)", + "hide": false, + "legendFormat": "idle Connections - {{pod}}", + "range": true, + "refId": "idle_connections" + }, + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "max(jfrt_db_connections_active_total{namespace=\"$namespace\", pod=~\"$pod\"} + jfrt_db_connections_idle_total{namespace=\"$namespace\", pod=~\"$pod\"}) by (pod)", + "hide": false, + "legendFormat": "Total Connections - {{pod}}", + "range": true, + "refId": "total" + } + ], + "title": "DB Connections", + "type": "timeseries" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "fixedColor": "blue", + "mode": "fixed" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + } + ] + }, + "unit": "short" + }, + "overrides": [ + { + "matcher": { + "id": "byFrameRefID", + "options": "slow_queries" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "red", + "mode": "fixed" + } + } + ] + } + ] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 212 + }, + "id": 89, + "interval": "$min_step", + "links": [ + { + "targetBlank": true, + "title": "View customer DB usage dashboard", + "url": "/d/ivRVpsdIk/customer-db-usage?${namespace:queryparam}&${__url_time_range}" + } + ], + "options": { + "legend": { + "calcs": [ + "max", + "mean", + "last", + "sum" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "sum(increase(db_query_duration_seconds_count{application=\"Artifactory\", namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\"}[$__rate_interval]))", + "interval": "2m", + "legendFormat": "Total queries", + "range": true, + "refId": "db_queries" + }, + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "sum(increase(db_query_duration_seconds_count{application=\"Artifactory\", namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\", slow=\"y\"}[$__rate_interval]))", + "hide": false, + "interval": "2m", + "legendFormat": "Slow queries", + "range": true, + "refId": "slow_queries" + } + ], + "title": "DB Queries Count", + "type": "timeseries" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "fixedColor": "blue", + "mode": "fixed" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 1, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 212 + }, + "id": 97, + "interval": "$min_step", + "links": [ + { + "targetBlank": true, + "title": "View customer DB usage dashboard", + "url": "/d/ivRVpsdIk/customer-db-usage?${namespace:queryparam}&${__url_time_range}" + } + ], + "options": { + "legend": { + "calcs": [ + "max", + "mean", + "last", + "sum" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "sum(rate(db_query_duration_seconds_count{namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\"}[$__rate_interval])) * 60", + "interval": "2m", + "legendFormat": "DB queries per minute", + "range": true, + "refId": "db_queries" + } + ], + "title": "DB Queries Rate (queries / minute)", + "type": "timeseries" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "fixedColor": "blue", + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 30, + "gradientMode": "opacity", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineStyle": { + "fill": "solid" + }, + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "normal" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + } + ] + }, + "unit": "short" + }, + "overrides": [ + { + "matcher": { + "id": "byFrameRefID", + "options": "slow_queries" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "red", + "mode": "fixed" + } + } + ] + } + ] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 220 + }, + "id": 180, + "interval": "$min_step", + "links": [ + { + "targetBlank": true, + "title": "View customer DB usage dashboard", + "url": "/d/ivRVpsdIk/customer-db-usage?${namespace:queryparam}&${__url_time_range}" + } + ], + "options": { + "legend": { + "calcs": [ + "max", + "mean", + "last", + "sum" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "sum(increase(db_query_duration_seconds_count{application=\"Artifactory\", namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\"}[$__rate_interval])) by (pod)", + "interval": "2m", + "legendFormat": "Total queries - {{pod}}", + "range": true, + "refId": "db_queries" + }, + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "sum(increase(db_query_duration_seconds_count{application=\"Artifactory\", namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\", slow=\"y\"}[$__rate_interval])) by (pod) > 0", + "hide": false, + "interval": "2m", + "legendFormat": "Slow queries - {{pod}}", + "range": true, + "refId": "slow_queries" + } + ], + "title": "DB Queries Count - Per Pod", + "type": "timeseries" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "fixedColor": "blue", + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 1, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 30, + "gradientMode": "opacity", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 220 + }, + "id": 165, + "interval": "$min_step", + "links": [ + { + "targetBlank": true, + "title": "View customer DB usage dashboard", + "url": "/d/ivRVpsdIk/customer-db-usage?${namespace:queryparam}&${__url_time_range}" + } + ], + "options": { + "legend": { + "calcs": [ + "max", + "mean", + "last", + "sum" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true, + "sortBy": "Total", + "sortDesc": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "sum(rate(db_query_duration_seconds_count{namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\"}[$__rate_interval])) by (pod) * 60", + "interval": "2m", + "legendFormat": "{{pod}}", + "range": true, + "refId": "db_queries" + } + ], + "title": "DB Queries Rate (queries / minute) - Per Pod", + "type": "timeseries" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "This graph lists queries that we suspect to be a source of a significant load on DB.", + "fieldConfig": { + "defaults": { + "color": { + "fixedColor": "blue", + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "bars", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "normal" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 228 + }, + "id": 95, + "interval": "$min_step", + "links": [ + { + "targetBlank": true, + "title": "View customer DB usage dashboard", + "url": "/d/ivRVpsdIk/customer-db-usage?${namespace:queryparam}&${__url_time_range}" + } + ], + "options": { + "legend": { + "calcs": [ + "max", + "mean", + "sum" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true, + "sortBy": "Total", + "sortDesc": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "topk(10,\n sum(\n rate(db_query_duration_seconds_sum{namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\"}[$__rate_interval])\n ) by (type,tables) >= 0.5\n)", + "interval": "2m", + "legendFormat": "{{type}} {{tables}}", + "range": true, + "refId": "intensive_queries" + } + ], + "title": "Intensive DB Queries - Wait Time / second", + "type": "timeseries" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "fixedColor": "blue", + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 1, + "barWidthFactor": 0.6, + "drawStyle": "bars", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + } + ] + }, + "unit": "s" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 228 + }, + "id": 90, + "interval": "$min_step", + "links": [ + { + "targetBlank": true, + "title": "View customer DB usage dashboard", + "url": "/d/ivRVpsdIk/customer-db-usage?${namespace:queryparam}&${__url_time_range}" + } + ], + "options": { + "legend": { + "calcs": [ + "max", + "mean", + "count" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true, + "sortBy": "Total", + "sortDesc": true + }, + "tooltip": { + "hideZeros": false, + "mode": "single", + "sort": "none" + } + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "max_over_time(db_query_duration_seconds_max{application=\"Artifactory\", namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\", slow=\"y\"}[ $__rate_interval]) > 0", + "hide": false, + "interval": "5m", + "legendFormat": "{{type}} {{tables}}", + "range": true, + "refId": "slow_queries" + } + ], + "title": "Slow Queries - Max Duration (sec)", + "type": "timeseries" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "This graph lists Artifactory activities that spend \"too much\" time, waiting for DB queries.", + "fieldConfig": { + "defaults": { + "color": { + "fixedColor": "blue", + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "bars", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "normal" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 236 + }, + "id": 112, + "interval": "$min_step", + "links": [ + { + "targetBlank": true, + "title": "View customer DB usage dashboard", + "url": "/d/ivRVpsdIk/customer-db-usage?${namespace:queryparam}&${__url_time_range}" + } + ], + "options": { + "legend": { + "calcs": [ + "max", + "mean", + "sum" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true, + "sortBy": "Total", + "sortDesc": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "sum(\n rate(db_query_by_trigger_seconds_sum{namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\"}[$__rate_interval])\n) by (trigger, pkg) >= 0.5", + "interval": "2m", + "legendFormat": "{{pkg}} - {{trigger}}", + "range": true, + "refId": "intensive_queries" + } + ], + "title": "DB-Intensive Artifactory Activities", + "transformations": [ + { + "id": "renameByRegex", + "options": { + "regex": "none - ", + "renamePattern": "(other)" + } + } + ], + "type": "timeseries" + } + ], + "title": "Database", + "type": "row" + }, + { + "collapsed": true, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 10 + }, + "id": 108, + "panels": [ + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "bars", + "fillOpacity": 66, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "normal" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "gridPos": { + "h": 9, + "w": 12, + "x": 0, + "y": 434 + }, + "id": 110, + "interval": "$min_step", + "options": { + "legend": { + "calcs": [ + "max", + "mean", + "sum" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true, + "sortBy": "Max", + "sortDesc": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "sum(rate(jobs_exec_seconds_sum{application=\"Artifactory\", namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\"}[$__rate_interval])) by (name) > 0.1", + "interval": "2m", + "legendFormat": "{{name}}", + "range": true, + "refId": "A" + } + ], + "title": "Scheduled Jobs with High Work Time (> 10% of thread time)", + "type": "timeseries" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "bars", + "fillOpacity": 66, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "normal" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "gridPos": { + "h": 9, + "w": 12, + "x": 12, + "y": 434 + }, + "id": 111, + "interval": "$min_step", + "options": { + "legend": { + "calcs": [ + "max", + "mean", + "sum" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true, + "sortBy": "Max", + "sortDesc": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "sum(rate(workqueue_execution_duration_seconds_sum{application=\"Artifactory\", namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\"}[$__rate_interval])) by (name) > 0.1", + "interval": "2m", + "legendFormat": "{{name}}", + "range": true, + "refId": "A" + } + ], + "title": "Work Queues with High Work Time (> 10% of thread time)", + "type": "timeseries" + } + ], + "title": "Heavy Background Activities", + "type": "row" + }, + { + "collapsed": true, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 11 + }, + "id": 126, + "panels": [ + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "The total number of binary downloads - per provider", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "links": [], + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 8, + "x": 0, + "y": 162 + }, + "id": 128, + "interval": "$min_step", + "options": { + "legend": { + "calcs": [ + "max", + "sum" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "sum(increase(jfsh_binaries_download_total{namespace=\"$namespace\", pod=~\"$pod\"}[$__rate_interval])) by (id)", + "interval": "2m", + "legendFormat": "__auto", + "range": true, + "refId": "A" + } + ], + "title": "Binary Downloads - Per Storage Provider", + "type": "timeseries" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "The total number of binary uploads - per storage provider", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "links": [], + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 8, + "x": 8, + "y": 162 + }, + "id": 130, + "interval": "$min_step", + "options": { + "legend": { + "calcs": [ + "max", + "sum" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "sum(increase(jfsh_binaries_upload_total{namespace=\"$namespace\", pod=~\"$pod\"}[$__rate_interval])) by (id)", + "interval": "2m", + "legendFormat": "__auto", + "range": true, + "refId": "A" + } + ], + "title": "Binary Uploads - Per Storage Provider", + "type": "timeseries" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "The total number of signed URL redirects per storage provider", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "links": [], + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 8, + "x": 16, + "y": 162 + }, + "id": 132, + "interval": "$min_step", + "options": { + "legend": { + "calcs": [ + "max", + "sum" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "sum(increase(jfsh_signed_url_redirect_total{namespace=\"$namespace\", pod=~\"$pod\"}[$__rate_interval])) by (source)", + "interval": "2m", + "legendFormat": "__auto", + "range": true, + "refId": "A" + } + ], + "title": "Signed URL Redirects - Per Storage Provider", + "type": "timeseries" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "fixedColor": "blue", + "mode": "fixed" + }, + "links": [], + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + } + ] + }, + "unit": "Bps" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "0.01 " + }, + "properties": [ + { + "id": "displayName", + "value": "1% of requests" + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "0.5 " + }, + "properties": [ + { + "id": "displayName", + "value": "50% of requests" + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "0.99 " + }, + "properties": [ + { + "id": "displayName", + "value": "99% of requests" + } + ] + } + ] + }, + "gridPos": { + "h": 5, + "w": 8, + "x": 0, + "y": 170 + }, + "id": 134, + "options": { + "minVizHeight": 75, + "minVizWidth": 75, + "orientation": "vertical", + "reduceOptions": { + "calcs": [ + "min" + ], + "fields": "", + "values": true + }, + "showThresholdLabels": false, + "showThresholdMarkers": true, + "sizing": "auto" + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "exemplar": false, + "expr": "avg(jfsh_binaries_download_speed{quantile=~\"0\\\\.01|0\\\\.5|0\\\\.99\", namespace=~\"$namespace\", pod=~\"$pod\"}) by (namespace, quantile)", + "format": "heatmap", + "instant": true, + "interval": "", + "legendFormat": "{{quantile}} ", + "range": false, + "refId": "A" + } + ], + "title": "Average Download Speed", + "type": "gauge" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "Bps" + }, + "overrides": [] + }, + "gridPos": { + "h": 10, + "w": 16, + "x": 8, + "y": 170 + }, + "id": 138, + "interval": "$min_step", + "options": { + "legend": { + "calcs": [ + "max", + "mean", + "lastNotNull" + ], + "displayMode": "table", + "placement": "right", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "sum(increase(jfsh_binaries_upload_total{namespace=\"$namespace\", pod=~\"$pod\"}[$__rate_interval])) by (id)", + "interval": "2m", + "legendFormat": "Upload [ {{id}} ]", + "range": true, + "refId": "Upload" + }, + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "sum(increase(jfsh_binaries_download_total{namespace=\"$namespace\", pod=~\"$pod\"}[$__rate_interval])) by (id)", + "hide": false, + "interval": "2m", + "legendFormat": "Download [ {{id}} ]", + "range": true, + "refId": "Download" + } + ], + "title": "Upload/Download Throughput", + "type": "timeseries" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "fixedColor": "blue", + "mode": "fixed" + }, + "links": [], + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + } + ] + }, + "unit": "Bps" + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "0.01 " + }, + "properties": [ + { + "id": "displayName", + "value": "1% of requests" + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "0.5 " + }, + "properties": [ + { + "id": "displayName", + "value": "50% of requests" + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "0.99 " + }, + "properties": [ + { + "id": "displayName", + "value": "99% of requests" + } + ] + } + ] + }, + "gridPos": { + "h": 5, + "w": 8, + "x": 0, + "y": 175 + }, + "id": 136, + "options": { + "minVizHeight": 75, + "minVizWidth": 75, + "orientation": "vertical", + "reduceOptions": { + "calcs": [ + "min" + ], + "fields": "", + "values": true + }, + "showThresholdLabels": false, + "showThresholdMarkers": true, + "sizing": "auto" + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "exemplar": false, + "expr": "avg(jfsh_binaries_upload_speed{quantile=~\"0\\\\.01|0\\\\.5|0\\\\.99\", namespace=~\"$namespace\", pod=~\"$pod\"}) by (namespace, quantile)", + "format": "heatmap", + "instant": true, + "interval": "2m", + "legendFormat": "{{quantile}} ", + "range": false, + "refId": "A" + } + ], + "title": "Average Upload Speed", + "type": "gauge" + } + ], + "title": "Storage Activity - Uploads / Downloads / Redirects", + "type": "row" + }, + { + "collapsed": true, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 12 + }, + "id": 144, + "panels": [ + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "red" + }, + { + "color": "#EAB839", + "value": 50000000 + }, + { + "color": "green", + "value": 50000000000 + } + ] + }, + "unit": "decbytes" + }, + "overrides": [ + { + "matcher": { + "id": "byFrameRefID", + "options": "used_bytes" + }, + "properties": [ + { + "id": "color", + "value": { + "fixedColor": "blue", + "mode": "fixed" + } + } + ] + } + ] + }, + "gridPos": { + "h": 8, + "w": 24, + "x": 0, + "y": 49 + }, + "id": 150, + "interval": "$min_step", + "maxPerRow": 2, + "options": { + "legend": { + "calcs": [ + "min", + "max", + "mean", + "last" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "none" + } + }, + "pluginVersion": "12.0.2", + "repeat": "pod", + "repeatDirection": "h", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "min(app_disk_free_bytes{namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\"})", + "legendFormat": "Free Disk Space (min)", + "range": true, + "refId": "free_bytes" + }, + { + "datasource": { + "type": "prometheus", + "uid": "${datasource}" + }, + "editorMode": "code", + "expr": "max(app_disk_used_bytes{namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\"})", + "hide": false, + "legendFormat": "Used Disk Space (max)", + "range": true, + "refId": "used_bytes" + } + ], + "title": "Disk Usage - $pod", + "type": "timeseries" + } + ], + "title": "Pod Disk Usage", + "type": "row" + } + ], + "preload": false, + "refresh": false, + "schemaVersion": 41, + "tags": [ + "jfrog", + "artifactory" + ], + "templating": { + "list": [ + { + "auto": true, + "auto_count": 500, + "auto_min": "15s", + "current": { + "text": "$__auto", + "value": "$__auto" + }, + "description": "The minimal step (time period) used to aggregate the data for. Using periods shorter than 75 seconds often causes Prometheus to bring no data at all.", + "label": "Granularity", + "name": "min_step", + "options": [ + { + "selected": false, + "text": "1m", + "value": "1m" + }, + { + "selected": false, + "text": "5m", + "value": "5m" + }, + { + "selected": false, + "text": "10m", + "value": "10m" + }, + { + "selected": false, + "text": "15m", + "value": "15m" + }, + { + "selected": false, + "text": "20m", + "value": "20m" + }, + { + "selected": false, + "text": "30m", + "value": "30m" + }, + { + "selected": false, + "text": "1h", + "value": "1h" + }, + { + "selected": false, + "text": "6h", + "value": "6h" + }, + { + "selected": false, + "text": "1d", + "value": "1d" + } + ], + "query": "1m,5m,10m,15m,20m,30m,1h,6h,1d", + "refresh": 2, + "type": "interval" + }, + { + "current": { + "text": "jfrog", + "value": "jfrog" + }, + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "definition": "label_values(kube_namespace_status_phase,namespace)", + "description": "The list of K8S namespaces, limited to 1000 values.", + "includeAll": false, + "label": "namespace", + "name": "namespace", + "options": [], + "query": { + "qryType": 1, + "query": "label_values(kube_namespace_status_phase,namespace)", + "refId": "PrometheusVariableQueryEditor-VariableQuery" + }, + "refresh": 1, + "regex": "", + "type": "query" + }, + { + "current": { + "text": "artifactory-0", + "value": "artifactory-0" + }, + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "definition": "label_values(kube_pod_info{namespace=\"$namespace\"}, pod)", + "description": "The Artifactory pod", + "includeAll": false, + "name": "pod", + "options": [], + "query": { + "query": "label_values(kube_pod_info{namespace=\"$namespace\"}, pod)", + "refId": "StandardVariableQuery" + }, + "refresh": 1, + "regex": "", + "sort": 1, + "type": "query" + }, + { + "allValue": ".*", + "current": { + "text": "All", + "value": [ + "$__all" + ] + }, + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "definition": "label_values(tomcat_threads_busy_threads{application=\"Artifactory\", namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\"},name)", + "description": "The Tomcat application ", + "hide": 2, + "includeAll": true, + "multi": true, + "name": "tomcat_app", + "options": [], + "query": { + "qryType": 1, + "query": "label_values(tomcat_threads_busy_threads{application=\"Artifactory\", namespace=\"$namespace\", pod=~\"$pod\", container=\"artifactory\"},name)", + "refId": "PrometheusVariableQueryEditor-VariableQuery" + }, + "refresh": 1, + "regex": "", + "sort": 1, + "type": "query" + } + ] + }, + "time": { + "from": "now-1h", + "to": "now" + }, + "timepicker": {}, + "timezone": "", + "title": "JFrog Artifactory Metrics", + "uid": "h_S4TyNIz", + "version": 3 +} diff --git a/grafana/XrayLogAnalyticsAndSystemMetrics.json b/grafana/XrayLogAnalyticsAndSystemMetrics.json index 5302541..faebf22 100644 --- a/grafana/XrayLogAnalyticsAndSystemMetrics.json +++ b/grafana/XrayLogAnalyticsAndSystemMetrics.json @@ -1015,8 +1015,8 @@ ] }, "timezone": "", - "title": "Xray (New)", + "title": "Xray Metrics", "uid": "siDGzFmMk", "version": 1, "weekStart": "" -} \ No newline at end of file +} diff --git a/grafana/XrayMetrics.json b/grafana/XrayMetrics.json new file mode 100644 index 0000000..05318b6 --- /dev/null +++ b/grafana/XrayMetrics.json @@ -0,0 +1,1143 @@ +{ + "annotations": { + "list": [ + { + "$$hashKey": "object:525", + "builtIn": 1, + "datasource": { + "type": "datasource", + "uid": "grafana" + }, + "enable": true, + "hide": true, + "iconColor": "rgba(0, 211, 255, 1)", + "name": "Annotations & Alerts", + "target": { + "limit": 100, + "matchAny": false, + "tags": [], + "type": "dashboard" + }, + "type": "dashboard" + } + ] + }, + "editable": true, + "fiscalYearStartMonth": 0, + "graphTooltip": 0, + "id": 141, + "links": [], + "panels": [ + { + "collapsed": false, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 0 + }, + "id": 80, + "panels": [], + "title": "General Information", + "type": "row" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 4, + "w": 6, + "x": 0, + "y": 1 + }, + "id": 81, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "percentChangeColorMode": "standard", + "reduceOptions": { + "calcs": [ + "mean" + ], + "fields": "", + "values": false + }, + "showPercentChange": false, + "textMode": "name", + "wideLayout": true + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "editorMode": "code", + "exemplar": false, + "expr": "topk(1,label_replace(kube_pod_container_info{namespace=\"$namespace\",container=\"xray-server\"}, \"version\", \"$1\", \"image\", \".+:(.+)\"))", + "instant": true, + "legendFormat": "{{version}}", + "range": false, + "refId": "A" + } + ], + "title": "Xray Version", + "type": "stat" + }, + { + "collapsed": true, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 5 + }, + "id": 68, + "panels": [ + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "fieldMinMax": false, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 6 + }, + "id": 78, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "single", + "sort": "none" + } + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "editorMode": "code", + "expr": "sum(rate(container_cpu_usage_seconds_total{namespace=\"$namespace\", pod=\"$pod\", pod=~\".*xray.*\", container!=\"\", container!=\"POD\"}[$__rate_interval])) by (container)", + "legendFormat": "{{container}}", + "range": true, + "refId": "A" + } + ], + "title": "CPU Usage per Container", + "type": "timeseries" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + } + ] + }, + "unit": "decbytes" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 6 + }, + "id": 72, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "single", + "sort": "none" + } + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "editorMode": "code", + "expr": "sum(container_memory_working_set_bytes{namespace=\"$namespace\",pod=\"$pod\", pod=~\".*xray.*\", container!=\"\", container!=\"POD\"}) by (container)", + "legendFormat": "{{container}}", + "range": true, + "refId": "A" + } + ], + "title": "Memory Usage per Container", + "type": "timeseries" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "left", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineStyle": { + "fill": "solid" + }, + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "percentage", + "steps": [ + { + "color": "green" + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 14 + }, + "id": 79, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "single", + "sort": "none" + } + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "editorMode": "code", + "expr": "avg(server_go_memstats_heap_objects_total{service=\"xray-auth-metrics\"})", + "legendFormat": "Total Heap Objects", + "range": true, + "refId": "A" + } + ], + "title": "Heap Objects", + "type": "timeseries" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "left", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineStyle": { + "fill": "solid" + }, + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "percentage", + "steps": [ + { + "color": "green" + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 14 + }, + "id": 76, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "single", + "sort": "none" + } + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "editorMode": "code", + "expr": "avg(server_go_memstats_heap_in_use_bytes{service=\"xray-auth-metrics\"}) / (1024 * 1024)", + "hide": false, + "legendFormat": "Heap Used MBytes", + "range": true, + "refId": "B" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "editorMode": "code", + "expr": "avg(server_go_memstats_heap_reserved_bytes{service=\"xray-auth-metrics\"}) / (1024 * 1024)", + "hide": false, + "legendFormat": "Heap Reserved MBytes", + "range": true, + "refId": "C" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "editorMode": "code", + "expr": "avg(server_go_memstats_heap_allocated_bytes{service=\"xray-auth-metrics\"}) / (1024 * 1024)", + "hide": false, + "legendFormat": "Heap Allocated MBytes", + "range": true, + "refId": "D" + } + ], + "title": "Heap Usage MBytes", + "type": "timeseries" + } + ], + "title": "CPU and Memory Usage", + "type": "row" + }, + { + "collapsed": true, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 6 + }, + "id": 83, + "panels": [ + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "% of Xray DB connection used", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 93 + }, + "id": 84, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "single", + "sort": "none" + } + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "editorMode": "code", + "expr": "avg(db_connection_pool_in_use_total{service=\"xray-auth-metrics\"})", + "legendFormat": "Active", + "range": true, + "refId": "A" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "editorMode": "code", + "expr": "avg(db_connection_pool_idle_total{service=\"xray-auth-metrics\"})", + "hide": false, + "legendFormat": "Idle", + "range": true, + "refId": "B" + } + ], + "title": "Database Connections Usage", + "type": "timeseries" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "% of Xray DB connection used", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "percent" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 93 + }, + "id": 85, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "single", + "sort": "none" + } + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "editorMode": "code", + "expr": "max(db_connection_pool_in_use_total{service=\"xray-auth-metrics\"} / db_connection_pool_max_open_total{service=\"xray-auth-metrics\"}*100)", + "legendFormat": "Percent", + "range": true, + "refId": "A" + } + ], + "title": "Database Connections Usage %", + "type": "timeseries" + } + ], + "title": "Database", + "type": "row" + }, + { + "collapsed": true, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 7 + }, + "id": 86, + "panels": [ + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "RabbitMQ queue message counts for all queues excluding the retry queues", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "Messages", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "yellow", + "value": 100 + }, + { + "color": "red", + "value": 1000 + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "gridPos": { + "h": 11, + "w": 24, + "x": 0, + "y": 43 + }, + "id": 87, + "options": { + "legend": { + "calcs": [ + "lastNotNull", + "max" + ], + "displayMode": "table", + "placement": "right", + "showLegend": true, + "sortBy": "Name", + "sortDesc": false + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "editorMode": "code", + "expr": "queue_messages_total{service=\"xray-auth-metrics\", queue_name!~\".*(R|r)etry.*\"}", + "legendFormat": "{{queue_name}}", + "range": true, + "refId": "A" + } + ], + "title": "RabbitMQ Queue Messages (no retry)", + "type": "timeseries" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "description": "RabbitMQ queue message counts for all queues", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "Messages", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 2, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "yellow", + "value": 100 + }, + { + "color": "red", + "value": 1000 + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "gridPos": { + "h": 11, + "w": 24, + "x": 0, + "y": 54 + }, + "id": 88, + "options": { + "legend": { + "calcs": [ + "lastNotNull", + "max" + ], + "displayMode": "table", + "placement": "right", + "showLegend": true, + "sortBy": "Name", + "sortDesc": false + }, + "tooltip": { + "hideZeros": false, + "mode": "multi", + "sort": "desc" + } + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "editorMode": "code", + "expr": "queue_messages_total{service=\"xray-auth-metrics\"}", + "legendFormat": "{{queue_name}}", + "range": true, + "refId": "A" + } + ], + "title": "RabbitMQ Queue Messages (all)", + "type": "timeseries" + } + ], + "title": "RabbitMQ", + "type": "row" + }, + { + "collapsed": true, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 8 + }, + "id": 82, + "panels": [ + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisBorderShow": false, + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "barWidthFactor": 0.6, + "drawStyle": "line", + "fillOpacity": 0, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "insertNulls": false, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "percentage", + "steps": [ + { + "color": "green" + } + ] + }, + "unit": "decbytes" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 94 + }, + "id": 74, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "hideZeros": false, + "mode": "single", + "sort": "none" + } + }, + "pluginVersion": "12.0.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "editorMode": "code", + "expr": "avg(server_app_disk_free_bytes{service=\"xray-auth-metrics\"})", + "legendFormat": "Free Disk Space", + "range": true, + "refId": "A" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus" + }, + "editorMode": "code", + "expr": "avg(server_app_disk_used_bytes{service=\"xray-auth-metrics\"})", + "hide": false, + "legendFormat": "Used Disk Space", + "range": true, + "refId": "B" + } + ], + "title": "Xray Services Disk Usage", + "type": "timeseries" + } + ], + "title": "Pod Disk Usage", + "type": "row" + } + ], + "preload": false, + "refresh": "", + "schemaVersion": 41, + "tags": [ + "jfrog", + "xray" + ], + "templating": { + "list": [ + { + "current": { + "text": "jfrog", + "value": "jfrog" + }, + "definition": "label_values(kube_namespace_status_phase,namespace)", + "description": "List of namespaces", + "label": "namespace", + "name": "namespace", + "options": [], + "query": { + "qryType": 1, + "query": "label_values(kube_namespace_status_phase,namespace)", + "refId": "PrometheusVariableQueryEditor-VariableQuery" + }, + "refresh": 1, + "regex": "", + "type": "query" + }, + { + "current": { + "text": "xray-0", + "value": "xray-0" + }, + "definition": "label_values(kube_pod_info{namespace=\"$namespace\"},pod)", + "description": "The Xray pod", + "label": "pod", + "name": "pod", + "options": [], + "query": { + "qryType": 1, + "query": "label_values(kube_pod_info{namespace=\"$namespace\"},pod)", + "refId": "PrometheusVariableQueryEditor-VariableQuery" + }, + "refresh": 1, + "regex": "", + "type": "query" + } + ] + }, + "time": { + "from": "now-1h", + "to": "now" + }, + "timepicker": {}, + "timezone": "", + "title": "JFrog Xray Metrics", + "uid": "siDGzFmMkx1", + "version": 4 +} diff --git a/helm/artifactory-ha-values.yaml b/helm/artifactory-ha-values.yaml deleted file mode 100644 index 9acd499..0000000 --- a/helm/artifactory-ha-values.yaml +++ /dev/null @@ -1,144 +0,0 @@ -global: - jfrog: - observability: - branch: master - prometheus: - release_name: prometheus -installerInfo: '{ "productId": "Helm_prometheus_artifactory/{{ .Chart.Version }}", "features": [ { "featureId": "ArtifactoryVersion/{{ default .Chart.AppVersion .Values.artifactory.image.version }}" }, { "featureId": "{{ if .Values.postgresql.enabled }}postgresql{{ else }}{{ .Values.database.type }}{{ end }}/0.0.0" }, { "featureId": "Platform/{{ default "kubernetes" .Values.installer.platform }}" }, { "featureId": "Channel/Helm_prometheus_artifactory" } ] }' -artifactory: - # For Artifactory versions <= 7.86.x please use artifactory.openMetrics.enabled instead of artifactory.metrics.enabled - metrics: - enabled: true - customInitContainersBegin: | - - name: "prepare-fluentd-conf-on-persistent-volume" - image: {{ include "artifactory-ha.getImageInfoByValue" (list . "initContainers") }} - imagePullPolicy: "{{ .Values.artifactory.image.pullPolicy }}" - command: - - 'sh' - - '-c' - - > - mkdir -p {{ .Values.artifactory.persistence.mountPath }}/etc/fluentd/; - curl https://raw.githubusercontent.com/jfrog/log-analytics-prometheus/{{ .Values.global.jfrog.observability.branch }}/fluent_metrics.conf.rt -o {{ .Values.artifactory.persistence.mountPath }}/etc/fluentd/fluentd.conf - volumeMounts: - - mountPath: "{{ .Values.artifactory.persistence.mountPath }}" - name: volume - customSidecarContainers: | - - name: "artifactory-fluentd-sidecar" - image: "releases-pts-observability-fluentd.jfrog.io/fluentd:4.15" - imagePullPolicy: "IfNotPresent" - volumeMounts: - - mountPath: "{{ .Values.artifactory.persistence.mountPath }}" - name: volume - env: - - name: JF_PRODUCT_DATA_INTERNAL - value: {{ .Values.artifactory.persistence.mountPath }} - - name: LOKI_URL - value: http://loki.{{ .Release.Namespace }}:3100 - - name: FLUENTD_CONF - value: ../../../..{{ .Values.artifactory.persistence.mountPath }}/etc/fluentd/fluentd.conf - -## Allows to add additional kubernetes resources -## Use --- as a separator between multiple resources -additionalResources: | - apiVersion: v1 - kind: Service - metadata: - name: {{ template "artifactory-ha.fullname" . }}-metrics - annotations: - prometheus.io/path: "/artifactory/api/v1/metrics" - prometheus.io/scheme: "http" - prometheus.io/port: "24231" - prometheus.io/scrape: "true" - labels: - app: {{ template "artifactory-ha.name" . }} - chart: {{ template "artifactory-ha.chart" . }} - component: {{ template "artifactory-ha.name" . }} - heritage: {{ .Release.Service }} - release: {{ .Release.Name }} - {{- with .Values.artifactory-ha.labels }} - {{ toYaml . | indent 4 }} - {{- end }} - spec: - type: ClusterIP - ports: - - name: metrics - port: 24231 - targetPort: 8082 - protocol: TCP - selector: - app: {{ template "artifactory-ha.name" . }} - component: {{ template "artifactory-ha.name" . }} - --- - apiVersion: monitoring.coreos.com/v1 - kind: ServiceMonitor - metadata: - name: servicemonitor-{{ template "artifactory-ha.name" . }} - labels: - metrics: jfrog - app: {{ template "artifactory-ha.name" . }} - chart: {{ template "artifactory-ha.chart" . }} - heritage: {{ .Release.Service }} - release: {{ .Values.global.prometheus.release_name }} - spec: - selector: - matchLabels: - app: {{ template "artifactory-ha.name" . }} - component: {{ template "artifactory-ha.name" . }} - endpoints: - - port: metrics - scheme: http - path: /artifactory/api/v1/metrics - interval: 15s - bearerTokenSecret: - key: token - name: jfrog-admin-token - --- - apiVersion: v1 - kind: Service - metadata: - name: {{ .Values.observability.name }}-metrics - annotations: - prometheus.io/path: "/observability/api/v1/metrics" - prometheus.io/scheme: "http" - prometheus.io/port: "24232" - prometheus.io/scrape: "true" - labels: - app: {{ .Values.observability.name }} - component: {{ .Values.observability.name }} - chart: {{ template "artifactory-ha.chart" . }} - heritage: {{ .Release.Service }} - release: {{ .Values.global.prometheus.release_name }} - spec: - type: ClusterIP - ports: - - name: metricsob - port: 24232 - targetPort: 8082 - protocol: TCP - selector: - app: {{ template "artifactory-ha.name" . }} - component: {{ template "artifactory-ha.name" . }} - --- - apiVersion: monitoring.coreos.com/v1 - kind: ServiceMonitor - metadata: - name: servicemonitor-{{ .Values.observability.name }} - labels: - metrics: jfrog - app: {{ .Values.observability.name }} - chart: {{ template "artifactory-ha.chart" . }} - heritage: {{ .Release.Service }} - release: {{ .Values.global.prometheus.release_name }} - spec: - selector: - matchLabels: - app: {{ .Values.observability.name }} - component: {{ .Values.observability.name }} - endpoints: - - port: metricsob - scheme: http - path: /observability/api/v1/metrics - interval: 15s - bearerTokenSecret: - key: token - name: jfrog-admin-token \ No newline at end of file diff --git a/helm/artifactory-values.yaml b/helm/artifactory-values.yaml index 4d006f8..6f929d6 100644 --- a/helm/artifactory-values.yaml +++ b/helm/artifactory-values.yaml @@ -33,7 +33,7 @@ artifactory: - name: JF_PRODUCT_DATA_INTERNAL value: {{ .Values.artifactory.persistence.mountPath }} - name: LOKI_URL - value: http://loki.{{ .Release.Namespace }}:3100 + value: http://loki.{{ .Release.Namespace }}.svc.cluster.local:3100 - name: FLUENTD_CONF value: ../../../..{{ .Values.artifactory.persistence.mountPath }}/etc/fluentd/fluentd.conf ## Allows to add additional kubernetes resources diff --git a/helm/loki-values.yaml b/helm/loki-values.yaml index e09ebdb..d5bd0a7 100644 --- a/helm/loki-values.yaml +++ b/helm/loki-values.yaml @@ -1,11 +1,88 @@ -# This loki-values is only compatible with loki 2.x. -# For loki 3.x helm values please visit https://grafana.com/docs/loki/latest/setup/install/helm/ -deploymentMode: SingleBinary loki: - commonConfig: - replication_factor: 1 + # Storage configuration required by Helm chart templates storage: - type: 'filesystem' - auth_enabled: false + type: filesystem + bucketNames: + chunks: chunks + ruler: ruler + admin: admin + + # Use structuredConfig instead of config block for better maintainability + structuredConfig: + auth_enabled: false + common: + path_prefix: /var/loki + replication_factor: 1 + storage: + filesystem: + chunks_directory: /var/loki/chunks + rules_directory: /var/loki/rules + schema_config: + configs: + - from: "2024-04-01" + store: tsdb + object_store: filesystem + schema: v13 + index: + prefix: loki_index_ + period: 24h + storage_config: + filesystem: + directory: /var/loki/chunks + pattern_ingester: + enabled: true + limits_config: + allow_structured_metadata: false + volume_enabled: true + ruler: + enable_api: true + storage: + type: local + local: + directory: /var/loki/rules + server: + http_listen_port: 3100 + grpc_listen_port: 9095 + memberlist: + join_members: + - loki-memberlist + +# Disable minio to use filesystem storage +minio: + enabled: false + +deploymentMode: SingleBinary + singleBinary: - replicas: 1 \ No newline at end of file + replicas: 1 + persistence: + enabled: true + size: 10Gi + # storageClassName: "gp2" # Uncomment and set your storage class if needed + +# Zero out replica counts of other deployment modes +backend: + replicas: 0 +read: + replicas: 0 +write: + replicas: 0 + +ingester: + replicas: 0 +querier: + replicas: 0 +queryFrontend: + replicas: 0 +queryScheduler: + replicas: 0 +distributor: + replicas: 0 +compactor: + replicas: 0 +indexGateway: + replicas: 0 +bloomCompactor: + replicas: 0 +bloomGateway: + replicas: 0 \ No newline at end of file diff --git a/helm/prometheus-grafana-values.yaml b/helm/prometheus-grafana-values.yaml new file mode 100644 index 0000000..298f984 --- /dev/null +++ b/helm/prometheus-grafana-values.yaml @@ -0,0 +1,29 @@ +# kube-prometheus-stack Configuration with Grafana and Prometheus Persistence +prometheus: + prometheusSpec: + # Enable persistent storage for Prometheus + storageSpec: + volumeClaimTemplate: + spec: + # storageClassName: "gp2" # Uncomment and set your storage class if needed + accessModes: ["ReadWriteOnce"] + resources: + requests: + storage: 50Gi + + # Optional: Set retention period + retention: 30d + retentionSize: 45GB + +grafana: + # Enable persistent storage for Grafana + persistence: + enabled: true + size: 10Gi + # storageClassName: "gp2" # Uncomment and set your storage class if needed + accessModes: + - ReadWriteOnce + + # Admin credentials + adminUser: admin + adminPassword: prom-operator # Change this to a secure password diff --git a/helm/xray-values.yaml b/helm/xray-values.yaml index 60a9fae..a50c9f5 100644 --- a/helm/xray-values.yaml +++ b/helm/xray-values.yaml @@ -12,7 +12,23 @@ xray: enabled: true replicaCount: 1 postgresql: - postgresqlPassword: xray + auth: + password: xray + + ## If Xray DB sync is needed, increase the resources to the following values: + # primary: + # resources: + # requests: + # memory: 2Gi + # cpu: 1 + # limits: + # memory: 16Gi + # cpu: 6 + # extendedConfiguration: | + # max_connections = 500 + # listen_addresses = '*' + # max_wal_size = '4GB' + rabbitmq-ha: enabled: false rabbitmq: @@ -48,7 +64,7 @@ common: - name: FLUENTD_CONF value: ../../../..{{ .Values.xray.persistence.mountPath }}/etc/fluentd/fluentd.conf - name: LOKI_URL - value: http://loki.{{ .Release.Namespace }}:3100 + value: http://loki.{{ .Release.Namespace }}.svc.cluster.local:3100 ## Allows to add additional kubernetes resources ## Use --- as a separator between multiple resources ## For an example, refer - https://github.com/jfrog/log-analytics-prometheus diff --git a/images/DashboardAddition-01.jpeg b/images/DashboardAddition-01.jpeg deleted file mode 100644 index 55a8bf5..0000000 Binary files a/images/DashboardAddition-01.jpeg and /dev/null differ diff --git a/images/DashboardAddition-02.jpeg b/images/DashboardAddition-02.jpeg deleted file mode 100644 index 5881494..0000000 Binary files a/images/DashboardAddition-02.jpeg and /dev/null differ diff --git a/images/DataSourceAddition-01.jpeg b/images/DataSourceAddition-01.jpeg deleted file mode 100644 index eabe40b..0000000 Binary files a/images/DataSourceAddition-01.jpeg and /dev/null differ diff --git a/images/DataSourceAddition-02.jpeg b/images/DataSourceAddition-02.jpeg deleted file mode 100644 index 0ef330d..0000000 Binary files a/images/DataSourceAddition-02.jpeg and /dev/null differ diff --git a/images/DataSourceAddition-03.jpeg b/images/DataSourceAddition-03.jpeg deleted file mode 100644 index 3af3673..0000000 Binary files a/images/DataSourceAddition-03.jpeg and /dev/null differ diff --git a/images/DataSourceAddition-04.jpeg b/images/DataSourceAddition-04.jpeg deleted file mode 100644 index 16cb66b..0000000 Binary files a/images/DataSourceAddition-04.jpeg and /dev/null differ diff --git a/images/ServiceDiscovery.jpeg b/images/ServiceDiscovery.jpeg deleted file mode 100644 index c5b4cef..0000000 Binary files a/images/ServiceDiscovery.jpeg and /dev/null differ