chore: output supported versions of integrations (#11372)

- Creates a `Generate Supported Integration Versions` workflow that
outputs the supported versions of integrations to a
`supported_versions_output.json` and `supported_versions_table.csv`. PR
here: #11767 and workflow
here:
https://github.com/DataDog/dd-trace-py/actions/runs/12383562860/job/34566489841
- in `scripts/freshvenvs.py`, separates the workflows for outputting the
outdated integrations (which is run in the `Generate Package Versions`
workflow), and for creating the supported version table.
- This workflow will be tied to a release, but can also be triggered
manually (via `workflow_dispatch`)

Future:
- There will be a mechanism for converting the `csv` file to the `rst`
format used by the ddtrace docs, and for generating the public datadoghq
docs (in markdown)

## Checklist
- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

## Reviewer Checklist
- [x] Reviewer has checked that all the criteria below are met 
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

Author: quinna-h

.github/workflows/generate-supported-versions.yml

@@ -0,0 +1,121 @@
+name: Generate Supported Integration Versions
+
+on:
+  workflow_dispatch: # can be triggered manually
+
+jobs:
+  generate-supported-versions:
+    name: Generate supported integration versions
+    runs-on: ubuntu-22.04
+    permissions:
+      actions: read
+      contents: write
+      pull-requests: write
+
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          persist-credentials: false
+
+      - name: Setup Python 3.7
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.7"
+
+      - name: Setup Python 3.8
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.8"
+
+      - name: Setup Python 3.9
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.9"
+
+      - name: Setup Python 3.10
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.10"
+
+      - name: Setup Python 3.11
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
+      - name: Setup Python 3.12
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+    
+      - name: Setup Python 3.13
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.13"
+
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v2
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v3
+
+      - name: Install system dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y libmariadb-dev
+
+      - name: Install Python dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install packaging
+          pip install requests
+          pip install riot==0.20.1
+          pip install wrapt==1.16.0
+      
+      - name: Install ddtrace
+        run: |
+          pip install -e .
+
+      - run: python scripts/freshvenvs.py generate
+      
+      - name: Generate table
+        run: python scripts/generate_table.py
+
+      - run: git diff
+      
+      - name: Create Pull Request
+        id: pr
+        uses: peter-evans/create-pull-request@v6
+        with:
+          token: ${{ secrets.GITHUB_TOKEN }}
+          branch: "update-supported-versions"
+          commit-message: "Update supported versions table"
+          delete-branch: true
+          base: main
+          title: "chore: update supported versions"
+          labels: changelog/no-changelog
+          body: |
+            Generates / updates the supported versions table for integrations.
+            This should be tied to releases, or triggered manually.
+            Workflow runs: [Generate Supported Integration Versions](https://github.com/DataDog/dd-trace-py/actions/workflows/generate-supported-versions.yml)  
+
+            ## Checklist
+            - [x] PR author has checked that all the criteria below are met
+            - The PR description includes an overview of the change
+            - The PR description articulates the motivation for the change
+            - The change includes tests OR the PR description describes a testing strategy
+            - The PR description notes risks associated with the change, if any
+            - Newly-added code is easy to change
+            - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
+            - The change includes or references documentation updates if necessary
+            - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))
+
+            ## Reviewer Checklist
+            - [ ] Reviewer has checked that all the criteria below are met
+            - Title is accurate
+            - All changes are related to the pull request's stated goal
+            - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes
+            - Testing strategy adequately addresses listed risks
+            - Newly-added code is easy to change
+            - Release note makes sense to a user of the library
+            - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
+            - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

scripts/freshvenvs.py

@@ -4,6 +4,7 @@
 from http.client import HTTPSConnection
 from io import StringIO
 import json
+from operator import itemgetter
 import os
 import pathlib
 import sys
@@ -21,7 +22,9 @@
 CONTRIB_ROOT = pathlib.Path("ddtrace/contrib")
 LATEST = ""
 
+excluded = {"coverage"}
 suite_to_package = {
+    "kafka": "confluent-kafka",
     "consul": "python-consul",
     "snowflake": "snowflake-connector-python",
     "flask_cache": "flask-caching",
@@ -30,11 +33,35 @@
     "asyncio": "pytest-asyncio",
     "sqlite3": "pysqlite3-binary",
     "grpc": "grpcio",
+    "google_generativeai": "google-generativeai",
     "psycopg2": "psycopg2-binary",
     "cassandra": "cassandra-driver",
     "rediscluster": "redis-py-cluster",
+    "dogpile_cache": "dogpile-cache",
+    "vertica": "vertica_python",
 }
 
+
+# mapping the name of the module to the name of the package (on pypi and as defined in lockfiles)
+mapping_module_to_package = {
+    "confluent_kafka": "confluent-kafka",
+    "snowflake": "snowflake-connector-python",
+    "cassandra": "cassandra-driver",
+    "rediscluster": "redis-py-cluster",
+    "vertica_python": "vertica-python",
+    "flask_cache": "flask-cache",
+    "flask_caching": "flask-caching",
+    "consul": "python-consul",
+    "grpc": "grpcio",
+    "graphql": "graphql-core",
+    "mysql": "pymysql",
+}
+
+
+supported_versions = []  # list of dicts
+pinned_packages = set()
+
+
 class Capturing(list):
     def __enter__(self):
         self._stdout = sys.stdout
@@ -77,27 +104,35 @@ def _get_riot_envs_including_any(modules: typing.Set[str]) -> typing.Set[str]:
             with open(f".riot/requirements/{item}", "r") as lockfile:
                 lockfile_content = lockfile.read()
                 for module in modules:
-                    if module in lockfile_content:
+                    if module in lockfile_content or (
+                        module in suite_to_package and suite_to_package[module] in lockfile_content
+                    ):
                         envs |= {item.split(".")[0]}
                         break
     return envs
 
 
 def _get_updatable_packages_implementing(modules: typing.Set[str]) -> typing.Set[str]:
-    """Return all packages that can be updated and have contribs implemented for them"""
+    """Return all packages have contribs implemented for them"""
     all_venvs = riotfile.venv.venvs
 
     for v in all_venvs:
         package = v.name
         if package not in modules:
             continue
         if not _venv_sets_latest_for_package(v, package):
-            modules.remove(package)
+            pinned_packages.add(package)
 
     packages = {m for m in modules if "." not in m}
     return packages
 
 
+def _get_all_modules(modules: typing.Set[str]) -> typing.Set[str]:
+    """Return all packages have contribs implemented for them"""
+    contrib_modules = {m for m in modules if "." not in m}
+    return contrib_modules
+
+
 def _get_version_extremes(package_name: str) -> typing.Tuple[Optional[str], Optional[str]]:
     """Return the (earliest, latest) supported versions of a given package"""
     with Capturing() as output:
@@ -134,16 +169,27 @@ def _get_version_extremes(package_name: str) -> typing.Tuple[Optional[str], Opti
 
 
 def _get_package_versions_from(env: str, packages: typing.Set[str]) -> typing.List[typing.Tuple[str, str]]:
-    """Return the list of package versions that are tested"""
+    """Return the list of package versions that are tested, related to the modules"""
+    # Returns [(package, version), (package, versions)]
     lockfile_content = pathlib.Path(f".riot/requirements/{env}.txt").read_text().splitlines()
     lock_packages = []
     for line in lockfile_content:
         package, _, versions = line.partition("==")
+        # remap the package -> module name
         if package in packages:
             lock_packages.append((package, versions))
+
     return lock_packages
 
 
+def _is_module_autoinstrumented(module: str) -> bool:
+    import importlib
+
+    _monkey = importlib.import_module("ddtrace._monkey")
+    PATCH_MODULES = getattr(_monkey, "PATCH_MODULES")
+
+    return module in PATCH_MODULES and PATCH_MODULES[module]
+
 def _versions_fully_cover_bounds(bounds: typing.Tuple[str, str], versions: typing.List[str]) -> bool:
     """Return whether the tested versions cover the full range of supported versions"""
     if not versions:
@@ -173,12 +219,25 @@ def _venv_sets_latest_for_package(venv: riotfile.Venv, suite_name: str) -> bool:
     return False
 
 
-def main():
-    all_required_modules = _get_integrated_modules()
-    all_required_packages = _get_updatable_packages_implementing(all_required_modules)
-    envs = _get_riot_envs_including_any(all_required_modules)
+def _get_all_used_versions(envs, packages) -> dict:
+    # Returns dict(module, set(versions)) for a venv, as defined in riotfiles.
+    all_used_versions = defaultdict(set)
+    for env in envs:
+        versions_used = _get_package_versions_from(env, packages)  # returns list of (package, versions)
+        for package, version in versions_used:
+            all_used_versions[package].add(version)
+    return all_used_versions
 
+
+def _get_version_bounds(packages) -> dict:
+    # Return dict(module: (earliest, latest)) of the module on PyPI
     bounds = dict()
+    for package in packages:
+        earliest, latest = _get_version_extremes(package)
+        bounds[package] = (earliest, latest)
+    return bounds
+
+def output_outdated_packages(all_required_packages, envs, bounds):
     for package in all_required_packages:
         earliest, latest = _get_version_extremes(package)
         bounds[package] = (earliest, latest)
@@ -194,10 +253,55 @@ def main():
         if not ordered:
             continue
         if not _versions_fully_cover_bounds(bounds[package], ordered):
-            print(
-                f"{package}: policy supports version {bounds[package][0]} through {bounds[package][1]} "
-                f"but only these versions are used: {[str(v) for v in ordered]}"
-            )
+            print(f"{package}")
+
+def generate_supported_versions(contrib_packages, all_used_versions, patched):
+    for mod in mapping_module_to_package:
+        contrib_packages.remove(mod)
+        contrib_packages.add(mapping_module_to_package[mod])
+        patched[mapping_module_to_package[mod]] = _is_module_autoinstrumented(mod)
+
+    # Generate supported versions
+    for package in contrib_packages:
+        ordered = sorted([Version(v) for v in all_used_versions[package]], reverse=True)
+        if not ordered:
+            continue
+        json_format = {
+            "integration": package,
+            "minimum_tracer_supported": str(ordered[-1]),
+            "max_tracer_supported": str(ordered[0]),
+        }
+
+        if package in pinned_packages:
+            json_format["pinned"] = "true"
+
+        if package not in patched:
+            patched[package] = _is_module_autoinstrumented(package)
+        json_format["auto-instrumented"] = patched[package]
+        supported_versions.append(json_format)
+
+    supported_versions_output = sorted(supported_versions, key=itemgetter("integration"))
+    with open("supported_versions_output.json", "w") as file:
+        json.dump(supported_versions_output, file, indent=4)
+
+def main():
+    all_required_modules = _get_integrated_modules()
+    all_required_packages = _get_updatable_packages_implementing(all_required_modules)  # these are MODULE names
+    contrib_modules = _get_all_modules(all_required_modules)
+    envs = _get_riot_envs_including_any(all_required_modules)
+    patched = {}
+
+    contrib_packages = contrib_modules
+    all_used_versions = _get_all_used_versions(envs, contrib_packages)
+    bounds = _get_version_bounds(contrib_packages)
+
+    if len(sys.argv) != 2:
+        print("usage: python scripts/freshvenvs.py <output> or <generate>")
+        return
+    if sys.argv[1] == "output":
+        output_outdated_packages(all_required_packages, envs, bounds)
+    if sys.argv[1] == "generate":
+        generate_supported_versions(contrib_packages, all_used_versions, patched)
 
 
 if __name__ == "__main__":

scripts/generate_table.py

@@ -0,0 +1,24 @@
+import csv
+import json
+
+
+print("Reading supported_versions_output.json")
+
+with open("supported_versions_output.json", "r") as json_file:
+    data = json.load(json_file)
+
+columns = ["integration", "minimum_tracer_supported", "max_tracer_supported", "auto-instrumented"]
+csv_rows = []
+
+for entry in data:
+    integration_name = entry.get("integration", "")
+    if entry.get("pinned", "").lower() == "true":
+        integration_name += " *"
+    entry["integration"] = integration_name
+    csv_rows.append({col: entry.get(col, "") for col in columns})
+
+with open("supported_versions_table.csv", "w", newline="") as csv_file:
+    print("Wrote to supported_versions_table.csv")
+    writer = csv.DictWriter(csv_file, fieldnames=columns)
+    writer.writeheader()
+    writer.writerows(csv_rows)

scripts/regenerate-riot-latest.sh

@@ -3,7 +3,7 @@ set -e
 
 DDTEST_CMD=scripts/ddtest
 
-pkgs=$(python scripts/freshvenvs.py | cut -d':' -f1)
+pkgs=$(python scripts/freshvenvs.py output)
 echo $pkgs
 
 if ! $DDTEST_CMD; then
@@ -20,7 +20,8 @@ for pkg in ${pkgs[*]}; do
         echo "No riot hashes found for pattern: $VENV_NAME"
     else
         echo "VENV_NAME=$VENV_NAME" >> $GITHUB_ENV
-        for h in ${RIOT_HASHES[@]}; do 
+        for h in ${RIOT_HASHES[@]}; do
+            echo "Removing riot lockfiles"
             rm ".riot/requirements/${h}.txt"
         done
         scripts/compile-and-prune-test-requirements