feat(preprod): Add build distribution CSV export endpoint#117539
Conversation
|
🚨 Warning: This pull request contains Frontend and Backend changes! It's discouraged to make changes to Sentry's Frontend and Backend in a single pull request. The Frontend and Backend are not atomically deployed. If the changes are interdependent of each other, they must be separated into two pull requests and be made forward or backwards compatible, such that the Backend or Frontend can be safely deployed independently. Have questions? Please ask in the |
df40292 to
d1debae
Compare
| InvalidSearchQuery: if the query string is invalid. | ||
| """ | ||
| queryset = queryset_for_query(query, organization) | ||
| queryset = queryset.filter(date_added__gte=get_size_retention_cutoff(organization)) |
There was a problem hiding this comment.
I'll follow up with a rename in another PR since it's used in a number of spots
0c545cf to
328774e
Compare
Add GET /organizations/{org}/builds/export/, a streaming CSV export of
build distribution stats. It reuses the builds list filters (query,
display, project, date range) through a new shared filtered_builds_queryset
helper, so the export always matches the on-screen list.
The endpoint streams rows via CsvResponder with a slim serializer that
reads only the columns it needs, skipping the heavy per-row transform the
list endpoint performs. It rejects exports over 10,000 rows (matching the
original Emerge limit) instead of silently truncating, guards against CSV
formula injection, and emits install_groups as a JSON array.
Refs EME-1035
Co-Authored-By: Claude <noreply@anthropic.com>
Add project_slug and build_configuration columns and order all columns to mirror the original Emerge build-distribution report (project_slug after app_name, install_groups retained). build_configuration is the build configuration name, e.g. Debug/Release. Restrict the export to installable builds using the same is_installable_artifact() check the /builds/ list uses for its is_installable flag, so the export matches what the UI labels installable. Because only installable builds are exported, each download_count is the build's real total, resolving the earlier raw-vs-gated question (confirmed with product). Switch the tests to name-based column lookups so column reordering can't silently break positional assertions. Refs EME-1035 Co-Authored-By: Claude <noreply@anthropic.com>
Neutralize spreadsheet formula injection based on the first non-whitespace character, so values with leading whitespace, tab, or CR before a formula trigger (which some apps strip before evaluating) are quoted too. Scope the export to the build-distribution row set: always filter to non-snapshot builds and ignore the display query param. The export columns are distribution stats, so size/snapshot views would be misleading, and this also removes a snapshot-join duplicate-row edge case. Refs EME-1035 Co-Authored-By: Claude <noreply@anthropic.com>
927e499 to
add8847
Compare
| queryset = queryset.select_related( | ||
| "mobile_app_info", "project", "build_configuration" | ||
| ).order_by("-date_added") |
There was a problem hiding this comment.
Bug: The CSV export for builds will always report a download_count of 0 because the database query is missing the necessary annotation to calculate this value.
Severity: MEDIUM
Suggested Fix
Add the .annotate_download_count() method to the queryset chain in src/sentry/preprod/api/endpoints/builds_export.py between lines 143 and 145. This will ensure the download_count is correctly calculated and included in the exported CSV.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: src/sentry/preprod/api/endpoints/builds_export.py#L143-L145
Potential issue: The CSV export for builds consistently reports a `download_count` of 0
for all artifacts. This occurs because the database queryset at lines 143-145 does not
include the `.annotate_download_count()` method. As a result, the `download_count`
attribute is not present on the model instances being iterated over. The code then falls
back to `getattr(item, "download_count", 0)`, which returns the default value of 0,
leading to incorrect data in the exported file.
Did we get this right? 👍 / 👎 to inform future reviews.
There was a problem hiding this comment.
BuildsExportEndpoint.get() builds its queryset via filtered_builds_queryset(), which traces back to the annotation:
builds_export.py:120 → filtered_builds_queryset(...)
builds_query.py:37 → queryset_for_query(query, organization)
artifact_search.py:222 → apply_filters(_base_searchable_queryset(), ...)
artifact_search.py:191-198 → _base_searchable_queryset() calls .annotate_download_count()
Backend half of EME-1035: a streaming CSV export of build distribution
stats.
`GET /organizations/{org}/builds-export/` streams a CSV of the
installable builds matching the current `query`, `project`, and
date-range filters. It is scoped to the build-distribution view
(non-snapshot builds) and ignores any `display` param. It shares a new
`filtered_builds_queryset()` helper with the builds list endpoint so
filtering can't drift, then narrows to installable builds — reusing the
same `is_installable_artifact()` check the list uses for its
`is_installable` flag, so the export matches what the UI labels
installable — and streams through `CsvResponder` without the list's
heavy per-row work (base-artifact lookup, size metrics, snapshot
derivation).
Exports over 10,000 rows are rejected with a 400 rather than silently
truncating. Cells are escaped against formula injection and the endpoint
is rate-limited.
Columns: `app_name, project_slug, artifact_id, app_id,
build_configuration, version, platform, install_groups, upload_date,
download_count` — mirrors the original Emerge report, with
`project_slug` inserted after `app_name`. `build_configuration` is the
configuration name (e.g. "Debug"/"Release"), blank when the build has
none.
Corresponding frontend changes to come in a subsequent PR.
Refs EME-1035
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: getsantry[bot] <66042841+getsantry[bot]@users.noreply.github.com>
Backend half of EME-1035: a streaming CSV export of build distribution
stats.
`GET /organizations/{org}/builds-export/` streams a CSV of the
installable builds matching the current `query`, `project`, and
date-range filters. It is scoped to the build-distribution view
(non-snapshot builds) and ignores any `display` param. It shares a new
`filtered_builds_queryset()` helper with the builds list endpoint so
filtering can't drift, then narrows to installable builds — reusing the
same `is_installable_artifact()` check the list uses for its
`is_installable` flag, so the export matches what the UI labels
installable — and streams through `CsvResponder` without the list's
heavy per-row work (base-artifact lookup, size metrics, snapshot
derivation).
Exports over 10,000 rows are rejected with a 400 rather than silently
truncating. Cells are escaped against formula injection and the endpoint
is rate-limited.
Columns: `app_name, project_slug, artifact_id, app_id,
build_configuration, version, platform, install_groups, upload_date,
download_count` — mirrors the original Emerge report, with
`project_slug` inserted after `app_name`. `build_configuration` is the
configuration name (e.g. "Debug"/"Release"), blank when the build has
none.
Corresponding frontend changes to come in a subsequent PR.
Refs EME-1035
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: getsantry[bot] <66042841+getsantry[bot]@users.noreply.github.com>
…17927) Adds a "Download CSV" button to the mobile-builds distribution view (shown only on the distribution display) that exports build distribution stats. It uses a native browser download — an anchor pointed at the `/builds-export/` endpoint — so the browser saves the file from the response's `Content-Disposition`. This is the simplest fit for a low-volume, row-capped export. A small, unit-tested `getBuildsExportHref` helper builds the URL from the list filters. The backend `/builds-export/` endpoint landed separately in #117539 (frontend and backend ship as separate PRs). Refs EME-1035 --------- Co-authored-by: Claude <noreply@anthropic.com>
The builds list and CSV export filtered every view by the size-analysis retention cutoff, so build-distribution builds could drop off once past that window even while still within installable-build retention. filtered_builds_queryset now picks the cutoff per display: distribution uses installable-build retention, size and snapshot use size-analysis. Refs #117539 Co-Authored-By: Claude <noreply@anthropic.com>
Backend half of EME-1035: a streaming CSV export of build distribution
stats.
`GET /organizations/{org}/builds-export/` streams a CSV of the
installable builds matching the current `query`, `project`, and
date-range filters. It is scoped to the build-distribution view
(non-snapshot builds) and ignores any `display` param. It shares a new
`filtered_builds_queryset()` helper with the builds list endpoint so
filtering can't drift, then narrows to installable builds — reusing the
same `is_installable_artifact()` check the list uses for its
`is_installable` flag, so the export matches what the UI labels
installable — and streams through `CsvResponder` without the list's
heavy per-row work (base-artifact lookup, size metrics, snapshot
derivation).
Exports over 10,000 rows are rejected with a 400 rather than silently
truncating. Cells are escaped against formula injection and the endpoint
is rate-limited.
Columns: `app_name, project_slug, artifact_id, app_id,
build_configuration, version, platform, install_groups, upload_date,
download_count` — mirrors the original Emerge report, with
`project_slug` inserted after `app_name`. `build_configuration` is the
configuration name (e.g. "Debug"/"Release"), blank when the build has
none.
Corresponding frontend changes to come in a subsequent PR.
Refs EME-1035
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: getsantry[bot] <66042841+getsantry[bot]@users.noreply.github.com>
…17927) Adds a "Download CSV" button to the mobile-builds distribution view (shown only on the distribution display) that exports build distribution stats. It uses a native browser download — an anchor pointed at the `/builds-export/` endpoint — so the browser saves the file from the response's `Content-Disposition`. This is the simplest fit for a low-volume, row-capped export. A small, unit-tested `getBuildsExportHref` helper builds the URL from the list filters. The backend `/builds-export/` endpoint landed separately in #117539 (frontend and backend ship as separate PRs). Refs EME-1035 --------- Co-authored-by: Claude <noreply@anthropic.com>

Backend half of EME-1035: a streaming CSV export of build distribution stats.
GET /organizations/{org}/builds-export/streams a CSV of the installable builds matching the currentquery,project, and date-range filters. It is scoped to the build-distribution view (non-snapshot builds) and ignores anydisplayparam. It shares a newfiltered_builds_queryset()helper with the builds list endpoint so filtering can't drift, then narrows to installable builds — reusing the sameis_installable_artifact()check the list uses for itsis_installableflag, so the export matches what the UI labels installable — and streams throughCsvResponderwithout the list's heavy per-row work (base-artifact lookup, size metrics, snapshot derivation).Exports over 10,000 rows are rejected with a 400 rather than silently truncating. Cells are escaped against formula injection and the endpoint is rate-limited.
Columns:
app_name, project_slug, artifact_id, app_id, build_configuration, version, platform, install_groups, upload_date, download_count— mirrors the original Emerge report, withproject_sluginserted afterapp_name.build_configurationis the configuration name (e.g. "Debug"/"Release"), blank when the build has none.Corresponding frontend changes to come in a subsequent PR.
Refs EME-1035