Skip to content

feat(preprod): Add build distribution CSV export endpoint#117539

Merged
jamieQ merged 7 commits into
masterfrom
feat/build-distribution-csv-export
Jun 16, 2026
Merged

feat(preprod): Add build distribution CSV export endpoint#117539
jamieQ merged 7 commits into
masterfrom
feat/build-distribution-csv-export

Conversation

@jamieQ

@jamieQ jamieQ commented Jun 12, 2026

Copy link
Copy Markdown
Member

Backend half of EME-1035: a streaming CSV export of build distribution stats.

GET /organizations/{org}/builds-export/ streams a CSV of the installable builds matching the current query, project, and date-range filters. It is scoped to the build-distribution view (non-snapshot builds) and ignores any display param. It shares a new filtered_builds_queryset() helper with the builds list endpoint so filtering can't drift, then narrows to installable builds — reusing the same is_installable_artifact() check the list uses for its is_installable flag, so the export matches what the UI labels installable — and streams through CsvResponder without the list's heavy per-row work (base-artifact lookup, size metrics, snapshot derivation).

Exports over 10,000 rows are rejected with a 400 rather than silently truncating. Cells are escaped against formula injection and the endpoint is rate-limited.

Columns: app_name, project_slug, artifact_id, app_id, build_configuration, version, platform, install_groups, upload_date, download_count — mirrors the original Emerge report, with project_slug inserted after app_name. build_configuration is the configuration name (e.g. "Debug"/"Release"), blank when the build has none.

Corresponding frontend changes to come in a subsequent PR.

Refs EME-1035

@linear-code

linear-code Bot commented Jun 12, 2026

Copy link
Copy Markdown

EME-1035

@github-actions github-actions Bot added Scope: Backend Automatically applied to PRs that change backend components Scope: Frontend Automatically applied to PRs that change frontend components labels Jun 12, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🚨 Warning: This pull request contains Frontend and Backend changes!

It's discouraged to make changes to Sentry's Frontend and Backend in a single pull request. The Frontend and Backend are not atomically deployed. If the changes are interdependent of each other, they must be separated into two pull requests and be made forward or backwards compatible, such that the Backend or Frontend can be safely deployed independently.

Have questions? Please ask in the #discuss-dev-infra channel.

Comment thread src/sentry/preprod/api/endpoints/builds_export.py Fixed
Comment thread src/sentry/preprod/api/endpoints/builds_export.py Outdated
Comment thread src/sentry/preprod/api/endpoints/builds_export.py Fixed
Comment thread src/sentry/preprod/api/endpoints/builds_export.py Outdated
Comment thread src/sentry/preprod/api/endpoints/builds_export.py Outdated
InvalidSearchQuery: if the query string is invalid.
"""
queryset = queryset_for_query(query, organization)
queryset = queryset.filter(date_added__gte=get_size_retention_cutoff(organization))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps we can rename this to "get_preprod_retention_cutoff` since snapshots and size retention rules will be the same and we're using them for both potential cases here

Image

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll follow up with a rename in another PR since it's used in a number of spots

jamieQ and others added 5 commits June 16, 2026 11:48
Add GET /organizations/{org}/builds/export/, a streaming CSV export of
build distribution stats. It reuses the builds list filters (query,
display, project, date range) through a new shared filtered_builds_queryset
helper, so the export always matches the on-screen list.

The endpoint streams rows via CsvResponder with a slim serializer that
reads only the columns it needs, skipping the heavy per-row transform the
list endpoint performs. It rejects exports over 10,000 rows (matching the
original Emerge limit) instead of silently truncating, guards against CSV
formula injection, and emits install_groups as a JSON array.

Refs EME-1035
Co-Authored-By: Claude <noreply@anthropic.com>
Add project_slug and build_configuration columns and order all columns to
mirror the original Emerge build-distribution report (project_slug after
app_name, install_groups retained). build_configuration is the build
configuration name, e.g. Debug/Release.

Restrict the export to installable builds using the same
is_installable_artifact() check the /builds/ list uses for its is_installable
flag, so the export matches what the UI labels installable. Because only
installable builds are exported, each download_count is the build's real total,
resolving the earlier raw-vs-gated question (confirmed with product).

Switch the tests to name-based column lookups so column reordering can't
silently break positional assertions.

Refs EME-1035
Co-Authored-By: Claude <noreply@anthropic.com>
Neutralize spreadsheet formula injection based on the first non-whitespace
character, so values with leading whitespace, tab, or CR before a formula
trigger (which some apps strip before evaluating) are quoted too.

Scope the export to the build-distribution row set: always filter to
non-snapshot builds and ignore the display query param. The export columns are
distribution stats, so size/snapshot views would be misleading, and this also
removes a snapshot-join duplicate-row edge case.

Refs EME-1035
Co-Authored-By: Claude <noreply@anthropic.com>
@jamieQ jamieQ force-pushed the feat/build-distribution-csv-export branch from 927e499 to add8847 Compare June 16, 2026 16:49
@jamieQ jamieQ marked this pull request as ready for review June 16, 2026 17:59
@jamieQ jamieQ requested review from a team as code owners June 16, 2026 17:59
Comment on lines +143 to +145
queryset = queryset.select_related(
"mobile_app_info", "project", "build_configuration"
).order_by("-date_added")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The CSV export for builds will always report a download_count of 0 because the database query is missing the necessary annotation to calculate this value.
Severity: MEDIUM

Suggested Fix

Add the .annotate_download_count() method to the queryset chain in src/sentry/preprod/api/endpoints/builds_export.py between lines 143 and 145. This will ensure the download_count is correctly calculated and included in the exported CSV.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: src/sentry/preprod/api/endpoints/builds_export.py#L143-L145

Potential issue: The CSV export for builds consistently reports a `download_count` of 0
for all artifacts. This occurs because the database queryset at lines 143-145 does not
include the `.annotate_download_count()` method. As a result, the `download_count`
attribute is not present on the model instances being iterated over. The code then falls
back to `getattr(item, "download_count", 0)`, which returns the default value of 0,
leading to incorrect data in the exported file.

Did we get this right? 👍 / 👎 to inform future reviews.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BuildsExportEndpoint.get() builds its queryset via filtered_builds_queryset(), which traces back to the annotation:

builds_export.py:120 → filtered_builds_queryset(...)
builds_query.py:37 → queryset_for_query(query, organization)
artifact_search.py:222 → apply_filters(_base_searchable_queryset(), ...)
artifact_search.py:191-198 → _base_searchable_queryset() calls .annotate_download_count()

@jamieQ jamieQ merged commit 291120a into master Jun 16, 2026
65 checks passed
@jamieQ jamieQ deleted the feat/build-distribution-csv-export branch June 16, 2026 20:04
vgrozdanic pushed a commit that referenced this pull request Jun 16, 2026
Backend half of EME-1035: a streaming CSV export of build distribution
stats.

`GET /organizations/{org}/builds-export/` streams a CSV of the
installable builds matching the current `query`, `project`, and
date-range filters. It is scoped to the build-distribution view
(non-snapshot builds) and ignores any `display` param. It shares a new
`filtered_builds_queryset()` helper with the builds list endpoint so
filtering can't drift, then narrows to installable builds — reusing the
same `is_installable_artifact()` check the list uses for its
`is_installable` flag, so the export matches what the UI labels
installable — and streams through `CsvResponder` without the list's
heavy per-row work (base-artifact lookup, size metrics, snapshot
derivation).

Exports over 10,000 rows are rejected with a 400 rather than silently
truncating. Cells are escaped against formula injection and the endpoint
is rate-limited.

Columns: `app_name, project_slug, artifact_id, app_id,
build_configuration, version, platform, install_groups, upload_date,
download_count` — mirrors the original Emerge report, with
`project_slug` inserted after `app_name`. `build_configuration` is the
configuration name (e.g. "Debug"/"Release"), blank when the build has
none.

Corresponding frontend changes to come in a subsequent PR.

Refs EME-1035

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: getsantry[bot] <66042841+getsantry[bot]@users.noreply.github.com>
billyvg pushed a commit that referenced this pull request Jun 17, 2026
Backend half of EME-1035: a streaming CSV export of build distribution
stats.

`GET /organizations/{org}/builds-export/` streams a CSV of the
installable builds matching the current `query`, `project`, and
date-range filters. It is scoped to the build-distribution view
(non-snapshot builds) and ignores any `display` param. It shares a new
`filtered_builds_queryset()` helper with the builds list endpoint so
filtering can't drift, then narrows to installable builds — reusing the
same `is_installable_artifact()` check the list uses for its
`is_installable` flag, so the export matches what the UI labels
installable — and streams through `CsvResponder` without the list's
heavy per-row work (base-artifact lookup, size metrics, snapshot
derivation).

Exports over 10,000 rows are rejected with a 400 rather than silently
truncating. Cells are escaped against formula injection and the endpoint
is rate-limited.

Columns: `app_name, project_slug, artifact_id, app_id,
build_configuration, version, platform, install_groups, upload_date,
download_count` — mirrors the original Emerge report, with
`project_slug` inserted after `app_name`. `build_configuration` is the
configuration name (e.g. "Debug"/"Release"), blank when the build has
none.

Corresponding frontend changes to come in a subsequent PR.

Refs EME-1035

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: getsantry[bot] <66042841+getsantry[bot]@users.noreply.github.com>
jamieQ added a commit that referenced this pull request Jun 18, 2026
…17927)

Adds a "Download CSV" button to the mobile-builds distribution view
(shown only on the distribution display) that exports build distribution
stats.

It uses a native browser download — an anchor pointed at the
`/builds-export/` endpoint — so the browser saves the file from the
response's `Content-Disposition`. This is the
simplest fit for a low-volume, row-capped export. A small, unit-tested
`getBuildsExportHref` helper builds the URL from the list filters.

The backend `/builds-export/` endpoint landed separately in #117539
(frontend and backend ship as separate PRs).

Refs EME-1035

---------

Co-authored-by: Claude <noreply@anthropic.com>
jamieQ added a commit that referenced this pull request Jun 23, 2026
The builds list and CSV export filtered every view by the size-analysis
retention cutoff, so build-distribution builds could drop off once past
that window even while still within installable-build retention.
filtered_builds_queryset now picks the cutoff per display: distribution
uses installable-build retention, size and snapshot use size-analysis.

Refs #117539
Co-Authored-By: Claude <noreply@anthropic.com>
sehr-m pushed a commit that referenced this pull request Jun 23, 2026
Backend half of EME-1035: a streaming CSV export of build distribution
stats.

`GET /organizations/{org}/builds-export/` streams a CSV of the
installable builds matching the current `query`, `project`, and
date-range filters. It is scoped to the build-distribution view
(non-snapshot builds) and ignores any `display` param. It shares a new
`filtered_builds_queryset()` helper with the builds list endpoint so
filtering can't drift, then narrows to installable builds — reusing the
same `is_installable_artifact()` check the list uses for its
`is_installable` flag, so the export matches what the UI labels
installable — and streams through `CsvResponder` without the list's
heavy per-row work (base-artifact lookup, size metrics, snapshot
derivation).

Exports over 10,000 rows are rejected with a 400 rather than silently
truncating. Cells are escaped against formula injection and the endpoint
is rate-limited.

Columns: `app_name, project_slug, artifact_id, app_id,
build_configuration, version, platform, install_groups, upload_date,
download_count` — mirrors the original Emerge report, with
`project_slug` inserted after `app_name`. `build_configuration` is the
configuration name (e.g. "Debug"/"Release"), blank when the build has
none.

Corresponding frontend changes to come in a subsequent PR.

Refs EME-1035

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: getsantry[bot] <66042841+getsantry[bot]@users.noreply.github.com>
sehr-m pushed a commit that referenced this pull request Jun 23, 2026
…17927)

Adds a "Download CSV" button to the mobile-builds distribution view
(shown only on the distribution display) that exports build distribution
stats.

It uses a native browser download — an anchor pointed at the
`/builds-export/` endpoint — so the browser saves the file from the
response's `Content-Disposition`. This is the
simplest fit for a low-volume, row-capped export. A small, unit-tested
`getBuildsExportHref` helper builds the URL from the list filters.

The backend `/builds-export/` endpoint landed separately in #117539
(frontend and backend ship as separate PRs).

Refs EME-1035

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components Scope: Frontend Automatically applied to PRs that change frontend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants