SENTRY_EVENT_RETENTION_DAYS greater than 90 days does not work as expected.

[Mandalavandalz]

Self-Hosted Version

24.1.1

CPU Architecture

x86_64

Docker Version

24.0.2

Docker Compose Version

2.0.1

Steps to Reproduce

We are updating Sentry from 9.1.2 to the latest version and want to keep all issues. After following all the hard stops 9.1.2 -> 21.1.0 -> 21.5.0 -> 21.6.3 -> 23.6.2 -> latest and setting SENTRY_EVENT_RETENTION_DAYS=3650 in .env during the update from 9.1.2 to 21.1.0, the Sentry update script correctly processes all issues, which then go into the ClickHouse table sentry_local. When counting the table, it appears that all are in there, but they cannot be selected from the WebUI, only with 90 days back (the default value of SENTRY_EVENT_RETENTION_DAYS). From 21.1.0 -> 21.5.0 again with SENTRY_EVENT_RETENTION_DAYS=3650 in .env, the events migrate from table sentry_local to table discover_local, but only those with 90 days back and so until the latest. Is there a way to keep all issues, it has also been tried to remove the sentry-cleanup cron as written in the README ("If you do not want the cleanup cron, you can remove the sentry-cleanup service from the docker-compose.yml file.") the result is the same - you see events only with 90 days back.

Expected Result

All issues are available and can be selected back through the WebUI, not just those from 90 days ago.

Actual Result

Only the issues from 90 days ago are visible in the WebUI.

Event ID

No response

[Mandalavandalz]

Also, after the update to the current latest (v24.1.1), with existing issues from 90 days ago and SENTRY_EVENT_RETENTION_DAYS=180 in .env, issues older than 90 days cannot be selected through the WebUI again. I let Sentry work for 2 days, and after 2 days, the oldest issues, which were visible before, cannot be selected during filtering from the WebUI with "Absolute date".

[azaslavsky]

Snuba team: I may be mistaken, but looking over https://github.com/search?q=repo%3Agetsentry%2Fsnuba%20DEFAULT_RETENTION_DAYS&type=code it seems like the DEFAULT_RETENTION_DAYS env variable is not actually ingested anywhere?

[lojanp]

can you please advise me? I have a self-hosted sentry running for about 40 days. I want to change
settings for SENTRY_EVENT_RETENTION_DAYS from 90 days to 30 days in .env file. What steps do I need to take for the changes to take effect? I don't want to lose data. I use the docker compose solution.
Well thank you.

[volokluev]

Snuba only supports retention days of 30, 60, 90 days. It is technically possible to do more but there are many places in the product which make the max 90 day assumption.

Patches are welcome to try and implement this

If you want to try and patch this for your specific instance, you could try doing a cron job to update the retention_days field on your clickhouse table to be as many days as you want:

ALTER TABLE UPDATE errors_local retention_days=3650 WHERE retention_days < 3650

Note: I have not tried this but could be worth exploring