Skip to content

feat: client-side ratelimiting and auto-retrying 429s #549

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Apr 24, 2024

Conversation

adamspofford-dfinity
Copy link
Contributor

ManagementCanister::install currently does not work for very large modules due to hitting 429 after the first fifty chunks. This PR fixes that in two ways: limiting concurrent requests to 50 via semaphore, and retrying any 429s that slip through.

@adamspofford-dfinity adamspofford-dfinity requested a review from a team as a code owner April 24, 2024 19:38
@lwshang
Copy link
Contributor

lwshang commented Apr 24, 2024

Can we add some tests?

I'm aware that it is not easy to test in this case. Because the 429 http status code came from the boundary nodes. We have to test against the mainnet or some testnets instead of a local replica. It is hard. But I still insist that tests are necessary.

I saw the test using mock http. That's great!

@adamspofford-dfinity adamspofford-dfinity merged commit dac18df into main Apr 24, 2024
24 checks passed
@adamspofford-dfinity adamspofford-dfinity deleted the spofford/max-concurrent branch April 24, 2024 22:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants