Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backport PRs from main as of March 20 #427

Merged
merged 34 commits into from
Mar 21, 2025
Merged

Conversation

nategraf
Copy link
Member

Copy link

github-actions bot commented Mar 20, 2025

🚀 Documentation Preview

Deployment URL: https://boundless-documentation-2mkc8lwdv-risczero.vercel.app

Updated at: 2025-03-21 01:10:08 UTC

@nategraf nategraf force-pushed the victor/backport-bananza branch from d127ff3 to 578254b Compare March 20, 2025 23:56
@nategraf nategraf enabled auto-merge (rebase) March 20, 2025 23:56
@nategraf nategraf force-pushed the victor/backport-bananza branch 2 times, most recently from 10adedc to 5157510 Compare March 21, 2025 00:44
derrix060 and others added 7 commits March 20, 2025 17:45
Co-authored-by: Victor Snyder-Graf <[email protected]>
Since
6646cdb#diff-3386414d78b149270fe6c73ec085c5e8c565c1caa4e7951de5ea45b1c4e347b8R3,
the script now requires the job as an argument instead of waiting for
stdin

Co-authored-by: Victor Snyder-Graf <[email protected]>
I’ve added a check for the JOB variable to ensure it’s passed as an
argument when the script is run. If it’s missing, a helpful error
message will be displayed instead of the script crashing.

This improves error handling and makes the script more user-friendly.

Co-authored-by: Victor Snyder-Graf <[email protected]>
)

Closes #390 

- Adds new public method to `boundless_market` -
`with_stake_balance_alert` which takes a warning and error threshold to
alert on
- Adds new config fields to broker to set these on the market in
`order_monitor`
- Adds to existing test to check that market is correctly alerting when
stake balance goes below thresholds
The error would panic and crash the instance, which was indefinitely
irrecoverable. This rejects that invalid case, such that the node will
continue.

The part I am unsure about is the sanity check within the executor for
keccak requests. This changes to error log and skip those requests, and
didn't seem like a clean mechanism to fail to proof as a whole, but
perhaps this should still go through to fail with the prove agent so
that it errors and then perhaps the tasks get cancelled through taskdb?
Haven't looked that in depth at the implications yet.
We now favor "Explorer" over "Indexer" in all comms
@nategraf nategraf force-pushed the victor/backport-bananza branch from 5157510 to 53a5061 Compare March 21, 2025 00:56
capossele and others added 5 commits March 20, 2025 18:03
It also drops all the .hex artifacts in favor of a single bytecode.rs

---------

Co-authored-by: Wolfgang Welz <[email protected]>
Co-authored-by: Victor Snyder-Graf <[email protected]>
Automatic testing of Bento when the bento/ subdirectory is modified.

---------

Co-authored-by: Austin Abell <[email protected]>
Co-authored-by: Victor Graf <[email protected]>
Partially handles BM-616

Co-authored-by: Richard Howard <[email protected]>
Motivated by a few things:
1. Env var naming was inconsistent
(`BOUNDLESS_MARKET_ADDR`/`BOUNDLESS_MARKET_ADDRESS` for ex) which led to
many footguns
2. makefile, justfile, and scripts meant a confusing UX
3. Confusingly named env files, changes to env files with private env
not hidden from git (annoying)
4. Env files had default values that were not clear what they were
referencing (broker `.env-compose` was not pointing to correct localnet
or testnet contract addresses)

Defaulting to using `just` for the commands, as most flexible for what
was needed and readability. Not opinionated about commands or their
names, if you feel like something should be different let me know. I
tried to organize and remove unnecessary commands where possible.
austinabell and others added 17 commits March 20, 2025 18:04
)

Closes issue: BM-535 Alert on low balance

Adds a new provider layer which can be configured to log warnings and
errors on balance thresholds. Implementing this as a layer on the RPC
Provider means all calls anywhere in the broker will have these checks
added automatically.

- Adds layer implementation
- Adds new config fields to set threshold
- Adds layer when initializing the broker provider
Partially handles BM-616
We've seen transient errors when calling the Bonsai API. This adds
retries to all API calls (previously we only retried calls to check job
status). This also adds retries to the entire proving workflow, to
better capture the case where are able to kick off a proving job and it
terminates, but it terminates with a retryable internal server error.

This should help with:
#410

---------

Co-authored-by: Willem Olding <[email protected]>
Co-authored-by: Victor Snyder-Graf <[email protected]>
Co-authored-by: Rami Lukata <[email protected]>
Co-authored-by: Angelo Capossele <[email protected]>
CI wasn't running any rust test on CI due to a wrong `ignored` keyword
Solves part of BM-598 in that we need multiple ways to limit the
capacity of orders taken on by the broker, and limiting the amount of
current locks is not a perfect solution but a reasonable mechanism that
would be useful in the short term or to limit risk.

Will open PR for review once more thoroughly tested and will try to make
sure no edge cases and minimimal overhead. Basically how this works is
querying the database for orders that have been selected to be locked
but not fulfilled, and then spawning tasks to handle pricing each. The
capacity is calculated on an interval, every 10s, where the config and
db will be checked, which will pick up when orders are fulfilled and
more capacity available. Whenever pricing successful, it will reduce
from the capacity to avoid extra tasks from being marked as locked.

May also refactor the design to do this, as this doesn't handle the case
where the order is planned to be locked in the future very well. The
issue with having the concurrent locks be monitored in the order monitor
(which actually calls lock) is that it isn't clear when the broker needs
to lock the order in by to make sure that it will complete it in time,
because that is done in pricing.

---------

Co-authored-by: Victor Snyder-Graf <[email protected]>
We've been seeing periodic failures on RPC requests made by the client
in order stream. In particular, the balance check is failing sometimes
which results in a 500 error being sent to the client. In order to
handle this more gracefully more often, this PR adds a retry layer to
the RPC client used by order-stream.
This update contains the default selector for a FakeReceipt's seal as
0xFFFFFFFF
Pointing at #413 and has a similar implementation, but can afford to be
a bit more naive about how it estimates the proving time. This is not an
ideal solution, but is helpful for the time being for issues like #351
where broker is overlocking orders.
Closes #388 

Uses reqwest middleware to add a HTTP-aware cache. See
https://docs.rs/http-cache-reqwest/latest/http_cache_reqwest/index.html.
This is used only for caching requests for assessor and set-builder
images at this time.

- Adds caching to requests to retrieve assessor and set builder images
- - Can trivially also add it to user images but this may need more
discussion
- Add new command line flag `--cache-images` to broker
- Adds new command line arg `--cache-dir` to broker (default
`/tmp/broker_cache/`)

---------

Co-authored-by: Austin Abell <[email protected]>
Co-authored-by: Cohan <[email protected]>
Co-authored-by: Sasha <[email protected]>
Co-authored-by: pote.eth <[email protected]>
Co-authored-by: Parker Thompson <[email protected]>
Co-authored-by: Victor Graf <[email protected]>
Co-authored-by: Rami Lukata <[email protected]>
Co-authored-by: Richard Howard <[email protected]>
Co-authored-by: Angelo Capossele <[email protected]>
@nategraf nategraf force-pushed the victor/backport-bananza branch from 53a5061 to f58d8a6 Compare March 21, 2025 01:07
@nategraf
Copy link
Member Author

nategraf commented Mar 21, 2025

Its ok for Release / assessor-elf to fail here. There are no changes to the assessor here and we are not going to deploy a new one.

Copy link
Contributor

@austinabell austinabell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

broker seems to be running fine with the backport 👍

@nategraf nategraf merged commit c357c5a into release-0.6.0 Mar 21, 2025
15 of 16 checks passed
@nategraf nategraf deleted the victor/backport-bananza branch March 21, 2025 01:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants