docs: refresh benchmarking.md with dedicated shuffle PVC results - #2171
Merged
Conversation
github-actions
Bot
added
the
documentation
…ted numbers Refresh with the SF1000 results from the run that mounts a dedicated 1000 GiB gp3 PVC per executor pod at /data and passes --work-dir /data to the executor. This isolates shuffle writes from the shared root EBS volume, matching the Spark spark-local-dir-1 pattern already in use on the same cluster. Impact vs the prior run (same commit, same cluster shape, no PVC): - Q8: 189.30 -> 45.74 s (4.1x) - Q18: 282.79 -> 78.32 s (3.6x) - comparable-subset total (19Q): 1080 -> 722 s Changes to the doc: - Environment: record the underlying r6i.24xlarge node type (96 vCPU, 768 GiB, EBS-only) and the new dedicated Ballista PVC. - New "Executor storage" section: why the dedicated volume matters, with a short EXPLAIN ANALYZE-based explanation of the pre-PVC contention on the root EBS bandwidth. - Ballista configuration: add --work-dir, --grpc-server-max-*-message-size entries; note the 16 MiB gRPC default causes Q11/Q21/Q22 to fail. - Results: refreshed Ballista column; mark Q11/Q21/Q22 as FAIL (open investigation on the client-submission channel's 16 MiB ceiling); publish a Q1-Q10 + Q12-Q20 subset total for engine comparison. - Reproducing: updated ballista-executor invocation with the new flags. Signed-off-by: Andy Grove <agrove@apache.org>
andygrove
force-pushed
the
bench-refresh-2026-07-23
branch
from
July 24, 2026 16:15
14a692e to
5f8ff48
Compare
This was referenced Jul 24, 2026
andygrove
added a commit
that referenced
this pull request
Jul 24, 2026
Add a workflow_dispatch trigger to the docs deploy workflow so it can be re-run on demand. The workflow previously only ran on push to main, so a dropped push event (as happened with #2171) leaves the change unpublished with no way to re-trigger the deploy.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #2161. Refresh the SF1000 numbers on
docs/source/contributors-guide/benchmarking.mdwith results from a run that mounts a dedicated 1000 GiB gp3 EBS PVC per executor at/dataand passes--work-dir /datato the executor. This isolates Ballista's shuffle writes from the container overlay FS / shared root EBS volume — matching thespark.kubernetes.executor.volumes.persistentVolumeClaim.spark-local-dir-1pattern that Spark already uses on the same cluster.Impact
Same commit (
main@696ca29b), same cluster shape, only difference is the dedicated shuffle PVC:Q8 in-suite (45.74 s) is now within 1 s of Q8 standalone (47.4 s in a solo run), confirming the pre-PVC slowdown was root-EBS bandwidth contention from prior queries' dirty pages, not anything Ballista-internal — see the analysis in #2168.
What changed in the doc
r6i.24xlargenode type (96 vCPU, 768 GiB, EBS-only) and the new dedicated Ballista PVC.EXPLAIN ANALYZE-based explanation of the pre-PVC contention.--work-dirand--grpc-server-max-*-message-sizerows; note that the 16 MiB gRPC default currently causes Q11/Q21/Q22 to fail.FAILpending investigation of the client-submission channel's 16 MiB ceiling; publish Q1–Q10 + Q12–Q20 subset totals so cross-engine comparison is on the same set of queries.ballista-executorinvocation with the new flags.Test plan