Skip to content

feat: implement DataFrame.checkpoint and DataFrame.localCheckpoint - #1882

Closed
shehabgamin with Copilot wants to merge 22 commits into
mainfrom
copilot/implement-dataframe-checkpoint-localcheckpoint
Closed

feat: implement DataFrame.checkpoint and DataFrame.localCheckpoint#1882
shehabgamin with Copilot wants to merge 22 commits into
mainfrom
copilot/implement-dataframe-checkpoint-localcheckpoint

refactor checkpoint cleanup follow-ups

70332d8
Select commit
Loading
Failed to load commit list.
Codecov / codecov/project succeeded May 13, 2026 in 0s

74.91% (-0.87%) compared to fff406d

View this Pull Request on Codecov

74.91% (-0.87%) compared to fff406d

Details

Codecov Report

❌ Patch coverage is 63.68421% with 138 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...es/sail-spark-connect/src/service/plan_executor.rs 58.40% 99 Missing ⚠️
crates/sail-common-datafusion/src/checkpoint.rs 62.85% 13 Missing ⚠️
crates/sail-plan/src/resolver/query/misc.rs 48.00% 13 Missing ⚠️
.../sail-session/src/session_manager/actor/handler.rs 61.11% 7 Missing ⚠️
crates/sail-catalog/src/manager/tracker.rs 91.42% 3 Missing ⚠️
crates/sail-catalog/src/manager/mod.rs 90.47% 2 Missing ⚠️
crates/sail-catalog/src/error.rs 0.00% 1 Missing ⚠️
@@            Coverage Diff             @@
##             main    #1882      +/-   ##
==========================================
- Coverage   75.77%   74.91%   -0.87%     
==========================================
  Files         942      944       +2     
  Lines      135320   142074    +6754     
==========================================
+ Hits       102544   106428    +3884     
- Misses      32776    35646    +2870     
Files with missing lines Coverage Δ
crates/sail-common/src/config/application.rs 69.56% <ø> (ø)
crates/sail-plan/src/resolver/query/mod.rs 84.15% <100.00%> (-13.53%) ⬇️
crates/sail-spark-connect/src/server.rs 65.71% <100.00%> (-17.41%) ⬇️
crates/sail-spark-connect/src/session.rs 63.49% <ø> (-11.73%) ⬇️
crates/sail-spark-connect/src/session_manager.rs 91.30% <100.00%> (-8.70%) ⬇️
crates/sail-catalog/src/error.rs 56.25% <0.00%> (-12.99%) ⬇️
crates/sail-catalog/src/manager/mod.rs 74.39% <90.47%> (-11.05%) ⬇️
crates/sail-catalog/src/manager/tracker.rs 81.11% <91.42%> (-18.89%) ⬇️
.../sail-session/src/session_manager/actor/handler.rs 43.41% <61.11%> (-7.73%) ⬇️
crates/sail-common-datafusion/src/checkpoint.rs 62.85% <62.85%> (ø)
... and 2 more

... and 50 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.