Skip to content

[ENH] Improve CG solver and add weights caching for better performance #17

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 14 commits into
base: main
Choose a base branch
from
Open

[ENH] Improve CG solver and add weights caching for better performance #17

wants to merge 14 commits into from

Conversation

Leguark
Copy link
Member

@Leguark Leguark commented Aug 5, 2025 •
edited
Loading

Enhanced Conjugate Gradient Solver with Improved Stability and GPU Support

This PR significantly improves the robustness and performance of our solver system with several key enhancements:

Solver Improvements

  • Completely refactored the ConjugateGradientSolver with enhanced stability features for ill-conditioned matrices
  • Added adaptive tolerance, regularization support, and robust convergence criteria
  • Implemented residual monitoring and restart capability for better convergence
  • Added detailed diagnostics and warnings for numerical issues

GPU Acceleration

  • Added GPU support for PyKeOps solver in the torch backend
  • Properly configured backend tensor handling for GPU operations
  • Fixed device selection logic in solver interface

Caching and Weights Management

  • Added ability to pass initial weights through the interpolation input
  • Implemented weights_x0 in solver input for warm starts
  • Added clear_cache method to WeightCache for better memory management

Numerical Stability

  • Fixed condition number computation and made plotting optional
  • Properly store condition number in kernel options during operations
  • Added safeguards against numerical breakdown in denominator calculations
  • Implemented stagnation detection for ill-conditioned systems

Experimental Features

  • Added Nyström preconditioner implementation (currently disabled by default)
  • Included configurable pivoting strategies for matrix approximation

These changes significantly improve the solver's ability to handle challenging numerical problems while maintaining performance.

Leguark added 14 commits July 23, 2025 11:52
@Leguark
[BUG] Fix condition number computation and optional plotting
99fd3dd 
Updated `_compute_conditional_number` to return the computed condition number and made plotting optional with a new `plot` argument. Adjusted kernel options to store condition number during numpy backend operations.
@Leguark
[BUG] Store condition number in kernel options during PyTorch backend…
4825ce5 
… operations
@Leguark
[CLN] Adjust import placement for conditional eigenvalue plotting
b29093b
@Leguark
[TEST] More benchmarks
dc60bf2
@Leguark
[ENH] Add GPU support for PyKeOps solver in torch backend
ee0776c 
Introduced a `use_gpu` parameter to enable GPU acceleration in the `pykeops_torch_cg` function. Adjusted solver backend logic to toggle between CPU and GPU based on this parameter.
@Leguark
[ENH] Refactor and optimize PyKeOps Conjugate Gradient solver
49d63a2 
Enhanced the `ConjugateGradientSolver` function with
@Leguark
[WIP] Trying to add preconditioners to pykeops
bd292cf
@Leguark
[CLN/WIP] Try to add preconditioners
cb1c007
@Leguark
[ENH] Introduce Adaptive Nyström Preconditioner
0484c4d 
Replaced helper function-based preconditioning with the adaptive Nyström preconditioner in the PyKeOps solver. This includes configurable strategies for pivot selection (`aggressive`, `conservative`, `minimal`) and fallback handling for robust kernel matrix approximation. Removed unused diagonal and Jacobi preconditioners. Cleaned up redundant code.
@Leguark
[ENH] Add verbose option to Conjugate Gradient Solver
86eedf5 
Introduced a `verbose` parameter to control diagnostic print statements during the iteration process.
@Leguark
[CLN] Removed unused funtions
a12127e
@Leguark
[CLN] Disable unused preconditioner logic in PyKeOps solver
19d44be
@Leguark
[CLN] Add note on GPU incompatibility for unused preconditioner logic
308a247
@Leguark
Add caching improvements and enhance solver stability
1a2c04b 
Introduced a `clear_cache` method in the `WeightCache` class for better memory management. Enhanced the numerical stability of the conjugate gradient solver with improved initialization and adaptive tolerances. Refactored interpolation logic to handle weights more efficiently, and adjusted benchmarking to test expanded solver configurations.
@Leguark Graphite App
Copy link
Member Author

Leguark commented Aug 5, 2025 •
edited
Loading

This stack of pull requests is managed by Graphite. Learn more about stacking.

This was referenced Aug 5, 2025
Open
Open
@Leguark Leguark changed the title [BUG] Fix condition number computation and optional plotting [ENH] Improve CG solver and add weights caching for better performance Sep 17, 2025
@Leguark Leguark marked this pull request as ready for review September 17, 2025 12:53
@Leguark Leguark mentioned this pull request Sep 19, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@Leguark