Skip to content

Add cupy backend for GPU support #1048

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 47 commits into from
Nov 6, 2025
Merged

Add cupy backend for GPU support #1048

merged 47 commits into from
Nov 6, 2025

Conversation

@s3alfisc
Copy link
Member

@s3alfisc s3alfisc commented Oct 23, 2025

We add a cupy backend https://docs.cupy.dev/en/latest/index.html to run FWL on the GPU.

Strategy: "direct" FWL OLS residualization instead of iterative solver.

Should help with problems with slow convergence.

Only tested on CPU via scipy, where results are matching. Cannot test on GPU as no Mac / no access to Cuda GPU.

Note: we can build a backend using a similar strategy via JAX and torch.

import pyfixest as pf

data = pf.get_data()

fit_rust = pf.feols("Y ~ X1 + X2  | f1", data = data, demeaner_backend="rust")
fit_cupy = pf.feols("Y ~ X1 + X2  | f1", data = data, demeaner_backend="cupy")

(pf.etable([fit_rust, fit_cupy], type = "md", digits = 8))
index                    est1             est2
------------  ---------------  ---------------
depvar                      Y                Y
----------------------------------------------
X1            -0.94952557***   -0.94952557***
                 (0.06637281)     (0.06637281)
X2            -0.17422527***   -0.17422527***
                 (0.01759569)     (0.01759569)
f1                          x                x
----------------------------------------------
Observations              997              997
----------------------------------------------
S.E. type                 iid              iid
R2                 0.48899391       0.48899391
R2 Within          0.23875790       0.23875790
----------------------------------------------

@s3alfisc s3alfisc linked an issue Oct 23, 2025 that may be closed by this pull request
pyfixest is MUCH slower than (Stata + Julia) reghdfejl #1042
Closed
@s3alfisc s3alfisc mentioned this pull request Oct 23, 2025
Closed
@codecov
Copy link

codecov bot commented Oct 23, 2025 •
edited
Loading

Codecov Report

❌ Patch coverage is 62.32877% with 55 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
pyfixest/estimation/cupy/demean_cupy_.py 60.93% 50 Missing ⚠️
pyfixest/estimation/backends.py 54.54% 5 Missing ⚠️
Flag Coverage Δ
core-tests 76.05% <62.32%> (-0.32%) ⬇️
tests-vs-r 15.82% <23.28%> (+0.15%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
pyfixest/estimation/demean_.py 55.64% <100.00%> (+2.25%) ⬆️
pyfixest/estimation/estimation.py 90.80% <ø> (ø)
pyfixest/estimation/literals.py 87.50% <100.00%> (ø)
pyfixest/estimation/backends.py 72.41% <54.54%> (-10.92%) ⬇️
pyfixest/estimation/cupy/demean_cupy_.py 60.93% <60.93%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@s3alfisc
Copy link
Member Author

s3alfisc commented Oct 23, 2025 •
edited
Loading

TODO

  • add tests
  • create GPU env
  • maybe fix numpy to pandas to sparse array conversion? could lead to poor performance

@review-notebook-app
Copy link

review-notebook-app bot commented Oct 31, 2025

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

@s3alfisc
Copy link
Member Author

s3alfisc commented Nov 2, 2025

pre-commit.ci autofix

pre-commit-ci bot and others added 15 commits November 2, 2025 11:15
@s3alfisc s3alfisc force-pushed the cupy branch 2 times, most recently from 6e07ac6 to 6388b11 Compare November 6, 2025 21:38
@s3alfisc s3alfisc merged commit ed8db98 into master Nov 6, 2025
6 of 8 checks passed
@s3alfisc s3alfisc deleted the cupy branch November 6, 2025 23:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

pyfixest is MUCH slower than (Stata + Julia) reghdfejl

2 participants

@s3alfisc