Skip to content

[Feature] Enable FP32 RMSNorm - #845

Open
bohnstingl wants to merge 9 commits into
torch-spyre:mainfrom
bohnstingl:rms_norm-fix
Open

[Feature] Enable FP32 RMSNorm#845
bohnstingl wants to merge 9 commits into
torch-spyre:mainfrom
bohnstingl:rms_norm-fix

Conversation

@bohnstingl

@bohnstingl bohnstingl commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Description

This PR enables the use of vLLM's native RMSNorm implementations.
We still need to keep our own custom OOT, as we need to enforce compilation.

Alternative to #742

Related Issues

#667 and #817

Test Plan

The behavior should be transparent to the user, so all existing tests should suffice.

Checklist

  • I have read the contributing guidelines
  • My code follows the project's code style (run bash format.sh)
  • I have added tests for my changes (if applicable)
  • I have updated the documentation (if applicable)
  • My commits include a Signed-off-by: line (DCO compliance)

cc @yannicks1

Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>
@bohnstingl
bohnstingl requested review from a team, dilipgb and yannicks1 as code owners September 10, 2026 17:35
@github-actions

Copy link
Copy Markdown
Contributor

👋 Hi! Thank you for contributing.
Just a reminder: Make sure that your code passes all the linting checks, otherwise your PR won't be able to be merged. To do so, run ./format.sh.
Now you are good to go 🚀.

We also recommend installing prek and configuring it to check your code before every local commit.

Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>
…s_norm-fix

Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>
Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>
Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>
Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>
Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>
Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>
…s_norm-fix

Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>
@joerunde

Copy link
Copy Markdown
Collaborator

@bohnstingl it looks like there's still a test failure here, and then the way that the test fails seems to hold the card open causing the next couple tests to fail :(

I'd hope that fixing the behavior leading to the failing test would fix the card busy problem too, but seems like there's still work to do here?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants