feat: ported nnx model converter #2765
Draft
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR introduces scripts to convert Linen model weights into nnx following the changes made to
models.pyin this pull request (PR #2709).The changes in PR #2709 result in an incompatibility, causing models trained with the older Linen weights (before PR #2709) to fail loading. These scripts provide a forward-compatibility path.
The added tools are:
tools/weight_inspector/convert_linen_checkpoint_to_nnx.py: Converts pre-existing Linen model weights to the current NNX-ported model format.tools/weight_inspector/compare_checkpoint.py: Compares the structure and values between two checkpoint files.Tests
To ensure the conversion works as expected:
Start with a checkpoint directory (
/pre-existing-model-path/0/) containing model weights trained before the models.py changes in PR feat: migrate models.py to use nnx #2709.Convert the pre-existing Linen model weights to the new NNX format:
The comparison script should report no significant structural differences, indicating a successful conversion.
NOTE: For numerical difference, we assume it was caused by different implentation and random generator of Linen and NNX implementation as the main reason.
Checklist
Before submitting this PR, please make sure (put X in square brackets):
gemini-reviewlabel.