feat(backends): add GNU Assembler Test Generator for RISC-V #1139
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
AFOliveira
requested changes
Sep 30, 2025
There was a problem hiding this comment.
Hello, @AnimeshAgarwal28.
Thanks for the PR, can you change it so that instead of merging into main, you merge it into my branch that has binutils generator? It's AFOliveira/addBinutils.
Also, if possible, can you post the artifacts this approach generates? Even if just an exapmle for the I extension.
AnimeshAgarwal28
changed the base branch from
main
to
AFOliveira/addBinutils
AnimeshAgarwal28
changed the base branch from
AFOliveira/addBinutils
to
main
AnimeshAgarwal28
changed the base branch from
main
to
AFOliveira/addBinutils
AnimeshAgarwal28
changed the base branch from
AFOliveira/addBinutils
to
main
Implement a test generator that creates GNU Assembler test files from the RISC-V unified database. The generator produces test files in the format expected by the binutils gas test suite. Signed-off-by: Animesh Agarwal <[email protected]>
AnimeshAgarwal28
force-pushed
the
main
branch
from
563ce25 to
1823a4f
Compare
AnimeshAgarwal28
changed the base branch from
main
to
AFOliveira/addBinutils
|
Hi @AFOliveira, I changed this PR to merge over your branch. I am attaching 2 sets of generated files, one for I extension and one for zicbom.: zicbom.s: zicbom.d: i.s: i.d: |
|
Are you only generating valid tests? I.e. not generating test that are supposed to fail? Per example, wrong number of operans, wrong operand type. |
|
No, the generator attempts to generate failing tests also (Currently the generator only generates invalid_operands errors), here is an example: i-fail.l: i-fail.s |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## AFOliveira/addBinutils #1139 +/- ##
=======================================================
Coverage 46.05% 46.05%
=======================================================
Files 11 11
Lines 4942 4942
Branches 1345 1345
=======================================================
Hits 2276 2276
Misses 2666 2666
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
AFOliveira
reviewed
Oct 1, 2025
AFOliveira
reviewed
Oct 1, 2025
AFOliveira
requested changes
Oct 1, 2025
Signed-off-by: Animesh Agarwal <[email protected]>
AnimeshAgarwal28
changed the title
|
Trying something - @claude is all dead code removed? |
There was a problem hiding this comment.
Pull Request Overview
This PR introduces a GNU Assembler test generator that automatically creates test files for the RISC-V toolchain from the unified database. The generator aims to streamline testing of RISC-V extensions by producing assembly source files, dump files, and error test cases in the format expected by the binutils gas test suite.
Key changes:
- Adds comprehensive test generator with constraint-aware assembly generation and realistic operand combinations
- Creates documentation detailing usage, architecture, and integration instructions
- Implements modular design with separate components for instruction grouping, assembly generation, and test orchestration
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| gas_test_generator_readme.md | Documentation covering usage, features, architecture, and extension guidelines |
| gas_test_generator.py | Main implementation with instruction loading, constraint extraction, assembly generation, and test file creation |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
- normalize extension tokens, extract dependencies from definedBy, and carry them into each instruction group - replace ad-hoc fail lists with structured error cases, reason tags - emit annotated pass and no-alias dumps from a single primary example selection per instruction Signed-off-by: Animesh Agarwal <[email protected]>
|
@AnimeshAgarwal28 can you solve copilot's review? |
AnimeshAgarwal28
force-pushed
the
main
branch
from
a75ecc4 to
8b72ae9
Compare
|
@AnimeshAgarwal28 Is this the version that you already tested on binutils and that compiled? |
|
Yes, a lot of the generated tests work as expected, there are still issues with some of the extensions. For eg. c-d.s does not pass because of a hyphen in the march string, this needs to be addressed. |
|
Ok, please let me know when you need help or a review! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces a test generator that creates GNU Assembler test files from the RISC-V unified database. The generator produces test files in the format expected by the binutils gas test suite.
Currently the generated test files although very close to upstream test files can be incorrect due to issues like:
Extension Dependencies: Instructions like ld and sd require the zilsd extension for RV32 but are incorrectly included in the base i extension tests.
Assembly Field Validation: Some instructions have malformed assembly patterns in the database, leading to invalid test cases (e.g., sspopchk.x1).
For usage instructions and more information about the generator check gas_test_generator_readme.md
This is a draft PR intended to gather community feedback/suggestions on approach and priorities before investing in accuracy improvements.