Skip to content

ZSTD decoder: cocotb utils and decoder testcases, verilog helper modules #2717

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 30 commits into from
Closed

ZSTD decoder: cocotb utils and decoder testcases, verilog helper modules #2717

wants to merge 30 commits into from

Conversation

laudominik
Copy link
Contributor

Added functionality:

  • multiple testcases covering the decoding of frames with "compressed" blocks of different type
  • cocotb testing utilities for common operations on XLS channels
  • collection of PnR benchmarks
  • ZSTD frame generator
  • tests of axi ram writer/reader procs as well as memreader & memwriter

@laudominik laudominik marked this pull request as ready for review July 29, 2025 09:23
@tmichalak
Copy link

@proppy

@mgielda
Copy link

mgielda commented Aug 1, 2025

It would be good to understand whether there are any major blockers to merging this - like Python deps etc. This will need the Google-internal CI to run, which I think needs our approval @proppy

rw1nkler and others added 27 commits August 21, 2025 08:33
@rw1nkler @laudominik
Add a separate test case for each test file
deb7955 
Signed-off-by: Robert Winkler <[email protected]>
@sgizler @laudominik
Add "pytype binary, library" hint
120c3b8
@koblonczek @laudominik
Add cocotb tests
29ef8ce 
Add cocotb testing utilities:
- XLSStruct for easier handling and serializing/deserializing XLS structs
- XLSChannel that serves as a dummy receiving channel
- XLSMonitor that monitors transactions on an XLS channel
- XLSDriver that can send data on an XLS channel
- LatencyScoreboard that can measure latency between corresponding transactions on input and output buses
- File-backed AXI memory python model

Add cocotb tests for:
- modules/zstd/memory/MemReader
- modules/zstd/memory/AxiWriter
- modules/zstd/memory/MemWriter

Co-authred-by: Pawel Czarnecki <[email protected]>
Co-authred-by: Robert Winkler <[email protected]>
Co-authred-by: Michal Czyz <[email protected]>
Signed-off-by: Michal Czyz <[email protected]>
Signed-off-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Robert Winkler <[email protected]>
Signed-off-by: Krzysztof Obłonczek <[email protected]>
@lpawelcz @laudominik
xls/modules/zstd: expose fifo verilog module
12bb81a 
Signed-off-by: Pawel Czarnecki <[email protected]>
@lpawelcz @laudominik
modules/zstd/cocotb: Add ZSTD frame generator library
f138fbe 
This reverts commit 04ad379225b706ddf492d440c673e77348d7a409.
@lpawelcz @laudominik
dependency_support: Add zstandard python library
811f5fe 
Signed-off-by: Pawel Czarnecki <[email protected]>
@lpawelcz @laudominik
modules/zstd: Add verilog simulation of the ZstdDecoder
2a71280 
Signed-off-by: Pawel Czarnecki <[email protected]>
@koblonczek @rw1nkler @laudominik
Use frame generator in tests
5b89e4c 
Co-authored-by: Robert Winkler <[email protected]>
Signed-off-by: Robert Winkler <[email protected]>
Signed-off-by: Krzysztof Obłonczek <[email protected]>
@lpawelcz @laudominik
modules/zstd/zstd_dec: Write decoded data to the memory
613a292 
ZstdDecoder:
  * Cocotb tests:
    * Move third-party verilog modules (AXI Interconnect) to external directory
    * Replace AXI Interconnect with AXI Crossbar that handles simultaneous AXI Read
      and Write transactions
    * Add reference memory and fill it with expected data for comparison
      against testbench memory at the end of the decoding

Internal-tag: [#67272]
Signed-off-by: Pawel Czarnecki <[email protected]>
@lpawelcz @laudominik
modules/zstd/zstd_dec: Remove stream-based output interface
9cf5c38 
Adjust cocotb test to:
* Removal of Repacketizer proc
* Removal of stream-based output channels from
  * SequenceExecutor
  * ZstdDecoder

Internal-tag: [#67272]
Signed-off-by: Pawel Czarnecki <[email protected]>
@lpawelcz @laudominik
modules/zstd/zstd_dec_cocotb_test: Improve Verilog simulation
e257130 
* Decode multiple ZSTD frames in a single cocotb testbench
* Add one cocotb testbench per type of the ZSTD frames:
  * Frames with RAW blocks only
  * Frames with RLE blocks only
  * Frames with Compressed blocks only (disabled)
  * Frames with mixed blocks (disabled)

Internal-tag: [#67272]
Signed-off-by: Pawel Czarnecki <[email protected]>
@lpawelcz @laudominik
modules/zstd/memory/mem_writer: Reduce the amount of random test cases
93c265d 
Caused by timeouts after rebase - looks like regression in the
performance of codegen

Signed-off-by: Pawel Czarnecki <[email protected]>
@lpawelcz @laudominik
modules/zstd/memory: Move verilog sources to rtl subdirectory
0c8a659 
Signed-off-by: Pawel Czarnecki <[email protected]>
@lpawelcz @laudominik
xls/modules/zstd/external: Add docstring and license for third party …
6e8fd9a 
…sources

Signed-off-by: Pawel Czarnecki <[email protected]>
@lpawelcz @laudominik
modules/zstd: Adjust cocotb test to removal of RESET CSR
8d73bd9 
Signed-off-by: Pawel Czarnecki <[email protected]>
@wsipak @laudominik @lpawelcz
Style changes
4ec4493 
Improve formatting, wording and fix lint issues

Co-authored-by: Dominik Lau <[email protected]>
Co-authored-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Wojciech Sipak <[email protected]>
@rw1nkler @laudominik
Extend cocotb tests of ZSTD decoder
6f647db 
Co-authored-by: Pawel Czarnecki <[email protected]>
Co-authored-by: Krzysztof Oblonczek <[email protected]>
Co-authored-by: Wojciech Sipak <[email protected]>
Co-authored-by: Dominik Lau <[email protected]>
Co-authored-by: Szymon Gizler <[email protected]>

Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler @laudominik
Collect P&R benchmarks
ceb1223 
Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler @laudominik
Add fixed version of RamDemux together with a test
e49a052 
Signed-off-by: Robert Winkler <[email protected]>
@sgizler @laudominik
Document decoder testing
7a5805d
@rw1nkler @laudominik
dependency_support: Add verilog_axi package
a058452 
Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler @laudominik
Use bazel package with verilog_axi
d092144 
Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler @laudominik
Downgrade dependencies
ba76f0f 
Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler @laudominik
Run tests selectively in GitHub CI
63b47c2 
Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler @laudominik
Remove zstandard python library
05e77f7 
Signed-off-by: Robert Winkler <[email protected]>
@laudominik
modules: zstd: update readme
c748093
@rw1nkler @laudominik
Add cocotb check for verifying huffman codes
98fb26d 
Signed-off-by: Robert Winkler <[email protected]>
@laudominik laudominik force-pushed the zstd_decoder_cocotb_verilog_github_pr branch from 991dc61 to be9a066 Compare August 21, 2025 06:35
@dplassgit dplassgit requested a review from proppy September 9, 2025 15:38
@rw1nkler
Copy link
Contributor

Superseded by #3035

@rw1nkler rw1nkler mentioned this pull request Sep 10, 2025
Open
@dplassgit dplassgit closed this Sep 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@proppy proppy Awaiting requested review from proppy

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
9 participants
@laudominik @tmichalak @mgielda @rw1nkler @dplassgit @sgizler @koblonczek @lpawelcz @wsipak