Lower RTIO to LLVM IR

[rniczh]

Context:

Description of the Change:

The conversion from RTIO to LLVM IR can be divided into three steps:

1. Schedule for finding feasible groups to meet the constraints of hardware
2. Extract frequency setting for each group (chain of frequency setting ahead of time)
3. Generate ARTIQ’s primitives in LLVM IR

Scheduling

The purpose of schedule step is to further distinguish which regions must be executed serially based on HW constraints derived from ARTIQ. Here we employ BFS to identify the first feasible group and repeat it with the remaining RTIO operation to find all feasible groups.

Currently, a major constraint is that events sharing the same channel must have a unique frequency setting. In other words, if a group contains the same channel but with different frequencies, it’s considered an illegal group.

Frequency Setting

For every feasible group, we execute their frequency setting serially before the group itself starts to run. This ensures that the corresponding channel has been updated to the correct frequency via SPI register and I/O updates on the device before the group execution begins. The IR graph shown here illustrates this intermediate state. Normally, the process lowers directly to LLVM IR, so this step represents a transient state within the IR, which will not appear in the final output of the pass, since the result should already be in LLVM IR.

And there is only one SPI bus available at the device. So each time you can only update one frequency. We also adopt a simple optimization: If that channel has already been updated to a certain frequency then we don’t need to do the same frequency setting again.

Lowering to LLVM IR

Converting from RTIO to LLVM IR is straight forward:

• rtio.empty
  Just call now_mu() to get the current timestamp of ARTIQ and return it

• rtio.pulse
  Involve three steps in rtio.pulse, we split each pulse to three steps during the frequency setting part.

  1. Setting Frequency to SPI
  2. Slack delay to prevent underflow (since setting the frequency to SPI take time on CPU)
     Because frequency setting takes time, it involves the setting from CPU to update value to SPI register, needing to do the delay on the cursor at the same time to ensure no underflow issue happens on the device.
  3. TTL ON + Delay + TTL OFF

  step 1 + 2:

   tail call fastcc void @at_mu(i64 %13)
   tail call fastcc void @_rtio_set_frequency(i32 5120, double 0x41721EAC006AA488, double 0.000000e+00, double 1.000000e+00)
   %16 = tail call fastcc i64 @now_mu()
   tail call fastcc void @at_mu(i64 %13)
   tail call fastcc void @delay_mu(i64 10000)
   %17 = tail call fastcc i64 @now_mu()
   %18 = icmp sgt i64 %16, %17
   %19 = select i1 %18, i64 %16, i64 %17
   tail call fastcc void @at_mu(i64 %19)
   %20 = tail call fastcc i64 @now_mu()
  

  step 3:

  %9 = tail call fastcc i64 @__rtio_sec_to_mu(double 0x3E8646DB8A138F8C)
  tail call fastcc void @at_mu(i64 %8)
  tail call fastcc void @rtio_output(i32 4864, i32 1)
  tail call fastcc void @delay_mu(i64 %9)
  tail call fastcc void @rtio_output(i32 4864, i32 0)
  %10 = tail call fastcc i64 @now_mu()
  

• rtio.sync
  We sync the events by collecting their timestamps and find the maximum value from them and jump to there. And transfer the timestamp through now_mu() as well

  %46 = icmp sgt i64 %44, %45
  %47 = select i1 %46, i64 %44, i64 %45
  tail call fastcc void @at_mu(i64 %47)
  %48 = tail call fastcc i64 @now_mu()
  

Benefits:

Possible Drawbacks:

Related GitHub Issues:

[github-actions]

Hello. You may have forgotten to update the changelog!
Please edit doc/releases/changelog-dev.md on your branch with:

• A one-to-two sentence description of the change. You may include a small working example for new features.
• A link back to this PR.
• Your name (or GitHub username) in the contributors section.

[mehrdad2m]

Thank @rniczh, Wow, this looks amazing! I'll add some minor comments gradually. I am also approving as I will be away from tomorrow to unblock things.

[mehrdad2m]

Note that at the time of the review, there is no tests added. We have to add tests to check the generated IR

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.19%. Comparing base (dcc2ae5) to head (42bd906).

Additional details and impacted files

@@                     Coverage Diff                      @@
##           rniczh/lower-ion-to-rtio    #2273      +/-   ##
============================================================
- Coverage                     97.21%   97.19%   -0.02%     
============================================================
  Files                            93       93              
  Lines                         11243    11248       +5     
  Branches                       1075     1078       +3     
============================================================
+ Hits                          10930    10933       +3     
- Misses                          246      247       +1     
- Partials                         67       68       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[paul0403]

Really amazing! Thanks Hongsheng! 💯