Adding a .simulate() to the AveragerProgram class?

[tristanlorriaux]

Hello there QICK team !
It's not a bug, but it's a feature (idea).
I was wondering if you would think possible to add in a next update a .simulate(soccfg) method to the QICK AveragerProgram class, that would return various numpy.ndarrays for instance for each t_proc channel. Generator channel could return pulse values, readout channel could return a 0 or 1 if they are reading or not for example, plus a timestamp array? In fact I don't know if it's possible but in this idea it would be awesome !
That would help for pulse shaping for the experiment, and would allow to see 'what we send'.
In any case, thank you for your incredible work, and the support you give to the package !
Best regards,
Tristan from QCircuits, Lyon 🚀

[meeg]

Thanks - something like this would certainly be helpful, the difficulty is in how far to go with it.

At a very basic level, it's easy to add something that tells you which generators and readouts are declared in a program (you can see this yourself by peeking at prog.ro_chs and prog.gen_chs). This is "robust" in the sense that I can be quite sure that a declared readout is going to get read out. But its power is very limited.

Then, it would not be a lot of work to write something that tells you your pulse and readout times, etc. under the assumption that they can just be calculated from the ts and sync_all()s and so on. But now this is not robust - there are many things that people can do (and often do) that will break this, like delay sweeps and conditional jumps, or just bad timeline management. It's dangerous to give people a tool like this that works in simple situations but is wrong in complicated ones, and doesn't know when it is wrong - like an autopilot in a vehicle with unreliable instruments.

A real simulator would solve that problem - something that steps through the ASM code and computes what the tProcessor will do, no assumptions. But this is a bit of a project, particularly if you want it to be really correct about how many cycles it takes the tProcessor to execute each instruction etc. There has been work put into this, but in the end we have not had time to really bring it to completion. The firmware design tools have simulators like this, of course, but they don't interface well to Python.

A separate problem is that we are working on a new version of the tProcessor (which we call v2, the current one is v1) which will have a completely new ASM language and program interface. So AveragerProgram will get replaced in the next few months by something which works similarly, but different, and we are not really going to do any more substantial work on v1 (you will still be able to use v1 of course, and we will fix bugs, but we think that people will prefer to use v2).

Anyway . . . your suggestion is a good one and appreciated, and I am happy to keep talking. I think we can converge on some ideas that are both easy and useful, and we will apply them to v2.

[tristanlorriaux]

Hi ! Thanks for you quick answer, as usual !

At a very basic level, it's easy to add something that tells you which generators and readouts are declared in a program (you can see this yourself by peeking at prog.ro_chs and prog.gen_chs). This is "robust" in the sense that I can be quite sure that a declared readout is going to get read out. But its power is very limited.

I agree this is very a robust raw idea, but it could take a lot of development for 'not so much' results.

A separate problem is that we are working on a new version of the tProcessor (which we call v2, the current one is v1) which will have a completely new ASM language and program interface. So AveragerProgram will get replaced in the next few months by something which works similarly, but different, and we are not really going to do any more substantial work on v1 (you will still be able to use v1 of course, and we will fix bugs, but we think that people will prefer to use v2).

So the basic idea is that the new tProc ASM would give more details that would allow to write a .simulate() method more robust and easier? It's super exciting to hear this !

The idea could be to converge to this kind of pulse sequence plot you can encounter for each tProc channel used, in this package for instance:
https://github.com/kaustubhmote/pulseplot/tree/master

Anyway, I'll wait for the future updates :) !

Best regards,
Tristan from QCircuits, Lyon 🚀

[meeg]

Yep, I know what you mean . . . everyone wants the pretty pulse sequence plot! I think the short answer is that it's not possible for the current AveragerProgram because a) a very large fraction of programs are written in a way that it's not possible to easily predict what the pulse sequence will look like, and b) we're not going to do more development on the tProc v1.

But I think it's a reasonable goal for tProc v2, to have the ability to generate a pulse sequence plot as long as your program only uses the basic functions of AveragerProgramV2 (if you use custom assembly language stuff and try to predict the pulse sequence, you will get an error). I'll leave this issue open as a reminder.

[DRNag2]

Following up on this thread. I've built a schedule visualizer for AveragerProgramV2, and
I'd like to offer it to the project.

Repo: https://github.com/DRNag2/QCVT (MIT)

@meeg, your "autopilot with unreliable instruments" point above was the design driver, so
I want to lead with that rather than with the plot.

It errors instead of guessing

Extraction has a strict mode that raises rather than drawing something plausible and wrong.
It raises on:

• any timed macro (anything carrying t_params) it doesn't explicitly recognize, so a
  future macro type fails loudly instead of being silently dropped from the timeline;
• resync(), because the drawn time is only an upper bound (at runtime it applies
  max(0, t - elapsed)), so a program containing one can't be pinned down statically;
• any failure to resolve a pulse or time parameter.

Untimed macros (register ops, loop control, labels) are ignored by design, and a single
iteration of each loop is drawn with swept parameters annotated as ranges rather than
unrolled.

How I verified this tool

Rather than trusting my own extraction, I cross-checked against the compiled tProc assembly,
which is a separate code path: parse TIME ... inc_ref, WPORT_WR ... @t and TRIG ... @t
out of prog.asm(), convert cycles with the tProc time clock, and compare to the extracted
microsecond times.

Across explicit offsets, interleaved delay/delay_auto, two generators on different fabric
clocks, arb and flat_top envelopes, and a signed gain sweep, every pulse and trigger
matched to 0.0000 ns, on qick 0.2.388, 0.2.406 and 0.2.422. Pulse lengths independently
match the summed waveform-memory segment lengths, including the three-segment flat_top.

Since AveragerProgramV2 resolves all timing at construction given only a QickConfig, this
is all verifiable on a laptop with no board. The test suite runs from a saved soccfg JSON.

What it reads

Public, documented accessors:

• get_pulse_param(), get_time_param()

Undocumented instance attributes — the reason I'm asking rather than just making a PR:

• prog.macro_list, macro.t_params
• prog.pulses[...] (.get_length(), .params)
• prog.envelopes[ch]["envs"][...]["data"]
• prog.ro_chs, prog.loop_dict
• soccfg["gens"], soccfg["readouts"]

Anything built on those will rot silently against your refactors, which is precisely the
failure mode you flagged above ( just on a slower timescale).

[meeg]

@DRNag2, this is very nice, thanks for doing this! I haven't run it or read through the code, but:

• the example visualization looks good and helpful
• from the examples, looks like you have provided good interfaces for using this thing
• it's great that you have a set of CI tests
• as far as what you write here - the strict-mode checks sound reasonable, it sounds like you verified the right things, and I think you are accessing the program in the right way

It's clearly necessary to access the instance attributes you're using, so the question is what to do about it. What are your thoughts? I am open to adding stuff to the QICK program classes and QickConfig. My first thoughts:

1. I think you access a relatively limited subset of the generator+RO information in QickConfig (e.g. clock freqs, maxv) and many of these are things that users might like to access for their own purposes, probably it makes sense to have documented accessors for that information? Maybe there are similar things in the program class (possibly ro_chs and loop_dict?).
2. Probably most of the things you access in the program class are things users don't usually want to access? I am OK with having accessors if it helps formalize your interface. But for my part I'd also be OK with clear comments in the QICK code that explain what is load-bearing for you.

What do you imagine as the path forward? I could see (and if you have something else in mind, go for it):

1. you maintain QCVT as a standalone project - you submit PR(s) to QICK as needed (changes to the class interfaces, links in the docs/demos to guide users to QCVT, fixes for anything I break in the QICK-QCVT interface), but you keep responsibility (maintaining code, handling user questions), and if you move on from QCVT that is not my problem. I am happy to do this without additional review of QCVT, it already looks good.
2. we keep QCVT as a standalone project, but transfer ownership to QICK - you submit PRs as above, and once we are both happy you transfer the repo. You can keep working on QCVT, but if you move on I will keep maintaining it on a best-effort basis. I will want to kick the tires a bit more on QCVT before transferring it, but I don't think I would take long.
3. we merge QCVT into QICK - you submit a PR that brings your repo into QICK, I review and merge, after that you can keep working on QCVT but if you move on I am prepared to maintain it as a core part of the QICK project. I will probably take a while to merge because I will want to read and understand all the code, and I'm busy with a lot of stuff.

In any of these cases, of course you get the credit (in terms of the copyright, any relevant publications, and whatever else is important to you) for the substantial work you've done on this, and you retain access to make changes to your code.

[DRNag2]

@meeg Thank you for your response. The above sounds great. I think generally having more documentation for accessors will make it easier to abstract and update QCVT as QICK continues to evolve. For the attributes the users don't typically access, I think adding accessors is a clean way to support QCVT's longevity (even if they're just helper functions with comments but not listed in official documentation).

I would prefer option 3, and I just made a pull request. I am happy to make a separate PR for the accessors based on your preference.

[meeg]

Thanks, all sounds good, and your PR looks like the right idea. I think it makes sense for you to make the accessor changes in the same PR.

I'll try to find time to review; if you haven't seen any activity from me in a week feel free to nudge.

[DRNag2]

Sure thing! I just made the accessor changes and updated the PR accordingly.

[DRNag2]

Hi @meeg, I'm following up as you suggested. Happy to make revisions as you see fit.