QICK throws an error in between loops: could not broadcast input array from shape (1007,2) into shape (1000,2)

[hk06319]

Hi @meeg,
I am trying to do the resonator spectroscopy with QICK, code is similar to the 06_qubit_demos.ipynb the examples codes. Here is the part of my code,

exp_cfg = { "ro_ch":[0,1],
            "res_ch":[0,1],              # Generator channal DAC229_T1_CH2
            "reps": 1000,               # number of repetitions
            "frequency": 9.5,        # Frequency in MHz
            "lo_frequency": 3024.442,# #3027.50, # LO freq in MHz
            "lo_mw_power": 5.0,     # LO Microwave power in dBm
            "pulse_length":1000,      # [Clock ticks]
            "readout_length":1000,   # [Clock ticks]
            "adc_trig_offset": 200,    # [Clock ticks]
            "sel": "product",          # ADC input path, "product" (default),"dds", "input"
            "adc_trig_offset_plot": 200, # [Clock ticks]
            "pulse_gain":12000,       # [DAC units]
            "soft_avgs":1,
            "relax_delay":10,
            "start":9.5, "step":0.0100, "expts":100,#400 0.0125*2
    }
class SingleToneSpectroscopyProgram(AveragerProgram):
    def initialize(self):
        cfg=self.cfg   
        freq= self.freq2reg(exp_cfg["frequency"], gen_ch=exp_cfg["res_ch"][0], ro_ch=exp_cfg["ro_ch"][0])
        phase_I = soccfg.deg2reg(0, gen_ch=exp_cfg["res_ch"][0])
        phase_Q = soccfg.deg2reg(-90, gen_ch=exp_cfg["res_ch"][1])
        self.declare_gen(ch=exp_cfg["res_ch"][0], nqz=1) #, ro_ch=exp_cfg["ro_ch"][0]
        self.declare_gen(ch=exp_cfg["res_ch"][1], nqz=1) #, ro_ch=exp_cfg["ro_ch"][0]
        self.set_pulse_registers(ch=exp_cfg["res_ch"][0], freq=freq, 
                                 phase=phase_I, gain=exp_cfg["pulse_gain"], 
                                 style='const', length=exp_cfg["pulse_length"])
        self.set_pulse_registers(ch=exp_cfg["res_ch"][1], freq=freq, 
                                 phase=phase_Q, gain=exp_cfg["pulse_gain"], 
                                 style='const', length=exp_cfg["pulse_length"]) 
        
        for ch in cfg["ro_ch"]: #configure the readout lengths and downconversion frequencies
            self.declare_readout(ch=ch, length=cfg["readout_length"],
                                 freq=exp_cfg["frequency"], gen_ch=cfg["res_ch"][0],sel=cfg["sel"])
        self.synci(200)  # give processor some time to configure pulses
    
    def body(self):
        self.measure(pulse_ch=self.cfg["res_ch"], 
             adcs=exp_cfg["ro_ch"],
             adc_trig_offset=self.cfg["adc_trig_offset"],
             wait=True,
             syncdelay=self.us2cycles(self.cfg["relax_delay"])) 

for f in tqdm(fpts):
        exp_cfg["frequency"]=f

        rspec=SingleToneSpectroscopyProgram(soccfg, exp_cfg)
        avgi,avgq=rspec.acquire(soc,progress=False

        amp=np.abs((avgi[0][0])+1j*(avgq[0][0]))
        real.append(np.real((avgi[0][0])+1j*(avgq[0][0])))
        imag.append(np.imag((avgi[0][0])+1j*(avgq[0][0])))
        amps.append(amp)

I am trying to run the above code multiple times with software loop. It runs correctly, but in between it throws the following error and code execution stops.
Error logs are:

>  File "E:\Himanshu\Cavity Setup\Code\Qick ZCU111\mw_readout_ZCU111_spectrum_DoubleCoil_AllAligned_xAxis_MultiComp_Interp.py", line 372, in <module>
>     avgi,avgq=rspec.acquire(soc,progress=False)
>               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   File "C:\ProgramData\Anaconda3\envs\pyrpl-env\Lib\site-packages\qick\averager_program.py", line 106, in acquire
>     return super().acquire(soc, rounds=self.rounds, load_envelopes=load_pulses, extra_args=extra_args, **kwargs)
>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   File "C:\ProgramData\Anaconda3\envs\pyrpl-env\Lib\site-packages\qick\qick_asm.py", line 1936, in acquire
>     while self.finish_round():
>           ^^^^^^^^^^^^^^^^^^^
>   File "C:\ProgramData\Anaconda3\envs\pyrpl-env\Lib\site-packages\qick\qick_asm.py", line 2298, in finish_round
>     self.acc_buf[ii].reshape((-1,2))[count*nreads:(count+new_points)*nreads] = d[ii]
>     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> ValueError: could not broadcast input array from shape (1007,2) into shape (1000,2)

I appreciate your support.
Thanks
Regards
Himanshu Kumar

[meeg]

It's not obvious why this would happen. Some questions:

• you're using Pyro, right? what is the Python version on your PC?
• how often does this error happen - e.g. if you have 1000 points in fpts how many do you get through before the error happens?

[hk06319]

It's not obvious why this would happen. Some questions:

• you're using Pyro, right? what is the Python version on your PC?
• how often does this error happen - e.g. if you have 1000 points in fpts how many do you get through before the error happens?

Yes, I am using Pyro4, the Python version in my Anaconda environment, where QICK is installed, is Python 3.9.12. I set fpts to 100 and repeat this 1760 times.
If I set the readout length to 1000, it occurs around 760. Another time I gave 700 length, it occurred at around the 500th point.

[meeg]

OK - partly as a workaround and partly for debugging, can you wrap your acquire() in a try-catch? maybe like this (with i_repeat, i_freq as counters to tell you which repeat and freq point you are on):

while True:
    try:
        avgi,avgq=rspec.acquire(soc,progress=False)
        break
    except:
        print("crashed", i_repeat, i_freq)