BBT export renders zotero v laggy and even unresponsive

[kzssc]

Debug log ID

QJMQH2PR-euc/6.7.122-7

What happened?

basically what the subject says, what's also indicative is the power consumption ramps up to 3,000s (versus <100 when BBT is not exporting) as reported here: #2522 (comment)

https://i.imgur.com/MtCXgK9.png

BTT: 6.7.122
7.0.0-beta.43+f803c0cc4
M1 13.6

[retorquere]

Is this an auto-export or a manual export?

[kzssc]

This one is manual, but auto export also runs into this.

[retorquere]

A debug log will drop here shortly; can you install that, then go into the preferences and click Export now for one auto-export, let it finish, then click Export now again for the same auto-export, let it finish, and then send a new debug log.

[github-actions]

🤖 this is your friendly neighborhood build bot announcing test build 6.7.122.2667.5171 ("logging")

Install in Zotero by downloading test build 6.7.122.2667.5171, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".

[kzssc]

4SNJ4PBS-euc/6.7.122.2667.5171-7 hope this helps!

[retorquere]

A new build will drop shortly, can you do the same procedure again please?

Part of the problem is that BBT drops its caches when you upgrade Zotero, and at least between QJMQH2PR-euc/6.7.122-7 and 4SNJ4PBS-euc/6.7.122.2667.5171-7 you did upgrade. That will make the first of the two very slow, but the 2nd (and following) should be substantially faster. The new build will tell.

[github-actions]

🤖 this is your friendly neighborhood build bot announcing test build 6.7.122.2667.5174 ("more logging")

Install in Zotero by downloading test build 6.7.122.2667.5174, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".

[kzssc]

Thanks for the quick actions and the insights! Will test it out.

In hindsight I'm less concerned about the need for cache rebuild but more on how much system resources it's using which renders Zotero practically unusable...perhaps de-prioritising it on CPU usage?

[retorquere]

The cache use has a direct influence here. A filled cache makes BBT much more efficient.

I can't really influence the CPU priority. If you want to I can explain how zotero translators work but the bottom line is that you need to have a filled cache to have a 10k+ export be tolerable qua performance impact.

[kzssc]

VSE847TS-euc/6.7.122.2667.5174-7 here you have it :))

[retorquere]

Blegh, the bulk of the time is spent in prep, and prep is what hinders the UI responsivity. I have been planning a solution for this but it's going to be quite a bit of work. Worth it but it might take me a few weeks to get this working.

[kzssc]

perhaps a related issue: when BBT is enabled - with auto-export turned off - adding new attachment files to zotero (via new built-in function) also renders UI very slow. Disabling BBT then, file import does not incur UI lag.

My guess is that citekey generation and association might be the cause of high CPU usage? Which diagnosis code should I attach, zotero's or BBT's?

[retorquere]

It's not key generation. No work is done for attachments as they don't get keys.

Which diagnosis code should I attach, zotero's or BBT's?

Always BBTs. I cannot access Zotero debug logs.

[kzssc]

Here you go 😃 : SGCT6TJW-euc/6.7.130-7

[retorquere]

There's another bug I must address first, but I'm in the middle of a restructuring that should lead to improvements for the problem you're facing. That will unfortunately take me about a week to get done, so it will go silent here for a bit.

[daeh]

Just wanted to add that I had auto export turned on and any modification to the library caused the UI to hang for 20-40 seconds. I disabled the automatic exports for now. Happy to submit logs if helpful.

Thanks!

[retorquere]

Already working on it.

[kzssc]

A minor update on this: I suspect that even with auto-export off - whenever Zotero syncs - this happens. So just to confirm: the sluggishness happens on two occasions: 1. when zotero syncs and 2. when bbt exports. Though I can't quite replicate the problem with 1 as it doesn't always happen.

Initially I suspected that's a Z7 bug (posted here: https://forums.zotero.org/discussion/109954/z7-sluggish-ui-due-to-sync#latest) but once I disabled BBT, UI works normally.

[kzssc]

I am not sure whether the following is a remedy for current issue or request for permanent function:

In auto-export, could an option of "auto-exporting with zotero launch" be created?

The current options for auto-export are a bit too frequent, I think. Another thing is I'm not entirely sure the mechanism for "When Idle": auto-export seems to kick-off even when I'm interacting with Zotero.

Many thanks as always, @retorquere !

[retorquere]

I'm working on a fix for the sluggishness. I don't see real value in "at launch", and it would be hard to get it to be both reliable and performant, but if "when idle" misbehaves for you, you can open a new issue for that and I'll look into it.

[retorquere]

The fix is ready; the work is being tracked/tested on #2315, but it's going into the next release.