Eta-template was not working like before

[acsr]

Debug log ID

S2M4W83X-refs-euc/6.7.160-6

What happened?

Note

After upgrading to BetterBibTex 6.7.160 my latest eta-template was not working like expected anymore. The day before it was OK!

Possible Cause:
it.items serialized from JSON were not seperated by parbreaks at the end of the nested output of the template.

The JSON l directly exported via BetterBibTeX json looked OK.

@retorquere Can you remember I had to work around differences between newlines and paragraph breaks to make it working for me in Logseq.

I am not sure if I need to change my template or something is missing in your update.
In any case it should not pass silently.

Example Export

In line 12 the next item starting at - ### The Most Dangerous Writing App is concatenated to the preceeding item without any delimiter.

- ### The Least Dangerous To-Do List
creators:: author: Manuel Ebert
url:: https://theleastdangeroustodolist.com
tags:: #[[Schreiben]], #[[Writing]], #[[Online Tools]], #[[Tools]], #[[Writing Software]], #[[Writing Blog Posts]], #[[Text Creation]], #[[Schreibblockade]], #[[Manu Ebert]], #[[The Most Dangerous Writing App]]
zotero:: [@ebertLeastDangerousToDo2023](zotero://select/groups/5387086/items/E6Q3V4RB)
  - #### abstractNote
    The satisfaction of checking things off your to-do list, without any of the hard work.
  - #### Relations
    - http://zotero.org/groups/3305/items/HPSPXE2V
    - http://zotero.org/groups/5387086/items/IJ6LI577
    - http://zotero.org/groups/5387086/items/IQ438J55
    - http://zotero.org/groups/5387086/items/Y428Z85I- ### The Most Dangerous Writing App
creators:: author: Manuel Ebert
url:: https://maebert.github.io/themostdangerouswritingapp
tags:: #[[Schreiben]], #[[Writing]], #[[Online Tools]], #[[Tools]], #[[Writing Software]], #[[Writing Blog Posts]], #[[Text Creation]], #[[Schreibblockade]], #[[Manu Ebert]], #[[The Most Dangerous Writing App]]
zotero:: [@ebertMostDangerousWriting2022](zotero://select/groups/5387086/items/IQ438J55)
  - #### abstractNote
    Don’t stop typing, or all progress will be lost.
  - #### Relations
    - http://zotero.org/groups/3305/items/K69CGHDJ
    - http://zotero.org/groups/5387086/items/E6Q3V4RB
    - http://zotero.org/groups/5387086/items/IJ6LI577
    - http://zotero.org/groups/5387086/items/Y428Z85I

If the issue is on my side now (due to silly workarounds in the eta-template), I m happy to change the template on my side.

Originally posted by @acsr in #2789 (comment) (now with Debug log ID!)

[retorquere]

it.items serialized from JSON were not seperated by parbreaks at the end of the nested output of the template.

Things aren't serialized from JSON. Things are serialized to JSON, and none of the fields included in S2M4W83X-refs-euc/6.7.160-6 have linebreaks or parbreaks, so there's no parbreaks for BBT to export.

Can you remember I had to work around differences between newlines and paragraph breaks to make it working for me in Logseq.

Yeah that was in the template, not the data. There's no newlines in the fields you are accessing, and BBT didn't put any there), and #2789 only made changes to import, not export. If you change the last line of your template into <% } %>, eta won't gobble the empty line before it, and you get the separation you want.

In any case it should not pass silently.

I don't know how to interpret this. I don't know what it means.

For experimentation you might find it easier to paste the code below into the ETA playground and toy around until you have what you want; I've just prepended the item contents of S2M4W83X-refs-euc/6.7.160-6 at the top of the eta template from S2M4W83X-refs-euc/6.7.160-6 (you'll notice that with the existing code, you don't get the separation you want there either):

<%-
it.items = [{"version":0,"itemType":"webpage","url":"https://theleastdangeroustodolist.com","date":"2023-1-12","accessDate":"2024-02-02T16:43:32Z","language":"English","abstractNote":"The satisfaction of checking things off your to-do list, without any of the hard work.","title":"The Least Dangerous To-Do List","creators":[{"firstName":"Manuel","lastName":"Ebert","creatorType":"author"}],"tags":[{"tag":"Manu Ebert"},{"tag":"Online Tools"},{"tag":"Schreibblockade"},{"tag":"Schreiben"},{"tag":"Text Creation"},{"tag":"The Most Dangerous Writing App"},{"tag":"Tools"},{"tag":"Writing"},{"tag":"Writing Blog Posts"},{"tag":"Writing Software"}],"collections":["C28L9G6G"],"relations":{},"dateAdded":"2024-02-18T21:27:04Z","dateModified":"2024-02-18T21:27:04Z","uri":"http://zotero.org/users/8230813/items/E9K2JP7K","attachments":[],"notes":[],"citationKey":"ebertLeastDangerousToDo2023","itemID":103,"itemKey":"E9K2JP7K","libraryID":1},{"version":0,"itemType":"webpage","url":"https://maebert.github.io/themostdangerouswritingapp","date":"2022-9-20","accessDate":"2024-02-02T16:43:14Z","language":"en","abstractNote":"Don’t stop typing, or all progress will be lost.","title":"The Most Dangerous Writing App","creators":[{"firstName":"Manuel","lastName":"Ebert","creatorType":"author"}],"tags":[{"tag":"Manu Ebert"},{"tag":"Online Tools"},{"tag":"Schreibblockade"},{"tag":"Schreiben"},{"tag":"Text Creation"},{"tag":"The Most Dangerous Writing App"},{"tag":"Tools"},{"tag":"Writing"},{"tag":"Writing Blog Posts"},{"tag":"Writing Software"}],"collections":["C28L9G6G"],"relations":{},"dateAdded":"2024-02-18T21:27:04Z","dateModified":"2024-02-18T21:27:04Z","uri":"http://zotero.org/users/8230813/items/JU9N9JRZ","attachments":[],"notes":[],"citationKey":"ebertMostDangerousWriting2022","itemID":102,"itemKey":"JU9N9JRZ","libraryID":1,"publicationTitle":"MDWA"}]
-%>
<%- for (const item of it.items) {
  const [ , kind, lib, key ] = item.uri.match(/^https?:\/\/zotero\.org\/(users|groups)\/((?:local\/)?[^/]+)\/items\/(.+)/)
  const select = (kind === 'users') ? `zotero://select/library/items/${key}` : `zotero://select/groups/${lib}/items/${key}`

  const relations = []
  if (item.relations) {
    for (const [kind, rels] of Object.entries(item.relations)) {
      for (const rel of rels) {
        relations.push(rel)
      }
    }
  }
-%>
- ### <%= item.title %>
<%- if (item.creators.length) { -%>
<%= "\n" %>creators:: <% item.creators.forEach((creator, i) => { %><%= creator.creatorType %>: <%= creator.name || `${creator.firstName || ''} ${creator.lastName || ''}`.trim() %><% if (i < item.creators.length - 1) { %>, <% } %><% }) %>
<%- } -%>

<%- if (item.url) { -%>
<%= "\n" %>url:: <%= item.url %>
<%- } -%>

<%- if (item.tags.length) { -%>
<%= "\n" %>tags:: <% item.tags.forEach((tag, i) => { %>#[[<%= tag.tag %>]]<% if (i < item.tags.length - 1){%>, <%}%><% }) %>
<%- } -%>

<%- if (item.citationKey) { -%>
<%= "\n" %>zotero:: [@<%= item.citationKey %>](<%= select %>)
<%- } -%>
<%= "\n" %>

<%- if (item.abstractNote) { -%>
<%= "\r" %>  - #### abstractNote
<%= "\r" %>    <%= item.abstractNote %>
<%- } -%>

<%- if (item.notes.length) { -%>
<%= "\r" %>  - #### Notes
<%- for (const note of item.notes) { -%>
<%= "\r" %>    - ##### Note [<%= note.key %>](<%= note.uri %> "URI Link")
<%~ note.note %>
<%- } -%>
<%- } -%>

<%- if (relations.length) { -%>
<%= "\r" %>  - #### Relations
<%- for (const rel of relations) { -%>
<%= "\r" %>    - <%= rel %>
<%- } -%>
<%- } -%>

<%- if (item.rights) { -%>
<%= "\r" %>  - #### Rights
<%= "\r" %>    - <%= item.rights %>
<%- } -%>

<%- } -%>

[retorquere]

It does seem to me at this point it would be easier to create your own translator rather than using eta.

[acsr]

Thank you for your efforts.
The tip with eta-playground is golden

You are right when pointing me to a special translator. I will see how far I can get.
In this case I skip updating your eta-template docs for now and focus getting the thing out of your way.

@retorquere wrote

In any case it should not pass silently.

I don't know how to interpret this. I don't know what it means.

In the Zen of Python there are two aphorisms relatet to this:

1. Explicit is better than implicit. (my fault here)
2. An error should never pass silently

On the second point: a closed ticket mentioning a special effect and its cause like mine can help others even if the mistake is on my side. If you have nothing to google, nobody can find it. Sometimes I find my own 10 years old solutions, I already forgot.

[acsr]

Removing the hyphens in the last line of the template worked for me! Obvious on second sight. I update my Gits

[retorquere]

In the Zen of Python there are two aphorisms related to this:

1. Explicit is better than implicit. (my fault here)

2. An error should never pass silently

But there wasn't an error that was passed silently. There was undesirable behavior that was expected given the input. I don't see how you could programmatically detect this and notify the user (and if such notification isn't the point, then I still don't know what, exactly, should not pass silently).

On the second point: a closed ticket mentioning a special effect

Much of what we have been discussing here is documented behavior from the eta project; that the hyphen gobbles input is just how eta works. If there is some issue from BBT that mentions a special effect from BBT itself, I don't know what issue this would be.

[acsr]

For your peace of mind: The error was mine not your code. Clear for me, hopefully now also for the audience. This is reflected in the title of the ticket. No idea why the– lets call it effect – on my side was correlating with the update, not caused. It just unveilled the flaws of my code.

The Zen of Python mentions error as an effect you should always publish as alert. My interpretation here is to share the impact. On the other hand, missing testing skills on my side wasted your rare time.

Proper exception handling can unveil the context in detail. But we do not have these tools in Zotero itself. Your hint with the eta playground ist an instant feedback tool. Like a Jupyter Notebook can speed up development. But the trick is to insert the sample content like in your example.

It is hard for the ignorant (me) to catch his ignorance because he is ignoring something.
I now close this ticket as solved.

[retorquere]

I don't mind the time, it's just that when I see a problem, I want to fix it, and if I don't understand the problem, I cannot fix it.

BTW comma-separation is easier (IMO) with

<%= "\n" %>tags:: <%= item.tags.map(tag => tag.tag).join(', ') %>

and

<%= "\n" %>creators:: <% item.creators
  .map(cr => `${cr.creatorType}: ${[cr.name, cr.firstName, cr.lastName].filter(n => n).join(', ')}`)
  .join(', ') %>

[acsr]

BTW comma-separation is easier (IMO) with

THX, looks almost pythonic. I am used to Python String KungFu and Jinja2 templates, but as Donald Knuth said: "Premature optimization is the root of all evil" but I have no fear when the dark force rises.

If I write my own translator I have to check for the suggested best practise for Zotero (including 7).
I would start here:
https://www.zotero.org/support/dev/translators
https://www.zotero.org/support/dev/translators/documentation

… In Zotero >= 2.0, translators are just JavaScript files. While Scaffold can ease translator development, some prefer to work directly on the filesystem. Here's how.

Hope this is not outdated, like other docs.

I originally came up to use the BetterBibTeX offer to craft an eta-template, when getting into limitations while using csl.

I am not remembering exactly the facts, but there were internal Zotero values from items seeming to be not reachable from the csl. The select item link magic was a blocker for me is my hunch.

[retorquere]

CSL discards a fair bit of information from the zotero items, but translators have all information.

There's nothing 7-specific for translators, what works on 6 will work on 7 and vice versa.

The doc you point to is still relevant.

[acsr]

So I am leaving here finally, it was a pleasure.