Zotero-better-bibtex: add date, origdate functions, and format-date filter

[Ypot]

KP76A4AB-euc

For citekeys referring to forum posts, I want to see complete date, with day and month. For books I use origyear.
So I want to use complete Date when no origyear exists. I have tried:
[origyear:(Date)]

But it doesn't work as I intended, because if there is a date in the fields, not origyear, it doesn't show the month and day, it just shows the year.
If I would edit it to:
[Date]
I works fine, showing year, month and day.

[label-gun[bot]]

It looks like you did not upload an debug report. The debug report is important; it gives @retorquere your current BBT settings and a copy of the problematic reference as a test case so he can best replicate your problem. Without it, @retorquere is effectively blind. Debug reports are useful for both bug analysis and enhancement requests; in the case of export enhancements, I need the copy of the references you have in mind.

1. If your issue relates to how BBT behaves around a specific reference(s), such as citekey generation or export, select at least one of the problematic reference(s), right-click it, and submit an BBT debug report from that popup menu. If the problem is with export, please do include a sample of what you see exported, and what you expected to see exported for these references.

2. If the issue does not relate to references and is of a more general nature, generate an debug report by restarting Zotero with debugging enabled (Help -> Debug Output Logging -> Restart with logging enabled), reproducing your problem, and selecting "Send Better BibTeX debug report..." from the help menu.

Once done, you will see a debug ID in red. Please post that debug id in the issue here.

Thank you!

[blip-bloop]

:robot: this is your friendly neighborhood build bot announcing test build 5.2.21.6365 ("adjust tests")

Install in Zotero by downloading test build 5.2.21.6365, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".

[retorquere]

In [origyear:(Date)], (Date) means the literal text Date. This is something I inherited when I decided tp copy the JabRef format -- in retrospect I should just have designed my own, but that's water under the bridge now.

For now, you might be helped using something like

[=forumPost][shorttitle][date] | [shorttitle][origyear]

In this:

• [=forumPost] means "break off key generation unless the item is a forum post". You can specify multiple types by separating them with a /.
• | means "If the pattern so far hasn't generated any text, restart with the following pattern". You can have multiple of these.
• [date] is new in 6365, and means "date if you have it, origdate if not"
• [origdate] is new in 6365, and means "origdate if you have it, date if not"
• [year] means "year if you have it, origyear if not"
• [origyear] means "origyear if you have it, year if not"

[blip-bloop]

:robot: this is your friendly neighborhood build bot announcing test build 5.2.21.6366 ("docs")

Install in Zotero by downloading test build 5.2.21.6366, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".

[retorquere]

Can you test whether 6366 provides a sufficient solution for you?

[blip-bloop]

:robot: this is your friendly neighborhood build bot announcing test build 5.2.21.6396 ("fixes #1488")

Install in Zotero by downloading test build 5.2.21.6396, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".

[Ypot]

Hi.
XUGAXILI-euc
It doesn't work it seems:

[origyear:(Date):prefix=.] generates-> .2019
[Date:prefix=.] generates -> .2019/01/2809:10:00

[retorquere]

[origyear:(Date):prefix=.] generates-> .2019

The filter (X) means "if the current block hasn't generated any text yet, output the literal text X, where X is Date in this case. But since origyear generates text, that will just be returned, prefixed with a .

[Date:prefix=.] generates -> .2019/01/2809:10:00

Here Date is a function, not a filter, and Date just means "whatever is in the Date field in zotero". I've added a filter format-date that will parse the date and change it into something regular. The default format is 0y-0m-0d, where a leading 0 means "right-adjust and fill with 0 if necessary, y means year, m means month, and d means day. Everything that's not one of those is just copied to the output. If you'd do [Date:format=date-0y-0m] you'd get 2019-01, if you do [Date:format-date=m.d.y] you'd get 1.28.2019, etc.

There isn't a way to say "if the current block didn't generate any text, call this function instead". There is a very limited way to do something like this with [>0] and | but [>0] looks at all blocks before it and starts the next pattern, it does not retain anything that's been partially generated.

[Ypot]

Thanks, then I did it as explained in post https://github.com/retorquere/zotero-better-bibtex/issues/1488#issuecomment-613313449

This is my Citekey generator:
[Auth:lower:capitalize][=forumPost/WebPage][Date:prefix=.][shorttitle3_3:lower:capitalize:prefix=.][Pages:prefix=.p.][Volume:prefix=.Vol.][NumberofVolumes:prefix=de] | [Auth:lower:capitalize][origyear:prefix=.][shorttitle3_3:lower:capitalize:prefix=.][Pages:prefix=.p.][Volume:prefix=.Vol.][NumberofVolumes:prefix=de]

Just the Date+hour doesn't fit well, I suppose format-date still can't parse the hour?

[retorquere]

Thanks, then I did it as explained in post #1488 (comment)

It's not the same. That comment describes date, you are using Date. These are not the same. [date] corresponds to [Date:format-date=0y-0m-0d].

[Auth:lower:capitalize][=forumPost/WebPage][Date:prefix=.][shorttitle3_3:lower:capitalize:prefix=.][Pages:prefix=.p.][Volume:prefix=.Vol.][NumberofVolumes:prefix=de] | [Auth:lower:capitalize][origyear:prefix=.][shorttitle3_3:lower:capitalize:prefix=.][Pages:prefix=.p.][Volume:prefix=.Vol.][NumberofVolumes:prefix=de]

The usual notation BTW is to put [=<type>] at the start. This will work all the same; [=<type>] just aborts the current pattern if there's a itemtype mismatch regardless of what was generated before.

Just the Date+hour doesn't fit well, I suppose format-date still can't parse the hour?

:format-date does not produce hour fields.

For all functions that start with a capital letter you must do all processing yourself. Functions that start with a lowercase letter have varying amounts of processing already applied. But if you request [Date:prefix=.], you'll get whatever is in the Zotero Date field, prefixed with a ., with no processing applied at all, and that will yield the time in the key if the time is in the Date field in Zotero.

[Ypot]

Thanks. I used "Date" because I want time in the Citekey: for Twitter or forums it can be important.

[retorquere]

In the citekey? To each their own 🤷 .

Man, that was quite the rabbithole. 6406 adds time support to format-date; I've also changed the spec format to be strftime-alike; I should be using (de facto or otherwise) standards more often instead of going it my own. That would make the date-and-time format: [Date:format-date=%Y-%m-%d.%H\:%M] (the backslash in the pattern is required to distinguish it from a start of a new filter). Not all strftime formatters are available, currently just yYmdHMS, and there's one minor addition in that yYmd can be prefixed with o to get their orig equivalents.

[Ypot]

Thanks a lot. It works great:
[=forumPost/WebPage][Auth:lower:capitalize][Date:format-date=%Y-%m-%d.%H\:%M\:%S:prefix=.][PublicationTitle1_1:lower:capitalize:prefix=.][shorttitle3_3:lower:capitalize:prefix=.][Pages:prefix=.p.][Volume:prefix=.Vol.][NumberofVolumes:prefix=de] | [Auth:lower:capitalize][origyear:prefix=.][PublicationTitle1_1:lower:capitalize:prefix=.][shorttitle3_3:lower:capitalize:prefix=.][Pages:prefix=.p.][Volume:prefix=.Vol.][NumberofVolumes:prefix=de]

A doubt: why are we using Date (with a capital letter) instead of date?

I'm not sure if the "o" prefix is working: [Date:format-date=%oY] nor [date:format-date=%oY]
KPKAIVCU-euc

[blip-bloop]

:robot: this is your friendly neighborhood build bot announcing test build 5.2.21.6408 ("also uses extra-fields for orig-date for #1488")

Install in Zotero by downloading test build 5.2.21.6408, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".

[retorquere]

Man that was another rabbit-hole.

A doubt: why are we using Date (with a capital letter) instead of date?

because before 6408, [date] was just an alias for [Date:format-date=%Y-%m-%d], as I outlined above.

I'm not sure if the "o" prefix is working: [Date:format-date=%oY] nor [date:format-date=%oY]

Using [date:format-date=%Y] would only work because format-date parses the input into a date, and the date resulting from [date] is easy to parse, but [date:format-date=%H] would always yield an empty string, because [date] never includes time info. [Date] is just the contents of the Zotero Date field. Whatever is in that field is present in [Date].

So, the explanation: %oY does work, but your Date field doesn't have orig info (it can, but in your case it is in another field), so there's nothing to pick up. The orig info is in the extra field, and Date doesn't look there.

6408 changes things a bit. [date] is still best thought of as an alias for [Date:format-date=%Y-%m-%d], but [date] can now take a format string directly, and [date] does merge orig-date from the extra field. So [date=%oY] will also pick up an orig-date from the extra field. I'll need to update the docs for all of that.

[retorquere]

So it works to your satisfaction now?

[retorquere]

I'd really prefer to know it works for you as intended before merging.

[retorquere]

I'm looking to cut a new release in the next few days and I'd prefer to roll this into that release. But for that I really do need to know that no other changes will be required for your use-case.

[Ypot]

Hi.
Sorry for the delay. It already worked great, now trying the new update 5.2.21.6408, something happened. It sent an error message saying that there were problems generating citekeys, and the addon started a setup like if a new installation. It doen't recognize my citekey format now and changed it to a basic [auth:lower][shorttitle3_3][year]

KNIJAPTV-euc

[retorquere]

Dammit. What was the format you used before?

[retorquere]

Oh hold on, I see it. On it.

[Ypot]

5.2.21.6406 works like a charm though. For me it's good.
HV4E6J8A-euc

[retorquere]

Damn. OK, so your format has an error, and that's checked during startup, but BBT should have allowed you to correct it. 6408 does stricter checking on errors.

The key has PublicationTitle1_1, and that's not meaningful to the format parser. I'll have to think about how to handle that on startup. 6406 would just ignore that section.

[retorquere]

What 6406 does is not good perse because it allows you to enter gibberish without feedback.

[retorquere]

And specifically, 6408 would never have allowed you to enter it.

[retorquere]

6430 will remove the invalid parts during startup and notify you about them but will leave the pattern unchanged otherwise.

[blip-bloop]

:robot: this is your friendly neighborhood build bot announcing test build 5.2.21.6431 ("save when valid")

Install in Zotero by downloading test build 5.2.21.6431, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".

[retorquere]

And specifically, 6408 would never have allowed you to enter it.

I had posted an explanation of this, but I don't see it anymore. Don't know what happened to it, but here goes again:

If you fill out a pattern, you'll have noticed that during typing, the field turns orange, and the tooltip shows an error. As long as the pattern is invalid, the field stays orange; if you get to a point where the pattern is valid, the field turns white.

The pattern is only saved when it's valid. If you take the focus away from the edit field, by e.g. tabbing to another field, clicking another field, or closing the preferences screen (indicating you are done editing the pattern) while it is still orange, the pattern is restored to the previous, assumed valid version. This assumption is in place because a) BBT really can't work without a key generator, and b) because you start with a valid pattern at first-run of BBT, and you can only save valid patterns, the previous pattern should always be valid.

versions before 6408 were slightly broken in that there was one specific edge case (being invalid input at the end of a property field -- fields that start with a capital letter) would be ignored by accident. It would never generate any output, so it would be useless in the pattern, but it was not recognized as an error by the formatter. So these versions would allow saving of an invalid pattern.

6408 and later checks for existence of property fields (and coincidentally allows for misspellings -- NumberofVolumes should have been an error in < 6408 but is now valid) so it detected the error, but then during startup demands a valid pattern, so falls back to the initial default.

6431 will strip invalid parts from the pattern during startup, and will notify you about that it does this. The valid parts remain. I have added the key you entered in the test suite so that this stripping is tested. If all parts end up being stripped, BBT will still fall back to the initial default.

Can you try 6431?

[Ypot]

Yes, it seems it removed this part: [PublicationTitle1_1:lower:capitalize:prefix=.]

[Ypot]

I had it on my config to add chapter to the citekey, but in fact I am not using citekeys for chapters, so it's ok.

For me, the update is good. But maybe you need to know that I think [date=%oY] doesn't seem to work.
And, BTW, if you think original-date should be in a different field, let me know. I just prefer original date, over the edition publication date ;)

[retorquere]

[date=%oY] does work for me. If it doesn't work for you, I'll need a new debug log. But your previous pattern had [date:%oY] (which is not valid, there is no filter %oY), not [date=%oY].

[retorquere]

[PublicationTitle1_1] is not valid; "functions" that start with a capital letter fetch zotero item fields, and there is no field PublicationTitle1_1, just PublicationTitle.

[retorquere]

Zotero doesn't have a field for original date; I was just explaining why Date didn't pick it up, not saying you should do it differently.

[retorquere]

The equivalent for PublicationTitle1_1 is [PublicationTitle:select=1,1:capitalize]

[retorquere]

I really need to know about the %oY issue. I need to cut a new release and I want to include this change.

[Ypot]

I really need to know about the %oY issue. I need to cut a new release and I want to include this change.

It works!
Thanks for the alternative option for PublicationTitle ^^

[=forumPost/WebPage][Auth:lower:capitalize][Date:format-date=%Y-%m-%d.%H\:%M\:%S:prefix=.][shorttitle3_3:lower:capitalize:prefix=.][Pages:prefix=.p.][Volume:prefix=.Vol.][NumberofVolumes:prefix=de] | [Auth:lower:capitalize][date=%oY:prefix=.][shorttitle3_3:lower:capitalize:prefix=.][Pages:prefix=.p.][Volume:prefix=.Vol.][NumberofVolumes:prefix=de]
Just for the record ;)

[retorquere]

Thanks for the confirmation, I'll probably cut the release this weekend, still a few more issues to get closure on.

Wrt property access, it's not an alternative, this is just how property access has worked all along. BBT < 6408 would silently ignore the error, but the way you wrote it earlier would never under any circumstance have yielded output under any version of bbt all the way back to the first version. When I said "equivalent" I meant "I see what you mean, here's how to do it" rather than "that doesn't work anymore, here's how to do it now".

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.