Zotero-better-bibtex: Support for already existing documents

[zavidos]

Hello and thanks for the great work on better bibtex.
I would like to know if it is possible (or suggest to implement if it is not) to substitute all zotero citations existing on a document with the corresponding bibtex citekey.
I would like to move a big document from word to scrivener, up to now I used zotero plugin for word and I'm moving to scrivener+pandoc workflow assisted by zotero+better bibtex plugin.

Thank you for the support

[retorquere]

I'm not sure what you mean. You mean change all references in an existing latex document? That's not possible, there's not enough information to do the mapping. Or do you mean change all BBT references to match what you already have in your documents? That may be possible, but you'd have to tell me what your references in your documents look like.

[retorquere]

Or do you mean a conversion of a word document + zotero references to pandoc + pandoc references?

[zavidos]

Exactly, from a document I prepared with word and Zotero, I want to move to scrivener using pandoc citation and better bibtex. So I would like to know if I can transform all the citation I already have in word made with Zotero to pandoc citekey like [@Scully2009]

Il giorno 16 dic 2017, alle ore 20:52, Emiliano Heyns [e-mail] ha scritto:

Or do you mean a conversion of a word document + zotero references to pandoc + pandoc references?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[retorquere]

Can you get me a sample document?

[retorquere]

And it'd be best of the citation style of the sample document is set to APA6.

[zavidos]

Sure, here you have an example of the document with my favorite citation style and the same with APA6. I attached also my zotero library exported in different formats. (what is the difference betweek bibtex json and csl json?
https://www.dropbox.com/s/lh0ii273w551uzf/test%20article.zip?dl=0 https://www.dropbox.com/s/lh0ii273w551uzf/test%20article.zip?dl=0

Thanks

Davide

Il giorno 17 dic 2017, alle ore 01:59, Emiliano Heyns <[email protected] [e-mail]> ha scritto:

Can you get me a sample document?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/retorquere/zotero-better-bibtex/issues/849#issuecomment-352188547, or mute the thread https://github.com/notifications/unsubscribe-auth/Ag_Vyh2NRdV1bTYVinsUltCvxNybsEJrks5tA9q7gaJpZM4REP01.

[retorquere]

Thanks, I was going to ask for a BBT JSON export. The CSL JSON format is what Pandoc and Citeproc-based bibliography processors support; BBT JSON is just a dump of the internal representation of the Zotero references which I use for testing and debugging.

In LibreOffice I can store the Zotero references in two ways; as bookmarks and as fields (settable from within LO). Do you have a similar choice in Word? If I'd have to guess you do, and you have it set to bookmarks. Can you set it to fields instead?

[zavidos]

I checked and it was already set to field, here https://www.dropbox.com/s/osizikfks45xgd3/test%20article%20bookmarks%20and%20field.zip?dl=0 you can find both the version (field and bookmarks).

So there is no existing way to give to zotero a document, even RTF and say “search for citation and substitute with the field citekey”?

When I was looking how to use zotero and scrivener I studied the “RTF scan” workflow, but the difference is that RTF scan is also a zotero style and you can write your docx or take an existing one with citations, change citations style to RTF scan, it would be nice if there was a citation style that change just to citekey, but maybe the problem is that he have to read the citekey from zotero

Il giorno 17 dic 2017, alle ore 08:52, Emiliano Heyns <[email protected] [e-mail]> ha scritto:

Thanks, I was going to ask for a BBT JSON export. The CSL JSON format is what Pandoc and Citeproc-based bibliography processors support; BBT JSON is just a dump of the internal representation of the Zotero references which I use for testing and debugging.

In LibreOffice I can store the Zotero references in two ways; as bookmarks and as fields (settable from within LO). Do you have a similar choice in Word? If I'd have to guess you do, and you have it set to bookmarks. Can you set it to fields instead?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/retorquere/zotero-better-bibtex/issues/849#issuecomment-352214374, or mute the thread https://github.com/notifications/unsubscribe-auth/Ag_VyjrANdthV1YT51DKVFMfOFi0PoKpks5tBDuNgaJpZM4REP01.

[retorquere]

There isn't such a mechanism currently, no, and I don't know the parts of Zotero that deal with word at all.

I really don't like RTF scan, because the labels it searches for are not necessarily unique, and I'd have to re-create the RTF-scan text scanner to be able to do citekey lookups. That's not something I want to invest my time in. The ODF scanner is lots better, but unfortunately it cannot be recreated using a CSL style, as the ID field is not accessible as a style variable.

Can you get me an RTF version of that article? Perhaps there info I'm looking for is kept in the RTF. The bookmarks version does have something that's parsable but they're not reference IDs.

[zavidos]

This https://www.dropbox.com/s/yk1i0ukuspfzvx2/test%20article%20-%20CSL%20RTF%20(field).docx?dl=0 is the same article with RTF style citation and set to "field."
I never used ODF scanner, is there a way to do it with other software? Will openoffice help me doing this? I have my phd thesis that is in progress and I really want to move it on scrivener but I can’t substitute all the citation manually, it will take forever

Il giorno 17 dic 2017, alle ore 09:27, Emiliano Heyns [e-mail] ha scritto:

There isn't such a mechanism currently, no, and I don't know the parts of Zotero that deal with word at all.

I really don't like RTF scan, because the labels it searches for are not necessarily unique, and I'd have to re-create the RTF-scan text scanner to be able to do citekey lookups. That's not something I want to invest my time in. The ODF scanner is lots better, but unfortunately it cannot be recreated using a CSL style, as the ID field is not accessible as a style variable.

Can you get me an RTF version of that article? Perhaps there info I'm looking for is kept in the RTF. The bookmarks version does have something that's parsable but they're not reference IDs.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/retorquere/zotero-better-bibtex/issues/849#issuecomment-352216377, or mute the thread https://github.com/notifications/unsubscribe-auth/Ag_VyucnESOBy8DAMk0ZGFzcdCyrIdQGks5tBEPngaJpZM4REP01.

[retorquere]

When I download that file it says it's a .docx file. Ideally, I'd get a bookmarks and a fields version, saved as a .rtf file.

I really have no idea how well a conversion to ODT would work -- I hardly ever use LibreOffice (recommended over OpenOffice these days) or Word. I really can't help out there, sorry. What I'm looking for is a way to generate a file that has identifiable citation markers while not having to mess with parsing docx directly, which I'm not getting into; I've so far been trying to find something that pandoc would convert to markdown while leaving reference IDs in place, but only the ODT -> markdown conversion seems to do this so far. I'm hoping the RTF versions fare better.

I'm also not aware of any way to convert citations to ODF-scan compatible markers, really.

[zavidos]

Oh sorry, I thought you wanted RTF style citation, not rtf format document. Here https://www.dropbox.com/s/67jbjhr4vtf0b03/RTF.zip?dl=0 you can find both APA and RTF scan citation style and both field and bookmarks documents in RTF format.

Maybe using RTF scan citation style in the word document and then creating with zotero a sort of table with citekey and RTF style citation for substitution.
For example, the citation " Wang et al., "The effect of fluoride ions on the corrosion behavior of pure titanium in 0.05 M sulfuric acid", 2014 “ have citekey @wang2014. This citation format is RTF scan. Can I export from zotero RTF scan citation for each document together with its citekey?

Il giorno 17 dic 2017, alle ore 09:47, Emiliano Heyns [e-mail] ha scritto:

When I download that file it says it's a .docx file. Ideally, I'd get a bookmarks and a fields version, saved as a .rtf file.

I really have no idea how well a conversion to ODT would work -- I hardly ever use LibreOffice (recommended over OpenOffice these days) or Word. I really can't help out there, sorry. What I'm looking for is a way to generate a file that has identifiable citation markers while not having to mess with parsing docx directly, which I'm not getting into; I've so far been trying to find something that pandoc would convert to markdown while leaving reference IDs in place, but only the ODT -> markdown conversion seems to do this so far. I'm hoping the RTF versions fare better.

I'm also not aware of any way to convert citations to ODF-scan compatible markers, really.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/retorquere/zotero-better-bibtex/issues/849#issuecomment-352217317, or mute the thread https://github.com/notifications/unsubscribe-auth/Ag_VytRIMlMT0JPe6Qza45gmk18BSsfpks5tBEiagaJpZM4REP01.

[retorquere]

If that could be done the problem would be simple to solve, but the citekey isn't available to the citation processor, nor is any other unique identifier of the reference (its internal URL, itemID or itemKey). If any of those would be available it'd be easy to replace the marks with the citekeys.

[retorquere]

AAMOF, if there's any unique ID in your refs (DOI, URL, doesn't matter what), this is solvable. Or if you could get someone to create something like rtf-scan.csl to put unique IDs in the formatted references. I don't know enough about CSL to help.

[zavidos]

So, just to see if I understood the problem correctly..
The citation processor, that in this case is part of zotero, sees only fields like: “series, title, volume, issn, doi, abstract, urldate, journal, author, month, year, keyword…” but it doesn’t see any custom field like “citekey”, so it is not possible to put citekey in a .csl style. However it would be possible to create a style that replace in the text one of the existing field (like DOI), then having in the text all the citation with just the doi replace them with citekey. Is it correct? But once I have a document in which all the citations are just DOI, how can I convert it to citekey? I need a table that associates DOI with citekey. Is it possible to export from zotero this kind of table? I saw that I can export "bibtex citation key quick copy” but is just a text with all the citekey. However it means that zotero can export citekey and can export all the other information in a .bib or .json, so in principle it should be easy to associate the two, right?

Il giorno 17 dic 2017, alle ore 10:38, Emiliano Heyns <[email protected] [e-mail]> ha scritto:

AAMOF, if there's any unique ID in your refs (DOI, URL, doesn't matter what), this is solvable. Or if you could get someone to create something like rtf-scan.csl to put unique IDs in the formatted references. I don't know enough about CSL to help.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/retorquere/zotero-better-bibtex/issues/849#issuecomment-352219815, or mute the thread https://github.com/notifications/unsubscribe-auth/Ag_VysX0zuNATz4OSERaP-5DlFYA6Uxiks5tBFSQgaJpZM4REP01.

[retorquere]

The citation processor, that in this case is part of zotero, sees only fields like: “series, title, volume, issn, doi, abstract, urldate, journal, author, month, year, keyword…” but it doesn’t see any custom field like “citekey”, so it is not possible to put citekey in a .csl style.

That is correct. Zotero does know about its own unique IDs (itemID/itemKey/itemURL) and even puts the itemID in the CSL id variable, but the citation processor (citeproc) doesn't seem to want to use it.

However it would be possible to create a style that replace in the text one of the existing field (like DOI), then having in the text all the citation with just the doi replace them with citekey. Is it correct?

Correct. A CSL style can address any reference field, so yes, it could output the DOI. If all in-text citations would consist of an unique ID for their corresponding reference I could then do something to look those up and replace them with the correct citekey. Doesn't specifically need to be the DOI field either, just as long as it's something that can be output in a CSL variable; you can play around with that here; id (which is the CSL name for the itemID) contains the internal ID of the reference, which would be enough for me, but you can't add the id variable in a CSL style.

But once I have a document in which all the citations are just DOI, how can I convert it to citekey? I need a table that associates DOI with citekey. Is it possible to export from zotero this kind of table? I saw that I can export "bibtex citation key quick copy” but is just a text with all the citekey. However it means that zotero can export citekey and can export all the other information in a .bib or .json, so in principle it should be easy to associate the two, right?

Yes. This is not the hard part. The hard part is getting the in-text citations to have unique IDs. The rtf-scan {....} fields sort of serve that purpose but I'd have to figure out how they are produced and I'd have to replicate that so I can do the lookup. I really don't want to do that.

[zavidos]

No wait.. the file .bib or .json already have the citekey inside. Because when I use [@authorYEAR] in pandoc and then compile using .bib file it associates the citekey with the proper citation. So in theory we have all the information. Using RTF scan citation style we have citations like “(Wang, Hu, Zheng, Ke, & Qiao, 2016)” in the text. All this informations are already stored in the .bib file together with the ID (citekey) that in this case is Wang2016. I can intuitively understand that the process is reversible because we have everything but I don’t know how to do it.

Or maybe the problem you are referring to is that RTF scan is not unique so we can’t be sure that the citekey associated with (Wang, Hu, Zheng, Ke, & Qiao, 2016) is wang2016. But the file .bib contains everything

Il giorno 17 dic 2017, alle ore 10:48, davide <[email protected] [e-mail]> ha scritto:

So, just to see if I understood the problem correctly..
The citation processor, that in this case is part of zotero, sees only fields like: “series, title, volume, issn, doi, abstract, urldate, journal, author, month, year, keyword…” but it doesn’t see any custom field like “citekey”, so it is not possible to put citekey in a .csl style. However it would be possible to create a style that replace in the text one of the existing field (like DOI), then having in the text all the citation with just the doi replace them with citekey. Is it correct? But once I have a document in which all the citations are just DOI, how can I convert it to citekey? I need a table that associates DOI with citekey. Is it possible to export from zotero this kind of table? I saw that I can export "bibtex citation key quick copy” but is just a text with all the citekey. However it means that zotero can export citekey and can export all the other information in a .bib or .json, so in principle it should be easy to associate the two, right?

Il giorno 17 dic 2017, alle ore 10:38, Emiliano Heyns <[email protected] [e-mail]> ha scritto:

AAMOF, if there's any unique ID in your refs (DOI, URL, doesn't matter what), this is solvable. Or if you could get someone to create something like rtf-scan.csl to put unique IDs in the formatted references. I don't know enough about CSL to help.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[zavidos]

rtf-scan seems to be just all the author separated by a comma and the last author separated by a &, plus the year. If there where no articles with the same first author in the same year in principle we just need to delete all the authors in between to transform (Wang, Hu, Zheng, Ke, & Qiao, 2016) into wang2016. But it doesn’t solve the uniqueness problem, however if it would be possible to export rtf-scan, so just export all the author and the year, and in another field export the citekey, then they could be substituted. Right?

Il giorno 17 dic 2017, alle ore 11:05, Emiliano Heyns [e-mail] ha scritto:

The citation processor, that in this case is part of zotero, sees only fields like: “series, title, volume, issn, doi, abstract, urldate, journal, author, month, year, keyword…” but it doesn’t see any custom field like “citekey”, so it is not possible to put citekey in a .csl style.

That is correct. Zotero does know about its own unique IDs (itemID/itemKey/itemURL) and even puts the itemID in the CSL id variable, but the citation processor (citeproc) doesn't seem to want to use it.

However it would be possible to create a style that replace in the text one of the existing field (like DOI), then having in the text all the citation with just the doi replace them with citekey. Is it correct?

Correct. A CSL style can address any reference field, so yes, it could output the DOI. If all in-text citations would consist of an unique ID for their corresponding reference I could then do something to look those up and replace them with the correct citekey. Doesn't specifically need to be the DOI field either, just as long as it's something that can be output in a CSL variable; you can play around with that here http://editor.citationstyles.org/visualEditor/; id (which is the CSL name for the itemID) contains the internal ID of the reference, which would be enough for me, but you can't add the id variable in a CSL style.

But once I have a document in which all the citations are just DOI, how can I convert it to citekey? I need a table that associates DOI with citekey. Is it possible to export from zotero this kind of table? I saw that I can export "bibtex citation key quick copy” but is just a text with all the citekey. However it means that zotero can export citekey and can export all the other information in a .bib or .json, so in principle it should be easy to associate the two, right?

Yes. This is not the hard part. The hard part is getting the in-text citations to have unique IDs. The rtf-scan {....} fields sort of serve that purpose but I'd have to figure out how they are produced and I'd have to replicate that so I can do the lookup. I really don't want to do that.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/retorquere/zotero-better-bibtex/issues/849#issuecomment-352221038, or mute the thread https://github.com/notifications/unsubscribe-auth/Ag_Vyr0NakI7UaQMz9zTwiKAqf2TS8-Jks5tBFrfgaJpZM4REP01.

[zavidos]

No sorry I was looking at apa6 style not at rtf scan when I said that it is only authors plus the year, but the question remain the same

Il giorno 17 dic 2017, alle ore 11:05, Emiliano Heyns [e-mail] ha scritto:

The citation processor, that in this case is part of zotero, sees only fields like: “series, title, volume, issn, doi, abstract, urldate, journal, author, month, year, keyword…” but it doesn’t see any custom field like “citekey”, so it is not possible to put citekey in a .csl style.

That is correct. Zotero does know about its own unique IDs (itemID/itemKey/itemURL) and even puts the itemID in the CSL id variable, but the citation processor (citeproc) doesn't seem to want to use it.

However it would be possible to create a style that replace in the text one of the existing field (like DOI), then having in the text all the citation with just the doi replace them with citekey. Is it correct?

Correct. A CSL style can address any reference field, so yes, it could output the DOI. If all in-text citations would consist of an unique ID for their corresponding reference I could then do something to look those up and replace them with the correct citekey. Doesn't specifically need to be the DOI field either, just as long as it's something that can be output in a CSL variable; you can play around with that here http://editor.citationstyles.org/visualEditor/; id (which is the CSL name for the itemID) contains the internal ID of the reference, which would be enough for me, but you can't add the id variable in a CSL style.

But once I have a document in which all the citations are just DOI, how can I convert it to citekey? I need a table that associates DOI with citekey. Is it possible to export from zotero this kind of table? I saw that I can export "bibtex citation key quick copy” but is just a text with all the citekey. However it means that zotero can export citekey and can export all the other information in a .bib or .json, so in principle it should be easy to associate the two, right?

Yes. This is not the hard part. The hard part is getting the in-text citations to have unique IDs. The rtf-scan {....} fields sort of serve that purpose but I'd have to figure out how they are produced and I'd have to replicate that so I can do the lookup. I really don't want to do that.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/retorquere/zotero-better-bibtex/issues/849#issuecomment-352221038, or mute the thread https://github.com/notifications/unsubscribe-auth/Ag_Vyr0NakI7UaQMz9zTwiKAqf2TS8-Jks5tBFrfgaJpZM4REP01.

[retorquere]

As I said -- the rtf-scan style is in principle intended to be matched this way, but I'm not taking on the work of doing the matching. If there's an easy way to make a match, I'll gladly help, but re-implementing the rtf-scanner built into Zotero is really outside the scope of BBT.

[zavidos]

Yes sure, I’m not asking you any hard work, you are already too kind in helping with this.

Is it possible to copy the citekey in another field? I saw that the field “note” that in zotero is called “extra” is available to csl style, so if it would be possible to have the citekey in that field (or another one but I think “extra” is never used) it would be easy

Il giorno 17 dic 2017, alle ore 11:22, Emiliano Heyns [e-mail] ha scritto:

As I said -- the rtf-scan style is in principle intended to be matched this way, but I'm not taking on the work of doing the matching. If there's an easy way to make a match, I'll gladly help, but re-implementing the rtf-scanner built into Zotero is really outside the scope of BBT.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/retorquere/zotero-better-bibtex/issues/849#issuecomment-352221702, or mute the thread https://github.com/notifications/unsubscribe-auth/Ag_VyiY9IJSL_rbwIpCXJ957hmcJUTDuks5tBF6sgaJpZM4REP01.

[retorquere]

That'd happen automatically if you pin the citekeys; pinning just means "copy the citekey to the extra field". If you don't use your extra field for anything else, that would make things super easy.

I was also looking at an exporter that would export a CSV with the rtf-scan field and the citekey, but I can't access the citation processor in the exporter, so I can't produce the rt-scan field without reimplementing the code that produces those.

[zavidos]

This is really interesting, however if I pin the citekey in the extra appears “bibtex: citekey” instead of the citekey only and surprisingly in my docx document using the new CSL I don’t see bibtex: citekey, it is clear, but if I remove manually “bibtex: “ i see correctly the citekey. So it seems the presence of bibtex: in the extra field prevent it to be correctly shown

Il giorno 17 dic 2017, alle ore 11:32, Emiliano Heyns [e-mail] ha scritto:

That'd happen automatically if you pin the citekeys; pinning just means "copy the citekey to the extra field". If you don't use your extra field for anything else, that would make things super easy.

I was also looking at an exporter that would export a CSV with the rtf-scan field and the citekey, but I can't access the citation processor in the exporter, so I can't produce the rt-scan field without reimplementing the code that produces those.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/retorquere/zotero-better-bibtex/issues/849#issuecomment-352222193, or mute the thread https://github.com/notifications/unsubscribe-auth/Ag_Vyi_s3HEXmlRqwnhlRcJWghbp2LOOks5tBGEugaJpZM4REP01.

[retorquere]

I see. I have an idea on why that happens; Zotero recommends the x: y as the recommended general format for adding metadata, and citeproc might strip all metadata regardless of whether it understands it itself. But if that's the case, I can't fix this route.

I'll think about another way, but it looks like we'll need help from the zotero authors to get this to work.

[zavidos]

so citeproc remove the field extra because of the “bibtex:” in front of the text. I guess that the addition of “bibtex:” is mandatory and not part of your plugin. Is it the only way to copy citeproc to another field? Maybe it is possible to search in the zotero database and remove all the "bibtex: “?

So if I understood correctly, the problem is not the word bibtex: but the fact that there is a word and a colon in the extra field. Right?

Il giorno 17 dic 2017, alle ore 20:18, Emiliano Heyns <[email protected] [e-mail]> ha scritto:

I see. I have an idea on why that happens; Zotero recommends the x: y as the recommended general format for adding metadata, and citeproc might strip all metadata regardless of whether it understands it itself. But if that's the case, I can't fix this route.

I'll think about another way, but it looks like we'll need help from the zotero authors to get this to work.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/retorquere/zotero-better-bibtex/issues/849#issuecomment-352242198, or mute the thread https://github.com/notifications/unsubscribe-auth/Ag_VylunUUKaf6O5ZDIVuTRyKls4rlFrks5tBNxlgaJpZM4REP01.

[retorquere]

The bibtex: is part of my plugin, but it's the way I recognize that data in the extra field is mine. I can't remove that without doing damage to how BBT works.

The ODT-refmarks format shows promise as a source document for the conversion, but currently pandoc dies if I try to convert such a document. I'm looking into that.

[zavidos]

perhaps you can add an option button that allows you to not add "bibtex:”? it doesn’t sound complicated but I have no idea actually so maybe is too much work

Il giorno 17 dic 2017, alle ore 20:26, Emiliano Heyns [e-mail] ha scritto:

The bibtex: is part of my plugin, but it's the way I recognize that data in the extra field is mine. I can't remove that without doing damage to how BBT works.

The ODT-refmarks format shows promise as a source document for the conversion, but currently pandoc dies if I try to convert such a document. I'm looking into that.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/retorquere/zotero-better-bibtex/issues/849#issuecomment-352242609, or mute the thread https://github.com/notifications/unsubscribe-auth/Ag_VyhON0RF0yIhj8UhYy0Zr8X52Gegrks5tBN5NgaJpZM4REP01.

[retorquere]

The bibtex: is an important part of how BBT works; removing it would mean the functionality would change from "pin citekey" to "add the citekey to the notefield", with no good means to remove it again. It's not an area I want to move into. The docx has enough info to do the mapping, but docx is a very convoluted format that's hard to parse. Pandoc can convert ODT+refmarks to markdown while retaining enough information to replace the citekeys, but currently pandoc crashes if I do

1. open word+bookmarks and save as odt+refmarks
2. convert odt+refmarks to markdown

If I just create my own odt+refmarks, pandoc converts without a problem.

[zavidos]

I don’t know the difference between bookmarks and refmarks. No other ways in your knowledge to copy one field into another in zotero? Moving the citekey to extra in another way will solve the problem quickly. Even having the CSL to write “bibtex: citekey” instead of nothing would solve because it is possible to remove "bibtex: “ from the word document. So maybe you know how to have word reporting the full “extra” field even if a word with a colon is present inside the field

Il giorno 17 dic 2017, alle ore 20:53, Emiliano Heyns [e-mail] ha scritto:

The bibtex: is an important part of how BBT works; removing it would mean the functionality would change from "pin citekey" to "add the citekey to the notefield", with no good means to remove it again. It's not an area I want to move into. The docx has enough info to do the mapping, but docx is a very convoluted format that's hard to parse. Pandoc can convert ODT+refmarks to markdown while retaining enough information to replace the citekeys, but currently pandoc crashes if I do

open word+bookmarks and save as odt+refmarks
convert odt+refmarks to markdown
If I just create my own odt+refmarks, pandoc converts without a problem.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/retorquere/zotero-better-bibtex/issues/849#issuecomment-352243985, or mute the thread https://github.com/notifications/unsubscribe-auth/Ag_VysuoSIhgPiGdEgEzjblsDeQjfQhFks5tBOSrgaJpZM4REP01.

[blip-bloop]

:robot: this is your friendly neighborhood build bot announcing test build 5155 ("one-off for #849 -- DO NOT RELEASE").

[retorquere]

Bookmarks and refmarks are just the two ways the zotero data can be stored in the ODT document, akin to how you can choose between bookmarks and fields in Word (appearantly -- I don't have a system with Office on hand).

I have zero knowledge on how citeproc and the zotero-word integration works -- you'd have to ask the citeproc and zotero devs.

Build 5155 will mangle the "pin citekeys" functionality to skip the bibtex: part, but I'd really recommend backing up your DB, as I have not tested this and I have no way to reverse this. If you copy these keys in the extra field, they will stay there as they are now, even if the citekey changes later -- BBT will regard them as just any user-entered text in the extra field and will not touch it in any way. No other build than 5155 will do this -- in the formal releases, pin-citekeys will include the bibtex: part. Again -- once these keys are copied into the extra field, you will have to manually remove them or restore the backup if you want them gone.

[zavidos]

ahaha thank you so much friendly neighbourhood built bot, I can't stop laughing

[zavidos]

Ok this should solve my problem, so my work in progress thesis could be moved to scrivener+pandoc workflow, however if another solution comes into your mind it would be nice for the other people in the same situation. I think is quite common to have a big document or a review that is continuously updated with time

[retorquere]

Hold on for a bit, I might have another solution

[retorquere]

I do realize this is a common scenario, but a clean solution would require help from citeproc and Zotero.

[zavidos]

I will keep thinking about, and see if I can get other help. In the mean time thank you so much for all the assistance and for the work you made with better bibtex. You should add a “buy me a beer” button in your github page

Il giorno 17 dic 2017, alle ore 22:18, Emiliano Heyns <[email protected] [e-mail]> ha scritto:

I do realize this is a common scenario, but a clean solution would require help from citeproc and Zotero.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/retorquere/zotero-better-bibtex/issues/849#issuecomment-352248441, or mute the thread https://github.com/notifications/unsubscribe-auth/Ag_VynhYMBgeVgD_C3C94ElNpooZVJg0ks5tBPiPgaJpZM4REP01.

[retorquere]

article.txt

[retorquere]

With 5156 and this style you should be able to get cite keys in your document.

If you want to edit it in the visual style editor, you will have top open it first in a text editor and replace "citekey" with "status"

pandoc-migration.csl.txt

[retorquere]

5157 has a hidden pref (citeprocNoteCitekey) which when enabled will replace the note field with the citekey for rendering citation/bibliographies in Word/LibreOffice. This style can be used as a starting point to display them. In principle it should be possible to hack any of the existing styles such as apa.csl to also include pandoc-compatible page markers etc, but I'm not going there myself. I'll be happy to leave this hidden pref in place in future releases.

[blip-bloop]

:robot: this is your friendly neighborhood build bot announcing test build 5157 ("hidden pref").

[zavidos]

Hello, sorry for the delay, it was night. If I understand correctly the new csl style doesn't cite the "extra" key but search directly for a field called citekey, at least this is what I understand because the name of the variable in the inline citation is not "note" anymore but became "citekey". However in this case I don't understand the role of 5157 and 5158. _"replace the note field with the citekey for rendering citation/bibliographies in Word/LibreOffice"_ means that instead of a note field now the file has a citekey field? And most important, how do I enable "hidden pref (citeprocNoteCitekey)"?

thanks

edit:
pandoc-migration csl never display anything, even with something in the "extra" field, while "better-bibtex-citekey.csl" shows correctly in the text if something is in the extra field but I have to use the 5155 to copy citekey into extra without "bibtex: ". You gave me pandoc-migration.csl saying I should use with 5156 but I don't have 5156.

[retorquere]

Forget pandoc-migration and 5156; 5157 will put the bare citekey in the CSL note variable if you enable the hidden pref citeprocNoteCitekey; that pref is currently not documentend but you can find how to set hidden prefs here.

With that enabled, you can just use the visual editor to display the note variable and that will have the citekey; I've made a very simple style that does this here.

[zavidos]

got it, really nice, it seems to work smoothly :)

[retorquere]

Super, I'll include this in the next release. You can just keep using 5057, it will automatically be upgraded.

[zavidos]

It seems the problems never end. The system in principle works, but I made my document with zotero 4, and now installed zotero 5. If I open the document and change citation style everything works, because the information are stored in the citation, but if I use “better-bibtex-citekeys.csl” the citation is broken, but if I cite again the same article two thing can happen, if I cite from the list of the already cited in the article it says “CSL STYLE ERROR: reference with no printed form.”, if I cite the same article but from the general list it is correctly displayed as @authorYEAR. So I assume the problem is that passing from zotero 4 to zotero 5 the citations are not properly linked to the software. Will your plugin work with zotero 4 if I restore an old backup?

Il giorno 18 dic 2017, alle ore 10:53, Emiliano Heyns <[email protected] [e-mail]> ha scritto:

Closed #849 https://github.com/retorquere/zotero-better-bibtex/issues/849.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/retorquere/zotero-better-bibtex/issues/849#event-1391260467, or mute the thread https://github.com/notifications/unsubscribe-auth/Ag_Vyq3Tkezj3TvjxiotziarJIl0xF2vks5tBalygaJpZM4REP01.

[retorquere]

No, the plugin will not work with zotero 4 at all. You can try the same style without BBT installed, it should just get you whatever is in the note field. If the note field isn't empty and you still get these errors then there's something wrong with the style. I'm not an expert at csl style development.

[retorquere]

A fast check would be to export your references to CSL JSON with the hidden Pref enabled. The note field should have the citekey everywhere.

[retorquere]

Another quick check is to select your references, right-click, and let Zotero generate a bibliography. If that works, then the error is inside the word plugin.

[zavidos]

No I think the problem is broken citation, because if I have an article (it happens just with some), that is cited in a paper and I change the citation style, let’s say from journal of applied electrochemistry to APA6, everything is fine, but if I change to betterbibtex or to the style I made to see the field “note” it give error. If I add a new citation and you know you have the list of the already cited articles in your document and the list of all the articles in zotero, if I chose the one from the already cited the citation is broken but if I chose from the whole list it is correctly displayed. Then I tried to modify for example the author and when I search for a paper to cite I see the old author in the “already cited” list, and the modified version in the “whole list”.

So i tried right click —> activate/deactivate field code (or something like that, I don’t have the english version) and

for the first one, the one that have no “note” field:

{ ADDIN ZOTERO_ITEM CSL_CITATION {"citationID":"a2gla06nt3o","properties":{"formattedCitation":"[[CSL STYLE ERROR: reference with no printed form.]]","plainCitation":"[[CSL STYLE ERROR: reference with no printed form.]]"},"citationItems":[{"id":"oKuj0T6e/ndNe8FUe","uris":["http://zotero.org/users/local/XP3Mr3h8/items/HAJ22HPA <http://zotero.org/users/local/XP3Mr3h8/items/HAJ22HPA>"],"uri":["http://zotero.org/users/local/XP3Mr3h8/items/HAJ22HPA <http://zotero.org/users/local/XP3Mr3h8/items/HAJ22HPA>"],"itemData":{"id":"oKuj0T6e/ndNe8FUe","type":"article-journal","title":"Formation of Porous Anodic Oxide Film on Titanium in Phosphoric Acid Electrolyte","container-title":"Journal of Materials Engineering and Performance","page":"59-66","volume":"24","issue":"1","source":"CrossRef","DOI":"10.1007/s11665-014-1262-7","ISSN":"1059-9495, 1544-1024","language":"en","author":[{"family":"Liu","given":"Z."},{"family":"Thompson","given":"G. E."}],"issued":{"date-parts":[["2015",1]]}}}],"schema":"https://github.com/citation-style-language/schema/raw/master/csl-citation.json <https://github.com/citation-style-language/schema/raw/master/csl-citation.json>"} }

for the second one, same article but took from the whole list:

{"citationID":"aoppvg4hnb","properties":{"formattedCitation":"[@liu2015c]","plainCitation":"[@liu2015c]"},"citationItems":[{"id":1523,"uris":["http://zotero.org/users/2837208/items/7MT3ZF26 <http://zotero.org/users/2837208/items/7MT3ZF26>"],"uri":["http://zotero.org/users/2837208/items/7MT3ZF26 <http://zotero.org/users/2837208/items/7MT3ZF26>"],"itemData":{"id":1523,"type":"article-journal","title":"Formation of Porous Anodic Oxide Film on Titanium in Phosphoric Acid Electrolyte","container-title":"Journal of Materials Engineering and Performance","page":"59-66","volume":"24","issue":"1","source":"CrossRef","DOI":"10.1007/s11665-014-1262-7","ISSN":"1059-9495, 1544-1024","language":"en","author":[{"family":"Liu","given":"Z."},{"family":"Thompson","given":"G. E."}],"issued":{"date-parts":[["2015",1]]},"note":"liu2015c"}}],"schema":"https://github.com/citation-style-language/schema/raw/master/csl-citation.json <https://github.com/citation-style-language/schema/raw/master/csl-citation.json>"} } 

in the second I have the note field, my guess is that when I change style word takes all the information from this text but without any connection with zotero because the connection for some articles is broken, so it cannot know that the field “note” is not empty, for the articles still connected to zotero the note is displayed correctly.

I don’t know why some are not connected anymore, in fact if I change the name of one author it is not updated in the bibliography (even after refresh)

[retorquere]

No I think the problem is broken citation, because if I have an article (it happens just with some), that is cited in a paper and I change the citation style, let’s say from journal of applied electrochemistry to APA6, everything is fine, but if I change to betterbibtex or to the style I made to see the field “note” it give error. If I add a new citation and you know you have the list of the already cited articles in your document and the list of all the articles in zotero, if I chose the one from the already cited the citation is broken but if I chose from the whole list it is correctly displayed. Then I tried to modify for example the author and when I search for a paper to cite I see the old author in the “already cited” list, and the modified version in the “whole list”.

I'm sorry but I don't quite follow what's going on here.

Can you enable the hidden pref, export your library to CSL JSON, and attach that here?

[retorquere]

Oh wait, you say the connection is broken, as in Zotero doesn't refresh the reference data? I wouldn't know how that works, exactly, but when I start LibreOffice without Zotero running and I try to change the citation style, I get a complaint from the word processor plugin, so for LO at least it actively tries to connect to Zotero.

[zavidos]

When I try to change style if zotero is not open it say “open zotero”, and when I change style it changes, but I think it is using data that are already in my document and it want zotero to be open just to read the citation style that I want to change, not to read articles data

Il giorno 18 dic 2017, alle ore 20:20, Emiliano Heyns [e-mail] ha scritto:

Oh wait, you say the connection is broken, as in Zotero doesn't refresh the reference data? I wouldn't know how that works, exactly, but when I start LibreOffice without Zotero running and I try to change the citation style, I get a complaint from the word processor plugin, so for LO at least it actively tries to connect to Zotero.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/retorquere/zotero-better-bibtex/issues/849#issuecomment-352369278, or mute the thread https://github.com/notifications/unsubscribe-auth/Ag_VykI6173F4FxqT4iUM_RPUtegeR1Dks5tBi5HgaJpZM4REP01.

[zavidos]

Ok I don’t know what causes this problem, but I would have never discovered it without the “better bibtex citekey” csl. However the missing connection prevent me to use this system (or any similar) to have @authYear. I think I can create a style that display only the DOI of the article, and this information doesn’t require connection with zotero because is already stored in the document. So how can I substitute all the DOI with the citekey? It should be possible and easy having both on the same .bib, no?

Il giorno 18 dic 2017, alle ore 20:20, Emiliano Heyns <[email protected] [e-mail]> ha scritto:

Oh wait, you say the connection is broken, as in Zotero doesn't refresh the reference data? I wouldn't know how that works, exactly, but when I start LibreOffice without Zotero running and I try to change the citation style, I get a complaint from the word processor plugin, so for LO at least it actively tries to connect to Zotero.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/retorquere/zotero-better-bibtex/issues/849#issuecomment-352369278, or mute the thread https://github.com/notifications/unsubscribe-auth/Ag_VykI6173F4FxqT4iUM_RPUtegeR1Dks5tBi5HgaJpZM4REP01.

[retorquere]

Without a connection to Zotero? Not possible, as the citekeys live in Zotero, not in the stored references in the Word doc.

[zavidos]

I know but the DOI is the same in zotero and in word document, so if I manage to display the DOI in the word document I will have on one side a document full of DOI and on the other zotero, which have the DOI and the citekey together. So I have a file that contain the DOI and the associated citekey and I just need a way to say “search for this DOI and substitute with citekey"

Il giorno 19 dic 2017, alle ore 00:14, Emiliano Heyns [e-mail] ha scritto:

Without a connection to Zotero? Not possible, as the citekeys live in Zotero, not in the stored references in the Word doc.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/retorquere/zotero-better-bibtex/issues/849#issuecomment-352422079, or mute the thread https://github.com/notifications/unsubscribe-auth/Ag_Vyo1OWrfbU9cOAQUh_KoRm7vdIp4tks5tBmVBgaJpZM4REP01.

[retorquere]

I'm not sure why you'd want that, but sure. It's not something I could do however. The note-trick is as far as I can take it I think; if those for some reason don't make it to the word doc you'll have to ask the Zoter devs, that's out of my hands.

[retorquere]

I'm willing to help out on this issue, but you will have to take point with the Zotero devs. I don't use word, and issues need a primary stakeholder. Do you want to move this forward by opening a discussion with the Zotero devs?

[zavidos]

Hi Emiliano, sorry for the late answer but I was without internet for a while, remote region.. Anyway for that particular document I managed to do it manually, there wasn’t many broken links, for the future is just better not to work with anyone else because if someone has the same article on zotero and make a citation, you can see the citation but is not linked to that article in your zotero, so one library at time

Il giorno 30 dic 2017, alle ore 19:58, Emiliano Heyns <[email protected] [e-mail]> ha scritto:

I'm willing to help out on this issue, but you will have to take point with the Zotero devs. I don't use word, and issues need a primary stakeholder. Do you want to move this forward by opening a discussion with the Zotero devs?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/retorquere/zotero-better-bibtex/issues/849#issuecomment-354535208, or mute the thread https://github.com/notifications/unsubscribe-auth/Ag_VyksX3p0qT0g0FqUbRHbdEtJS_2u7ks5tFfslgaJpZM4REP01.

[retorquere]

I should try that sometimes, going without internet.

So are you saying this isn't worth pursuing? I'm having some difficulty understanding that later part of your message.

[zavidos]

Oh basically I discovered why some of that links were broken, it was because I worked from two different zoteros (with another person), and even having the same articles in the library, the produced citation is not linked with my library, so when you change CSL style you get a new style but if you want the citelink it is not possible, that’s why I wanted the DOI inline, so I could try to replace the DOI with citekey automatically and the DOI is not dependent on zotero library link.

So it is just dangerous to work with someone else on different zotero libraries, for that document I found the broken citations and replaced manually. All the system now works, even the substitution of old citation with citekey in word documents that I want to move to scrivener. It just needs non-broken citation link to zotero.

Is it clear? I’m not sure to have been clear enough.

p.s.
I was in western Tasmania, so more than a choice I was obliged not to use internet, but really interesting experience

Il giorno 01 gen 2018, alle ore 10:42, Emiliano Heyns [e-mail] ha scritto:

I should try that sometimes, going without internet.

So are you saying this isn't worth pursuing? I'm having some difficulty understanding that later part of your message.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/retorquere/zotero-better-bibtex/issues/849#issuecomment-354628342, or mute the thread https://github.com/notifications/unsubscribe-auth/Ag_Vyro7CkJsVvppmlM74jh4BhX3iCOqks5tGBvbgaJpZM4REP01.

[retorquere]

Ah, OK. So the option is still useful then? Then I'll merge it into a new
release.

On Jan 1, 2018 00:48, "zavidos" [e-mail] wrote:

Oh basically I discovered why some of that links were broken, it was
because I worked from two different zoteros (with another person), and even
having the same articles in the library, the produced citation is not
linked with my library, so when you change CSL style you get a new style
but if you want the citelink it is not possible, that’s why I wanted the
DOI inline, so I could try to replace the DOI with citekey automatically
and the DOI is not dependent on zotero library link.

So it is just dangerous to work with someone else on different zotero
libraries, for that document I found the broken citations and replaced
manually. All the system now works, even the substitution of old citation
with citekey in word documents that I want to move to scrivener. It just
needs non-broken citation link to zotero.

Is it clear? I’m not sure to have been clear enough.

p.s.
I was in western Tasmania, so more than a choice I was obliged not to use
internet, but really interesting experience

Il giorno 01 gen 2018, alle ore 10:42, Emiliano Heyns <
[email protected]> ha scritto:

I should try that sometimes, going without internet.

So are you saying this isn't worth pursuing? I'm having some difficulty
understanding that later part of your message.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub retorquere/zotero-better-bibtex/issues/849#issuecomment-354628342>, or
mute the thread Vyro7CkJsVvppmlM74jh4BhX3iCOqks5tGBvbgaJpZM4REP01>.

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/retorquere/zotero-better-bibtex/issues/849#issuecomment-354628496,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAIEDP-6fE8Ff9bEwc_DRrybYNzX7zrMks5tGB04gaJpZM4REP01
.

[zavidos]

Yes absolutely the option is the only easy way to move documents from word to scrivener+pandoc workflow. Yes merge it please :)

Il giorno 01 gen 2018, alle ore 11:17, Emiliano Heyns [e-mail] ha scritto:

Ah, OK. So the option is still useful then? Then I'll merge it into a new
release.

On Jan 1, 2018 00:48, "zavidos" [e-mail] wrote:

Oh basically I discovered why some of that links were broken, it was
because I worked from two different zoteros (with another person), and even
having the same articles in the library, the produced citation is not
linked with my library, so when you change CSL style you get a new style
but if you want the citelink it is not possible, that’s why I wanted the
DOI inline, so I could try to replace the DOI with citekey automatically
and the DOI is not dependent on zotero library link.

So it is just dangerous to work with someone else on different zotero
libraries, for that document I found the broken citations and replaced
manually. All the system now works, even the substitution of old citation
with citekey in word documents that I want to move to scrivener. It just
needs non-broken citation link to zotero.

Is it clear? I’m not sure to have been clear enough.

p.s.
I was in western Tasmania, so more than a choice I was obliged not to use
internet, but really interesting experience

Il giorno 01 gen 2018, alle ore 10:42, Emiliano Heyns <
[email protected]> ha scritto:

I should try that sometimes, going without internet.

So are you saying this isn't worth pursuing? I'm having some difficulty
understanding that later part of your message.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub retorquere/zotero-better-bibtex/issues/849#issuecomment-354628342>, or
mute the thread Vyro7CkJsVvppmlM74jh4BhX3iCOqks5tGBvbgaJpZM4REP01>.

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/retorquere/zotero-better-bibtex/issues/849#issuecomment-354628496,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAIEDP-6fE8Ff9bEwc_DRrybYNzX7zrMks5tGB04gaJpZM4REP01
.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/retorquere/zotero-better-bibtex/issues/849#issuecomment-354629614, or mute the thread https://github.com/notifications/unsubscribe-auth/Ag_Vykgz05Qjax2guvLyYkPCBaUeqAYLks5tGCQNgaJpZM4REP01.

[retorquere]

I'll add the docs for this later, but it will be in the next release.

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.