Zotero-better-bibtex: Store Zotero Links in bibtex URL field

[notuntoward]

Often, Zotero downloads a link attachment, which points to the URL where I got the paper. It looks like this:

and if I click on it, I'm sent to web page where I got the paper.

It would be nice if this URL -- or any other relevant URL in the Zotero DB -- was stored in the exported .bib file. In this example, there is a DOI, but since I haveBBT Preferences --> Export --> Add URLs to BibTeX export set to "in the'url' field, I thought the URL would be stored.

What is stored is this:

@article{Jaramillo20evIdentLoadMon,
  title = {Supervised {{Non}}-{{Intrusive Load Monitoring Algorithm}} for {{Electric Vehicle Identification}}},
  author = {Jaramillo, A. M. and Laverty, D. and Rinc{\'o}n, J. and Hastings, John and Morrow, D. J.},
  year = {2020},
  doi = {10.1109/I2MTC43012.2020.9128529},
  abstract = {Transport sector electrification represents an increase in the number of electric vehicles (EV), producing significant variations in the distribution network dynamics. As a result, bidirectional power flow, overload and load unbalances are caused at the low voltage level due to unexpected increased load peaks. Non-intrusive load monitoring (NILM) methods have been developed as a strategy for energy management systems, applied to the customer side producing energy savings. This research presents a NILM methodology based on a low complexity conventional supervised machine learning pipeline. Our approach uses Principal Component Analysis (PCA) and Random Forest (RF) to detect the presence of a charging electric vehicle on the electricity network. By processing low sampling rate active power data, this approach provides a simple but feasible method that can be applied to smart meters. This provides useful data analysis for distribution network operators (DNO) to effectively deal with variability caused by these low carbon loads in the distribution grid. Achieving an overall efficacy of 92.68\%, the proposed method can be compared with other state of the art methods developed under higher complexity techniques.},
  file = {papers\\Jaramillo20evIdentLoadMon.pdf},
  journal = {2020 IEEE International Instrumentation and Measurement Technology Conference (I2MTC)}
}

My Report ID is: 98IBZ925-euc

[retorquere]

I need a debug log from right-clicking the item under consideration, not from the help menu

[notuntoward]

It's at the end of the intial bug report (My Report ID is: 98IBZ925-euc) or do you need some other thing?

[retorquere]

That was sent from the Help menu. I need one from right-clicking the item, and selecting "send log" from that popup menu.

[notuntoward]

Whoops. When I right-click the item, and selecting "send log" from that popup menu, I get another popup that starts with

The data below is about to be submitted...

and has Backand Nextbuttons on the bottom. But they are grayed out, as if they are inactivated. Only the Cancelbutton is not grayed out.

If I click Next, nothing happens.

[retorquere]

Can you send me a help-menu debug log after that happens?

[notuntoward]

Sure. This was from Zotero->Help->Send Better Bibtex Debug Report

F4BJSY9Y-euc

[retorquere]

I have a fix for that but unfortunately the github release pipeline has stalled so I can't get it to you. I've notified GH support, they're usually fairly quick on these things.

[retorquere]

They're on it.

[github-actions[bot]]

:robot: this is your friendly neighborhood build bot announcing test build 5.2.128.540 ("in-mem translation is a thing")

Install in Zotero by downloading test build 5.2.128.540, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".

[retorquere]

With build 540 you chould be able to send a right-click debug ID again

[notuntoward]

Thanks. I right clicked on the Jaramillo entry above, and sent the following bug report

LKYWXTLP-euc

[retorquere]

Right, so this article has an URL field -- why not have the URL there rather than a link-attachment?

[notuntoward]

Exactly.

[retorquere]

No, I mean -- why not fill in the URL in the URL field in Zotero rather than having it as an attached link in Zotero. The URL field has a fairly stable meaning in Zotero, which is why I translate it as I do. Attachments don't carry the same meaning.

[notuntoward]

Zotero itself makes these link attachments, either when it finds metadata
from a web page, or when it gets a new PDF and uses that to find the
metadata -- I'm not sure exactly what triggers it to do that.

In any case, yes I could search for instances where it does this, and
manually copy the URL and paste it to the Zotero URL field, but ...

On Tue, Mar 16, 2021 at 1:27 PM Emiliano Heyns @.*>
wrote:

No, I mean -- why not fill in the URL in the URL field in Zotero rather
than having it as an attached link in Zotero. The URL field has a
fairly stable meaning in Zotero, which is why I translate it as I do.
Attachments don't carry the same meaning.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/retorquere/zotero-better-bibtex/issues/1765#issuecomment-800581080,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA5MM44SR6Q45EJCYIDTODLTD65LTANCNFSM4ZG34ZRQ
.

[retorquere]

I understand that this is an inconvenience, but I'd like to first see if we can get that addressed in Zotero. Maybe there is a reason for the choice to add it as an attachment rather than in the field, and I'd like to know that reason before I interpret in a certain way.

[tea4or2wo]

Hi,

I run into the same problem with items coming from SUDOC (French national academic catalog), such as https://www.sudoc.fr/242479162

I assume it has to do with Zotero and not BBT but I also provide a debug ID: 9PRYF5CE-refs-euc

Do we have some idea about why URL is not stored in URL field by Zotero on some websites?

The thing is that I have more than 2000 references coming from SUDOC in my Zotero library and all of them are URI orphans in my .bib file. I'm reporting for a potential _ex post_ fix for these items.

If you think I should refer to the Zotero team, I will post on the Zotero forum that I have just checked about problems with SUDOC.

On a minor note, a similar problem happens with IMDb: Zotero breaks the URI, storing the last part in Extra field and leaving URL field empty.

[retorquere]

@tea4or2wo your sample represents another problem -- you have two "linked URL" attachments so even if I would use the linked url in lieu of the item url, I wouldn't know which to pick.

[retorquere]

On a minor note, a similar problem happens with IMDb: Zotero breaks the URI, storing the last part in Extra field and leaving URL field empty.

That's something to be fixed in Zotero.

[retorquere]

The thing is that I have more than 2000 references coming from SUDOC in my Zotero library and all of them are URI orphans in my .bib file. I'm reporting for a potential _ex post_ fix for these items.

Can you right-click the library that has those items and send a support log for that? I can create a javascript runner script to fix those in bulk.

[retorquere]

It's also possible to do this in a BBT postscript, which would leave the items in zotero untouched, but it seems to me you would rather have the url actually stored in the item.

[retorquere]

This should help:

let fixes = ``

if (!fixes) {
  const csv = []
  for (const item of await Zotero.Items.getAll(Zotero.Libraries.userLibraryID, true)) {
    if (item.getField('url') || item.isNote() || item.isAttachment()) continue
    const urls = item.getAttachments()
      .map(itemID => Zotero.Items.get(itemID))
      .filter(att => att.attachmentLinkMode === Zotero.Attachments.LINK_MODE_LINKED_URL)
      .map(att => att.getField('url'))
    if (urls.length) {
      csv.push([item.itemID, item.getField('title'), ''].concat(urls).join('\t'))
    }
  }
  return csv.join('\n')
}
else {
  const urlFieldID = Zotero.ItemFields.getID('url')
  for (let fix of fixes.split(/\r?\n/)) {
    fix = fix.trim()
    if (fix) {
      fix = fix.split('\t')
      if (fix[2]) {
        const item = await Zotero.Items.getAsync(parseInt(fix[0]))
        if (Zotero.ItemFields.isValidForType(urlFieldID, item.itemTypeID)) {
          item.setField('url', fix[2])
          await item.saveTx()
        }
      }
    }
  }
}

when you run this through Tools -> Developer -> Run Javascript, you can copy the output and paste it into Excel. You now have rows with the following data in position:

0: itemID. do not change this
1: title. This is only informative so you can easier tell which item it concerns
2: empty
3+: existing url attachments

for each of the items you want to change, copy the desired URL to the empty cell (position 2); once ready, copy the data in excel, and paste it between the back-quotes in the first line (the line with fixes), and run the script again. This will update all items where you have copied the URL to the formerly empty cell.

[tea4or2wo]

Thank you for taking time working on my problem.

Your code works like a charm, I got clean URLs back into my .bib file for +2500 refs.

I'm preparing a broader post for the Zotero forum, I'll post a pointer here as soon as it gets on tracks.

The missing URL depends on the Zotero translator used on the webpage.

For SUDOC, Zotero uses the PICA library catalog translator and there is a MR for adding links to the URL field here.