Zotero-better-bibtex: zotero://open-pdf/ by citekey

[somelinguist]

I just discovered that you can select items in Zotero using the URL scheme zotero://select/items/@[citekey]. This is very useful.

Would it also be possible to implement a URL that would open an attached PDF using the citekey?

The use case for this would be to allow directly opening the attached PDF from another program given the citation information (citekey and page number).

It looks like the current implementations of zotero://open-pdf/ require the item id, which isn't usually available in .bib/.json files exported by BBT. Being able to do this by citekey would be more useful from other programs than the internal Zotero item id.

[retorquere]

That's not possible, sorry. I don't actually hook the URL handler, I hook the lookup that gets the ID and have it look for the citekey if the "id" starts with @, but I don't know if the lookup is being called for zotero://select/items/ or zotero://open-pdf/ or one of the many other places it's being called. And for open-pdf, I'd have to have the lookup return "the first pdf attachment on the item" rather than "the item" without any context to decide between the two.

What should in principle be possible is to use another prefix character than @ to mean "first pdf attachment". Feel free to suggest one.

[retorquere]

Also I don't know if the open-pdf handler knows how to open a specific page. If it doesn't, I can't add it, that code is not accessible to plugins.

[somelinguist]

Ok. Thanks for explaining. I think I understand better now. So if another prefix-character was used, you could return that item id for the PDF attachment rather than the item id itself, which in theory would work with zotero://open-pdf/. Is that correct?

If so, I'm not sure what prefix would be best. I saw that you can also use bbt:citation instead of @citation for the current functionality, is that correct? Something like bbtpdf: would be fine with me then. @ is nice and short, but I'm not sure how many other plugins do something like this that might clash. Otherwise I'd say something like #. What do you think?

All of the zotero://open-pdf/ schemes can handle page numbers, so if the proper ID can be sent, then Zotero itself should take care of the rest:

zotero://open-pdf/library/items/[itemKey]?page=[page]
zotero://open-pdf/groups/[groupID]/items/[itemKey]?page=[page]
zotero://open-pdf/[libraryID]_[key]/[page]

[retorquere]

Yes, that's correct. # is a reserved character in urls, so that won't work. The prefix would work.

Since I'm only adding to the key recognition, anything that is supported outside that would still work.

[somelinguist]

I forgot about reserved characters in URLs. :) The longer form would be more than adequate.

[retorquere]

Sorry for the delay. It's not possible to do this reliably, sorry. The function I am patching is synchronous (meaning I can only call other synchronous functions), but in order to get the attachments of an item, I must load it, and that cannot be done reliably with synchronous methods.

[retorquere]

Except if you're OK with the URL being different. If an URL of the type http://127.0.0.1:23119/better-bibtex/get-pdf?citekey=... would be OK, that can be done.

[somelinguist]

Hi. Thanks for the explanation. No problems with the delay. A different URL would be fine. Would it still be able to open the attachment using the PDF reader determined by Zotero? That was the main functionality of using the Zotero URL.

Thanks.

[retorquere]

I haven't checked. I'll see if I can get to that in the weekend.

[retorquere]

Would the caller be able to follow redirects? I'd rather not reimplement the open-pdf logic, and it'd be doable to just have http://127.0.0.1:23119/better-bibtex/open-pdf?citekey=... redirect to the appropriate zotero://open-pdf/.. url. At least I think this would be possible; the Zotero webserver has a lot of limitations, and ISTR I can't set the requisite headers. What would be certainly doable is to have http://127.0.0.1:23119/better-bibtex/open-pdf?citekey=... return the open-pdf as the response body.

[somelinguist]

Sorry for taking so long to get back with you. Yes, it wouldn't be worth it to duplicate the open-pdf logic. Either redirects or the url in the body of the response seem fine to me. I'm not sure what @nathanlesage thinks about an implementation in Zettlr? It's my understanding that Electron can handle redirects by default with ClientRequest. If a redirect isn't possbile from BBT's end, it wouldn't be too much more difficult to follow a link given in the request body. Either way, it would probably be helpful for the response also to indicate whether any attachments exist.

[nathanlesage]

Hello there!

Quite some discussion that has evolved here :D I really did not anticipate that it would be so hard to implement an x-callback interface for that, but thank you for keeping up the discussion, there's a lot to be learned by reading your comments.

However, maybe to give some more context: The whole discussion is here because we are currently discussing to enable some form of "back-linking" from within Zettlr, which can display citations using CiteKeys. That means: Users export their library preferrably using BBT to a CSL JSON format, and we only have the CiteKey available. The idea was to implement a context-menu entry "Open PDF" that pings Zotero and has it open the PDF.

If I understand the sources I found via Googling, I think the open-pdf-URI is defined within Zotero itself, meaning that BBT also just hooks into callbacks, so whenever a "open-pdf"-command is pinged towards Zotero, the only thing Zotero does is call a Callback function that receives the passed arguments, and can do something with them, right? If so, I'm asking whether BBT is the right place to go for…? Does Zotero itself have access to CiteKeys? If so, wouldn't it make more sense to open an issue on their GitHub repo and ask for implementation…? (That is: "If there's an @ in front of the ID, try the search key instead of the Zotero IDs", which, as far as I see it, don't contain @-chars at all) I mean, as of now, Zotero simply does nothing when a URI handler is unknown or yields no results, so it would be compatible even to users who do not have BBT installed (Zotero would simply not open the respective item).

What do you think?

[retorquere]

@somelinguist wrote:

Sorry for taking so long to get back with you.

No problem

Yes, it wouldn't be worth it to duplicate the open-pdf logic. Either redirects or the url in the body of the response seem fine to me.

I've checked, and I can't send a proper HTTP redirect, as I can only affect the body, mimetype, and response code, but not headers.

I'm not sure what @nathanlesage thinks about an implementation in Zettlr? It's my understanding that Electron can handle redirects by default with ClientRequest. If a redirect isn't possbile from BBT's end, it wouldn't be too much more difficult to follow a link given in the request body. Either way, it would probably be helpful for the response also to indicate whether any attachments exist.

In what way? I could return a JSON object with pretty much anything.

@nathanlesage wrote:

If I understand the sources I found via Googling, I think the open-pdf-URI is defined within Zotero itself,

That is correct

meaning that BBT also just hooks into callbacks, so whenever a "open-pdf"-command is pinged towards Zotero, the only thing Zotero does is call a Callback function that receives the passed arguments, and can do something with them, right?

That is also correct. As Zotero doesn't have a concept of citekeys however, BBT has so-called "monkey-patches" in place to bring the concept into Zotero; I runtime-patch Zotero functions to alter their behavior where a formal API to do the same doesn't exist.

There are some limitations to what a monkey-patch can do; I can wrap an entire function, possibly modify the parameters, and then decide whether to call the original function or to handle the behavior of the original function entirely.

Most importantly, I can't change parts inside functions, and I cannot access inner variables/functions of exposed functions. In the case of the url-handler patching I do, Zotero's URL handler calls a function I can wrap which parses the URL parameters into an internal ID. In my wrapper, I look whether the parameter starts with @; if it does, I look in BBTs database and give the response as if the original was handed the corresponding <library>_<itemKey>; if it does not start with @, I pass control to the wrapped (original) function without changes. So I don't actually change anything about Zotero's URL handler, I just change the behavior of one of the functions it calls. I don't know for what purpose this now-wrapped function is called -- AAMOF, there's a few other places it's called, and there it would likewise respond to @citekey if that information would be passed there.

If so, I'm asking whether BBT is the right place to go for…? Does Zotero itself have access to CiteKeys?

Nope, not currently. Zotero doesn't have a concept of citekeys; citekeys live in a private BBT database, and Zotero knows nothing about its existence (I mean it could, but just doesn't). At some point in the not too far but not clearly near future, Zotero will in fact add citekeys to all items (it's been discussed for a long time, and it is actually planned), and then I'll re-figure out how much of this is necessary. But for the foreseeable future, if you want inspectable, stable citekeys, BBT is the only game in town for Zotero.

If so, wouldn't it make more sense to open an issue on their GitHub repo and ask for implementation…? (That is: "If there's an @ in front of the ID, try the search key instead of the Zotero IDs", which, as far as I see it, don't contain @-chars at all) I mean, as of now, Zotero simply does nothing when a URI handler is unknown or yields no results, so it would be compatible even to users who do not have BBT installed (Zotero would simply not open the respective item).

It totally would, as soon as Zotero gets around to implementing the citekey field. It's been discussed for years; a lot of the architecture changes that are needed to do it are in place it seems, but there's no concrete ETA. When it drops, I'll just migrate my keys there, and drop the BBT-private DB.

[nathanlesage]

Ah, alright. So until then a fix is needed. Mh, you said you could return a JSON object with pretty much anything via a normal HTTP call, correct? I mean, a good small addition could be that I query BBT for this object, look for a PDF and issue a simple "Open"-command to the underlying operating system with its file path. This way I'd again take back some functionality to within Zettlr, but it's just about one additional command I'd need to implement, which shouldn't be too problematic. All the necessary things to implement that on my side are already in the codebase.

[retorquere]

That would be possible, but @somelinguist did say he wanted to open using the pdf opener configured in Zotero (which is what open-pdf does). I could return the appropriate zotero:// url. Or we could try to convince zotero to make a change so that if you call open-pdf on a non-attachment, it opens the first pdf attachment of it. Then no further changes would be required on the bbt side.

[blip-bloop]

:robot: this is your friendly neighborhood build bot announcing test build 5.1.176.5509 ("return attachment paths")

Install in Zotero by downloading test build 5.1.176.5509, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".

[retorquere]

5509 will return both the open-pdf link and the full path to the attachment:

$ curl http://localhost:23119/better-bibtex/json-rpc -X POST -H "Content-Type: application/json" -H "Accept: application/json" --data-binary '{"jsonrpc": "2.0", "method": "item.attachments", "params": ["bentley_academic_2011"] }'
{"jsonrpc":"2.0","result":[{"open":"zotero://open-pdf/library/items/EXUS7TAI","path":"/Users/emile/.BBTZ5TEST/zotero/storage/EXUS7TAI/Shelah - 1972 - A combinatorial problem; stability and order for m.pdf"}],"id":null}

[nathanlesage]

That was fast! I'll give it a try soon!

Just one question, as this does not make use of the x-callbacks by Zotero: is the domain http://localhost:23119 the same everywhere or may it be that the port changes ...? I remember that I've seen that port number again, so just for additional failsafes!

[retorquere]

I'm not sure what x-callbacks refers to (I've not used it before), but http://localhost:23119 is stable for Zotero. For Juris-M I think it's http://localhost:24119; you can find the value (which the user could hypothetically have changed, but I'd bet that would break just about everything) in the hidden preferences as httpServer.port.

Mind that this "server" runs inside the Zotero client and can only and exclusively be accessed from the same computer that the Zotero client runs on.

edit: 😄 three days is ridiculously slow by my standards. I was about to apologize for the delay when I saw your comment.

[nathanlesage]

I'm not sure what x-callbacks refers to

Pretty cool thing -- they are responsible, for instance, that when you click on a Twitter link on your mobile phone the Twitter app opens instead of Safari/Chrome! Each program can register such a thing and then other people can simply "open" such a URI without having to actually open a HTTP connection to the receiver. Basically a super-simple REST implementation :)

Mind that this "server" runs inside the Zotero client and can only and exclusively be accessed from the same computer that the Zotero client runs on.

Obviously, yes!

edit: three days is ridiculously slow by my standards. I was about to apologize for the delay when I saw your comment.

Oh, I feel you -- I haven't responded to a lot of issues in weeks now because I've got other deadlines to attend to and I already feel bad as hell for that :smile:

[retorquere]

Ah I see - I'm not sure zotero uses this.

[nathanlesage]

It does! Basically, the web server is only needed for systems that don't support such URIs, on macOS, any zotero://-link will automatically be passed to Zotero :)

Edit: I think you're also using macOS, so just give it a try by copying the URI to the web browser, it'll open it, bypassing the webserver

[retorquere]

No I get that, and it works, I just don't think they use x-callbacks for that. They install a protocol handler and that makes sure you're directed to Zotero -- think about it, if you paste a zotero:// URL in the browser and that works, it can't be an x-callbacks handler, because there wasn't a web server to hand it out.

http://127.0.0.1:23119/ is an internal server that is used by the browser plugins (and I think by either the Word or LibreOffice plugins) to communicate with Zotero. I've just repurposed that server for a bunch of my own stuff. I don't think before BBT started using it many people ever saw that URL.

[retorquere]

I'm prepping for a new release. I'd love for 5509 to be in it, but changes are stacking up, and it's getting harder to merge stuff; I made some significant changes in the release I'm prepping.

[nathanlesage]

Oh yes, apologies, protocol handler is the right term. x-callbacks are basically the same, but I never got the distinction (because on Windows these protocol handlers also work).

And please take your time with the new release, I'm already grateful for your efforts! :)

[retorquere]

It's more that the release is more or less ready; if the output from 5509 is what you want, I will include it in the new release.

[nathanlesage]

Aaaaah okay, sorry, I misunderstood you there. I'll have a test run, hopefully this evening. I'll get back to you asap!

[retorquere]

Thanks. Pretty keen to get this release out the door because I've made some pretty fantastic performance improvements if I do say so myself.

[retorquere]

So...

[nathanlesage]

Didn't manage yesterday, but I'll give it a shot today!

[nathanlesage]

Just was able to test it; it works perfectly! Actually I found out, as you also return the fully qualified path, I'm currently opting for filtering out the paths, sorting them with PDFs in front, and then simply open the first attachment. But the approach is good because other applications have thus access to the full list of available attachments!

[retorquere]

Alright, then I'll merge this.

[somelinguist]

Thanks for this!

[retorquere]

You're welcome

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.