Zotero-better-bibtex: missing collection returns 500 error instead of 404 error on pull export

[danmackinlay]

Report ID:
N/A

Exporter used:
biblatex

Expected behavior:
I have a script that checks for bib references matching collections in my zotero account and matches them with files for my blog.
For example if there is a file called speech_transcription.md in my blog then the script will attempt to pull
http://127.0.0.1:23119/better-bibtex/collection?/1/speech_transcription.biblatex.
If, however, there is no such collection, it should return a 404 error, and I believe it did previously.

Actual behavior:
It returns a 500 error, and message content

Error: Collection '/1/speech_transcription' does not exist

I say "should" here because a) that seems logical to me that the web server should return 404 errors when content is not found, and b) AFAICT my script, which stopped working recently, previously relied on that behaviour, so I _think_ that was what BBT previously did.
However, my script is a hack job of a script so I am not 100% certain of (b), I'm sure @retorquere will know better than I.

However, even if I am wrong about (b) can I request that BBT returns something other than a 500 error if a collection is not found whe doing a pull export? The reason miscellaneous (Zotero? BBT?) failures are fairly common even if a collection exists during a pull export, and those also return a 500 error.
I could distinguish between the different failure modes by parsing the response body, but this looks fragile. Will BBT always return exactly Error: Collection '(.*)' does not exist?
If there is no reliable way to distinguish a pull export URL pointing to absent content from a BBT/Zotero networking error, then I do not know how to detect when I have renamed or deleted a collection in Zotero.

[label-gun[bot]]

It looks like you did not upload an debug report. The debug report is important; it gives @retorquere your current BBT settings and a copy of the problematic reference as a test case so he can best replicate your problem. Without it, @retorquere is effectively blind. Debug reports are useful for both bug analysis and enhancement requests; in the case of export enhancements, I need the copy of the references you have in mind.

This request is much more likely than not to apply to you, too, _even if you think it unlikely_, and even if it does not, there's no harm in sending a debug log that turns out to be unnecessary. @retorquere will more often than not just end up saying "please send a debug log first". Let's just skip over the unnecesary delay this entails. Sending a debug log is very easy:

1. If your issue relates to how BBT behaves around a specific reference(s), such as citekey generation or export, select at least one of the problematic reference(s), right-click it, and submit an BBT debug report from that popup menu. If the problem is with export, please do include a sample of what you see exported, and what you expected to see exported for these references.

2. If the issue does not relate to references and is of a more general nature, generate an debug report by restarting Zotero with debugging enabled (Help -> Debug Output Logging -> Restart with logging enabled), reproducing your problem, and selecting "Send Better BibTeX debug report..." from the help menu.

Once done, you will see a debug ID in red. Please post that debug id in the issue here.

Thank you!

[danmackinlay]

OKOK, here is a debug ID D1853377946 showing me starting up Zotero in Debug mode then running

curl 'http://127.0.0.1:23119/better-bibtex/collection?/1/speech_transcription.biblatex' 

It is a very long debug log. I think the important part is likely just this bit

(5)(+0377859): GET /better-bibtex/collection?/1/speech_transcription.biblatex HTTP/1.1 Host: 127.0.0.1:23119 User-Agent: curl/7.72.0 Accept: */*

(5)(+0000054): HTTP/1.0 500 Internal Server Error X-Zotero-Version: 5.0.91 X-Zotero-Connector-API-Version: 2 Content-Type: text/plain Error: Collection '/1/speech_transcription' does not exist

[retorquere]

D1853377946 is a Zotero debug ID. I have no access to those.

[danmackinlay]

Ah, sorry I misread label-bot's instructions. 9SQYJBMK-apse is the BBT report number

[danmackinlay]

Maybe even a BBT report showing a non-missing response and a missing response?

8VES7FX2-apse is such a report number.

To produce this one, I executed the following commands

$ curl 'http://127.0.0.1:23119/better-bibtex/collection?/1/speech_transcription.biblatex'                                                                     
Error: Collection '/1/speech_transcription' does not exist⏎                                                                                                             

$ curl 'http://127.0.0.1:23119/better-bibtex/collection?/1/stochastic_taylor_expansion.biblatex'                                                             
@incollection{AitSahaliaOperator2010,
  title = {Operator {{Methods}} for {{Continuous}}-{{Time Markov Processes}}},
  booktitle = {Handbook of {{Financial Econometrics}}: {{Tools}} and {{Techniques}}},
  author = {Aït-Sahalia, Yacine and Hansen, Lars Peter and Scheinkman, José A.},
  date = {2010},
  pages = {1--66},
  publisher = {{Elsevier}},
  doi = {10.1016/B978-0-444-50897-3.50004-3},
  url = {https://www.princeton.edu/~yacine/AHS_operatormethods_handbook_10.pdf},
  urldate = {2020-01-27},
  isbn = {978-0-444-50897-3},
  langid = {english}
}

@article{AriffinStochastic2017,
  title = {Stochastic {{Taylor}} Expansion of Derivative-Free Method for Stochastic Differential Equations},
  author = {Ariffin, Noor Amalina Nisa and Rosli, Norhayati},
  date = {2017-09-28},
  journaltitle = {Malaysian Journal of Fundamental and Applied Sciences},
  volume = {13},
  doi = {10.11113/mjfas.v13n3.633},
  url = {https://mjfas.utm.my/index.php/mjfas/article/view/633},
  urldate = {2020-01-24},
  langid = {english},
  number = {3}
}

@incollection{JacobLevyType2001,
  title = {Lévy-{{Type Processes}} and {{Pseudodifferential Operators}}},
  booktitle = {Lévy {{Processes}}: {{Theory}} and {{Applications}}},
  author = {Jacob, Niels and Schilling, René L.},
  editor = {Barndorff-Nielsen, Ole E. and Resnick, Sidney I. and Mikosch, Thomas},
  date = {2001},
  pages = {139--168},
  publisher = {{Birkhäuser}},
  location = {{Boston, MA}},
  doi = {10.1007/978-1-4612-0197-7_7},
  url = {https://doi.org/10.1007/978-1-4612-0197-7_7},
  urldate = {2020-02-04},
  isbn = {978-1-4612-0197-7},
  langid = {english}
}

@article{KloedenApproximation1992,
  title = {The Approximation of Multiple Stochastic Integrals},
  author = {Kloeden, P. E. and Platen, E. and Wright, I. W.},
  date = {1992-01-01},
  journaltitle = {Stochastic Analysis and Applications},
  volume = {10},
  pages = {431--441},
  doi = {10.1080/07362999208809281},
  url = {https://doi.org/10.1080/07362999208809281},
  urldate = {2020-01-24},
  number = {4}
}

@book{KloedenNumerical2010,
  title = {Numerical {{Solution}} of {{Stochastic Differential Equations}}},
  author = {Kloeden, Peter E. and Platen, Eckhard},
  date = {2010-12-15},
  publisher = {{Springer Berlin Heidelberg}},
  location = {{Berlin, Heidelberg}},
  url = {https://public.ebookcentral.proquest.com/choice/publicfullrecord.aspx?p=3099793},
  isbn = {978-3-642-08107-1},
  langid = {english}
}

@incollection{KloedenStochastic1992,
  title = {Stochastic {{Taylor Expansions}}},
  booktitle = {Numerical {{Solution}} of {{Stochastic Differential Equations}}},
  author = {Kloeden, Peter E. and Platen, Eckhard},
  editor = {Kloeden, Peter E. and Platen, Eckhard},
  date = {1992},
  pages = {161--226},
  publisher = {{Springer}},
  location = {{Berlin, Heidelberg}},
  doi = {10.1007/978-3-662-12616-5_5},
  url = {https://doi.org/10.1007/978-3-662-12616-5_5},
  urldate = {2020-01-24},
  isbn = {978-3-662-12616-5},
  langid = {english},
  series = {Applications of {{Mathematics}}}
}

@article{KloedenStratonovich1991,
  title = {Stratonovich and {{Ito Stochastic Taylor Expansions}}},
  author = {Kloeden, P. E. and Platen, E.},
  date = {1991},
  journaltitle = {Mathematische Nachrichten},
  volume = {151},
  pages = {33--50},
  doi = {10.1002/mana.19911510103},
  langid = {english},
  number = {1}
}

@online{PapapantoleonStrong2010,
  title = {Strong {{Taylor}} Approximation of Stochastic Differential Equations and Application to the {{L}}\textbackslash 'evy {{LIBOR}} Model},
  author = {Papapantoleon, Antonis and Siopacha, Maria},
  date = {2010-10-04},
  url = {http://arxiv.org/abs/0906.5581},
  urldate = {2020-05-19},
  archivePrefix = {arXiv},
  eprint = {0906.5581},
  eprinttype = {arxiv},
  primaryClass = {math, q-fin}
}

@article{RosslerStochastic2004,
  title = {Stochastic {{Taylor Expansions}} for the {{Expectation}} of {{Functionals}} of {{Diffusion Processes}}},
  author = {Rößler, Andreas},
  date = {2004-01-01},
  journaltitle = {Stochastic Analysis and Applications},
  volume = {22},
  pages = {1553--1576},
  doi = {10.1081/SAP-200029495},
  url = {http://arxiv.org/abs/1310.6169},
  urldate = {2020-01-24},
  archivePrefix = {arXiv},
  eprint = {1310.6169},
  eprinttype = {arxiv},
  number = {6}
}

@article{SchoutensStochastic2001,
  title = {Stochastic {{Taylor Expansions}} for {{Poisson Processes}} and {{Applications}} towards {{Risk Management}}},
  author = {Schoutens, Wim and Leuven, K U and Studer, Michael},
  date = {2001-02-16},
  pages = {24},
  url = {https://www.eurandom.tue.nl/reports/2001/005-report.pdf},
  langid = {english}
}

Is that clarifying at all?

[blip-bloop]

:robot: this is your friendly neighborhood build bot announcing test build 5.2.82.7286 ("more finegrained errors")

Install in Zotero by downloading test build 5.2.82.7286, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".

[danmackinlay]

Incredible. With this new build I have references back I didn't even know I was missing! Thank you @retorquere

[retorquere]

Only I can close issues 😄 . I keep them around as reminders to merge & release; that's now running.

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.