Latex-workshop: latexmk fails with a bibtex error "Not writing to .blg (openout_any = p)"

[emersonmello]

Preliminary questions [Required]

Disable all the other extensions except for LaTeX Workshop, and check that you still see this issue. [Required]

You still see this issue?: Yes

Make sure to visit the wiki FAQ before filling an issue.

You visited the wiki?: Yes

And I found this issue here

If your issue is with compiling a document (not having to do with finding the root file of a project), check first that you can compile manually.

You can compile a TeX document manually?: Yes, using the line below.

latexmk -pdf -synctex=1 -interaction=nonstopmode -file-line-error  main.tex 

Describe the bug [Required]

I can't compile a simple .tex + .bib using latexmk recipe and LaTeX Workshop default settings. But, I'm able to compile using
latexmk on terminal and I'm able to compile using pdflatex -> bibtex -> pdflatex x2 recipe.

To Reproduce

Steps to reproduce the behavior:

1. Use Latex Workshop default settings and recipes
2. Create a minimal .tex file

\documentclass[a4paper,12pt]{article}
\usepackage{lipsum}

\begin{document}
\section{Introduction}
\lipsum[1-3]

\nocite{*}
\bibliographystyle{plain}
\bibliography{references.bib}

\end{document}

1. Create a minimal .bib file

@book{goossens94,
  title={{The LATEX companion}},
  author={Goossens, M. and Mittelbach, F. and Samarin, A.},
  year={1994},
  publisher={Addison-Wesley}
}

@book{lamport94,
  title={{La T E X: A Document Preparation System}},
  author={Lamport, L. and others},
  year={1994},
  publisher={Addison-Wesley, Reading, Massachusetts,}
}

1. Click on Recipe: latexmk
2. See error dialog pop-up

Expected behavior

A pdf file with the references.

Logs [Required]

Please paste the whole log messages here, not parts of ones. The log should start with Initializing LaTeX Workshop. It is very important to identify problems.

LaTeX Workshop Output [Required]

[19:19:19] Initializing LaTeX Workshop.
[19:19:19] Extension root: /Users/user/.vscode/extensions/james-yu.latex-workshop-8.13.2
[19:19:19] Creating PDF file watcher.
[19:19:19] Creating LaTeX Workshop http and websocket server.
[19:19:19] LaTeX Workshop initialized.
[19:19:19] Current workspace folders: undefined
[19:19:19] Current workspaceRootDir: 
[19:19:19] Found root file from active editor: /Users/user/Downloads/t/main.tex
[19:19:19] Root file changed: from undefined to /Users/user/Downloads/t/main.tex
[19:19:19] Start to find all dependencies.
[19:19:19] Instantiating a new file watcher for /Users/user/Downloads/t/main.tex
[19:19:19] Creating Bib file watcher.
[19:19:19] Parsing a file and its subfiles: /Users/user/Downloads/t/main.tex
[19:19:19] Found .bib file: /Users/user/Downloads/t/references.bib
[19:19:19] Added to bib file watcher: /Users/user/Downloads/t/references.bib
[19:19:19] Parsing .bib entries from /Users/user/Downloads/t/references.bib
[19:19:19] Parse fls file.
[19:19:19] Cannot find fls file: /Users/user/Downloads/t/main.fls
[19:19:19] Server created on 127.0.0.1:50461
[19:19:19] Added to file watcher: /Users/user/Downloads/t/main.tex
[19:19:19] Snippet data loaded.
[19:19:19] LaTeX Workshop version: 8.13.2
[19:19:20] Parsed 2 bib entries from /Users/user/Downloads/t/references.bib.
[19:19:23] LOG command invoked.
[19:19:23] LOG command invoked.
[19:19:23] Current workspace folders: undefined
[19:19:23] Current workspaceRootDir: 
[19:19:23] Found root file from active editor: /Users/user/Downloads/t/main.tex
[19:19:23] Keep using the same root file: /Users/user/Downloads/t/main.tex.
[19:19:26] Current workspace folders: undefined
[19:19:26] Current workspaceRootDir: 
[19:19:26] Found root file from active editor: /Users/user/Downloads/t/main.tex
[19:19:26] Keep using the same root file: /Users/user/Downloads/t/main.tex.
[19:19:32] RECIPES command invoked.
[19:19:32] BUILD command invoked.
[19:19:32] Current workspace folders: undefined
[19:19:32] Current workspaceRootDir: 
[19:19:32] Found root file from active editor: /Users/user/Downloads/t/main.tex
[19:19:32] Keep using the same root file: /Users/user/Downloads/t/main.tex.
[19:19:32] Building root file: /Users/user/Downloads/t/main.tex
[19:19:32] Build root file /Users/user/Downloads/t/main.tex
[19:19:32] Recipe step 1: latexmk, -synctex=1,-interaction=nonstopmode,-file-line-error,-pdf,-outdir=/Users/user/Downloads/t,/Users/user/Downloads/t/main
[19:19:32] cwd: /Users/user/Downloads/t
[19:19:32] LaTeX build process spawned. PID: 33165.
[19:19:34] LaTeX log parsed with 0 messages.
[19:19:34] Recipe returns with error: 12/null. PID: 33165. message: Rc files read:
  NONE
Latexmk: This is Latexmk, John Collins, 13 September 2020, version: 4.70a.
Rule 'pdflatex': The following rules & subrules became out-of-date:
      'pdflatex'
------------
Run number 1 of rule 'pdflatex'
------------
------------
Running 'pdflatex  -synctex=1 -interaction=nonstopmode -file-line-error -recorder -output-directory="/Users/user/Downloads/t"  "/Users/user/Downloads/t/main.tex"'
------------
Latexmk: Non-existent bbl file '/Users/user/Downloads/t/main.bbl'
 No file main.bbl.
Latexmk: Log file says output to 'main.pdf'
Latexmk: Found bibliography file(s) [references.bib]
Rule 'bibtex /Users/user/Downloads/t/main': File changes, etc:
   Changed files, or newly in use since previous run(s):
      '/Users/user/Downloads/t/main.aux'
------------
Run number 1 of rule 'bibtex /Users/user/Downloads/t/main'
------------
------------
Running 'bibtex  "/Users/user/Downloads/t/main.aux"'
------------

bibtex: Not writing to /Users/user/Downloads/t/main.blg (openout_any = p).
Collected error summary (may duplicate other messages):
  bibtex /Users/user/Downloads/t/main: Could not open bibtex log file for '/Users/user/Downloads/t/main'
Latexmk: Use the -f option to force complete processing,
 unless error was exceeding maximum runs, or warnings treated as errors.
.
[19:19:34] Cleaning auxiliary files and retrying build after toolchain error.
[19:19:34] File cleaned: /Users/user/Downloads/t/main.aux
[19:19:34] File cleaned: /Users/user/Downloads/t/main.fls
[19:19:34] File cleaned: /Users/user/Downloads/t/main.log
[19:19:34] File cleaned: /Users/user/Downloads/t/main.fdb_latexmk
[19:19:34] Recipe step 1: latexmk, -synctex=1,-interaction=nonstopmode,-file-line-error,-pdf,-outdir=/Users/user/Downloads/t,/Users/user/Downloads/t/main
[19:19:34] cwd: /Users/user/Downloads/t
[19:19:34] LaTeX build process spawned. PID: 33171.
[19:19:36] LaTeX log parsed with 0 messages.
[19:19:36] Recipe returns with error: 12/null. PID: 33171. message: Rc files read:
  NONE
Latexmk: This is Latexmk, John Collins, 13 September 2020, version: 4.70a.
Rule 'pdflatex': The following rules & subrules became out-of-date:
      'pdflatex'
------------
Run number 1 of rule 'pdflatex'
------------
------------
Running 'pdflatex  -synctex=1 -interaction=nonstopmode -file-line-error -recorder -output-directory="/Users/user/Downloads/t"  "/Users/user/Downloads/t/main.tex"'
------------
Latexmk: Non-existent bbl file '/Users/user/Downloads/t/main.bbl'
 No file main.bbl.
Latexmk: Log file says output to 'main.pdf'
Latexmk: Found bibliography file(s) [references.bib]
Rule 'bibtex /Users/user/Downloads/t/main': File changes, etc:
   Changed files, or newly in use since previous run(s):
      '/Users/user/Downloads/t/main.aux'
------------
Run number 1 of rule 'bibtex /Users/user/Downloads/t/main'
------------
------------
Running 'bibtex  "/Users/user/Downloads/t/main.aux"'
------------

bibtex: Not writing to /Users/user/Downloads/t/main.blg (openout_any = p).
Collected error summary (may duplicate other messages):
  bibtex /Users/user/Downloads/t/main: Could not open bibtex log file for '/Users/user/Downloads/t/main'
Latexmk: Use the -f option to force complete processing,
 unless error was exceeding maximum runs, or warnings treated as errors.
.

Developer Tools Console [Required]

notificationsAlerts.ts:40 Recipe terminated with error.
onDidChangeNotification @ notificationsAlerts.ts:36

Desktop [Required]

• OS: macOS 10.15.7
• VS Code version: 1.49.2
• Extension version: 8.13.2
• TeX distribution version: TeX Live 2020 / MacTeX-2020 Full Distribution

Additional questions

Are you using VSCodium?

No

Are you using LaTeX Workshop with VS Code Remote?

No

[jlelong]

I confirm the bug with latexmk 4.70a. Everything was working fine with 4.69a. From latexmk's Changelog

Correction of default values of $biber and $bibtex.
Improved handling of location of bbl and aux files on an intial run.
Change of default: running bibtex no longer uses temporary change of
directory, since the bug in bibtex that it evades isn't present in
recent versions.

The real problem is calling bibtex with a full path bibtex "/Users/user/Downloads/t/main.aux"

[emersonmello]

Hi @jlelong ,

Thanks!

You are right. I'm getting same problem when using bibtex tool with %OUTDIR%

On my setting.json

"latex-workshop.latex.outDir": "%DIR%/build",
"latex-workshop.latex.tools": [
{
            "name": "pdflatex",
            "command": "pdflatex",
            "args": [
                "-synctex=1",
                "-interaction=nonstopmode",
                "-file-line-error",
                "-output-directory=%OUTDIR%",
                "%DOC%"
            ],
            "env": {}
        },
        {
            "name": "bibtex",
            "command": "bibtex",
            "args": [
                "%OUTDIR%/%DOCFILE%"
            ],
            "env": {}
        }
]

And the recipe: pdflatex -> bibtex -> pdflatex x2 results in the same error.

bibtex: Not writing to /Users/user/Downloads/t/build/main.blg (openout_any = p).

On terminal I'm able to compile with success.

mkdir build
pdflatex -output-directory=build main.tex
bibtex build/main
pdflatex -output-directory=build main.tex
pdflatex -output-directory=build main.tex

[jlelong]

More precisely, bibtex can only write to a subdirectory of the current one. Hence, bibtex build/main works.
However, note that the path passed to bibtex is not resolved and therefore paths like ../build/main fail.

[jlelong]

From latexmk man page

   $bibtex_fudge [0]
         When using bibtex, whether to take special action to allow older versions of bibtex  to  work  when  $out_dir  or
        $aux_dir is specified.  With bibtex from about 2019, this special action is longer be required; hence the default
         is not to do it.

In 4.69a, the default value for bibtex_fudge was 1. You can set it back by adding to your latexmkrc

$bibtex_fudge=1;

[BerriJ]

This problem was also discussed here. I mailed John Collins about this problem. Let's see what he says.

Besides that: another option of solving this problem temporarily is modifying latex-workshop.latex-tools like so:

"latex-workshop.latex.tools": [
            {
                "name": "latexmk",
                "command": "latexmk",
                "args": [
                    "-synctex=1",
                    "-interaction=nonstopmode",
                    "-file-line-error",
                    "-pdf",
                    //"-outdir=%OUTDIR%",
                    "%DOC%"
                ],
                "env": {}
            }
...

[jlelong]

I also mailed John yesterday evening. I am afraid he will get a lot of mails!
It is possible to set bibtex_fudge=1 directly in the recipe configuration by adding to the args array

"-e", "$bibtex_fudge=1"

[emersonmello]

I also mailed John yesterday evening. I am afraid he will get a lot of mails!
It is possible to set bibtex_fudge=1 directly in the recipe configuration by adding to the args array

"-e", "$bibtex_fudge=1"

@jlelong, it worked like a charm.

@BerriJ , I prefer to use a outdir 🙄, so @jlelong's workaround is perfect for me.

[BerriJ]

John Collins stated here that this will be fixed in the next version which should be out "soon". :-)