Sphinx: Cannot generate PDFs with Sphinx version 1.6.1 on Windows. Perl Missing.

[NormanDunbar]

Subject: Cannot generate PDFs with Sphinx version 1.6.1 on Windows. Perl Missing.

Problem

• After upgrading to Sphinx 1.6.1 from 1.5.x, I am no longer able to generate pdf documents with make latexpdf which was working before. The problem is that latexmk is now in use and requires perl to be installed. I don't have perl installed and I am unable to install additional software as this is a work computer.

I have looked around, Google, and here on the issues list, and I'm unable to find anything that works. The command make latexpdf PDFLATEX=pdflatex as found here, doesn't work - it gives exactly the same errors about perl being missing. (See below)

Procedure to reproduce the problem

make latexpdf

Error logs / results

Running Sphinx v1.6.1
loading pickled environment... done
building [mo]: targets for 0 po files that are out of date
building [latex]: all documents
updating environment: 0 added, 0 changed, 0 removed
looking for now-outdated files... none found
processing UpdatingSuperBASICManual.tex...index Part1 p1_intro p1_software Part2 p2_download Part3 p3_rst Part4 P4_intro

resolving references...
writing... done
copying images... [100%] images\LogIssue.png
copying TeX support files...
done
build succeeded.
latexmk -pdf -dvi- -ps-  'UpdatingSuperBASICManual.tex'
latexmk: The script engine could not be found.
latexmk: Data: scriptEngine="perl.exe"
Makefile:33: recipe for target 'UpdatingSuperBASICManual.pdf' failed
mingw32-make: *** [UpdatingSuperBASICManual.pdf] Error 1

Build finished; the PDF files are in build/latex.

Expected results

I would expect that when I request a pdf to be built, it would be built. As it did in the previous version, or, in the event that a component of the build has changed dramatiacally, as in this case, that the install/upgrade notes would make the additional dependencies known, and/or an option to use a different utility be supplied to revert to the previous working operation.

Windows isn't the best of operating systems for having decent utilities installed by default! ;-)

Reproducible project / your project

• Not required? Any project will do, just don't have perl installed.

Environment info

• OS: Windows 7. 64 bit.
• Python version: 3.5.1
• Sphinx version: 1.6.1
• MikTex: 2.9

Workaround?

I have a workaround which I didn't need to use under Sphinx 1.5.x:

make latexpdf ... cd build\latex pdflatex whatever.tex
That generates the pdf as desired but using pdflatex as provided by MikTex. I have seen a mention _somewhere_ that MikTex provides perl but at least at version 2.9, it does not.

[jfbu]

Very sorry. Independently of the issue with Latexmk, unfortunately we could not provide a Makefile which works both on GNU/Linux and Windows due to diverging shell syntax.

http://www.sphinx-doc.org/en/stable/builders.html#sphinx.builders.latex.LaTeXBuilder

This was discussed at #3082. It is possible though that your make executable does understand GNU/Linux shell syntax then your problem is only with missing Latexmk install.

The homepage of Latexmk.

It is provided with MikTeX but you may need extra Perl install. This

https://tex.stackexchange.com/questions/336914/cannot-use-latexmk-on-miktex-2-9

mentions Strawberry Perl v5.24.0 and then says that after updating MikTeX it works fine.

But you mentioned you could not add additional software to your work computer. Then Plan B below.

But, as I mentioned currently the Latexmk support file latexmkrc which Sphinx puts in build repertory is with Unix shell syntax. So even if you have a working Windows Latexmk, you may still encounter problems.

Plan B: There is a make.bat file provided by sphinx-quickstart. It does not use Latexmk. You should use the suitable command on Windows for using this make.bat, rather than executing a make (very sorry that I personnally have no Windows hence do not even know the right command name to execute make.bat).

Possibly you need to create empty project with sphinx-quickstart and copy over the make.bat to your project, if you erased it.

[jfbu]

added precision: it is for Japanese documents that we had most trouble with shell syntax in Latexmk support file (latexmkjarc). For non-Japanese documents, the situation is much more favorable, and it could work (but unfortunately you can not add Perl to your work computer).

The incompatible change was announced http://www.sphinx-doc.org/en/stable/changes.html#dependencies

[jfbu]

oh no... from looking at the make.bat template, I see that latexpdf target will cd to the latex build directory and then do a make there. But it looks as if Sphinx puts no make.bat into the latex build repertory. So the latexpdf target for the top level make.bat will end up trying to use Latexmk indeed.

Am I overlooking something @shimizukawa ? it looks that definitely make latexpdf is broken on Windows...

[jfbu]

side note: http://tug.ctan.org/support/latexmk/INSTALL

[jfbu]

As workaround

1. create a _fortex repertory at top level in your project source

2. add latex_additional_files = ['_fortex/Makefile'] to your conf.py

3. Create a file _fortex/Makefile with the following contents

# Makefile for Sphinx LaTeX output

ALLDOCS = $(basename $(wildcard *.tex))
ALLPDF = $(addsuffix .pdf,$(ALLDOCS))
ALLDVI = $(addsuffix .dvi,$(ALLDOCS))
ALLPS  = $(addsuffix .ps,$(ALLDOCS))

# Prefix for archive names
ARCHIVEPREFIX =
# Additional LaTeX options
LATEXOPTS =
# format: pdf or dvi
FMT = pdf

LATEX = latex
PDFLATEX = pdflatex
MAKEINDEX = makeindex


all: $(ALLPDF)
all-pdf: $(ALLPDF)
all-dvi: $(ALLDVI)
all-ps: $(ALLPS)

all-pdf-ja:
    for f in *.pdf *.png *.gif *.jpg *.jpeg; do extractbb $$f; done
    for f in *.tex; do platex -kanji=utf8 $(LATEXOPTS) $$f; done
    for f in *.tex; do platex -kanji=utf8 $(LATEXOPTS) $$f; done
    for f in *.tex; do platex -kanji=utf8 $(LATEXOPTS) $$f; done
    -for f in *.idx; do mendex -U -f -d "`basename $$f .idx`.dic" -s python.ist $$f; done
    for f in *.tex; do platex -kanji=utf8 $(LATEXOPTS) $$f; done
    for f in *.tex; do platex -kanji=utf8 $(LATEXOPTS) $$f; done
    for f in *.dvi; do dvipdfmx $$f; done

zip: all-$(FMT)
    mkdir $(ARCHIVEPREFIX)docs-$(FMT)
    cp $(ALLPDF) $(ARCHIVEPREFIX)docs-$(FMT)
    zip -q -r -9 $(ARCHIVEPREFIX)docs-$(FMT).zip $(ARCHIVEPREFIX)docs-$(FMT)
    rm -r $(ARCHIVEPREFIX)docs-$(FMT)

tar: all-$(FMT)
    mkdir $(ARCHIVEPREFIX)docs-$(FMT)
    cp $(ALLPDF) $(ARCHIVEPREFIX)docs-$(FMT)
    tar cf $(ARCHIVEPREFIX)docs-$(FMT).tar $(ARCHIVEPREFIX)docs-$(FMT)
    rm -r $(ARCHIVEPREFIX)docs-$(FMT)

gz: tar
    gzip -9 < $(ARCHIVEPREFIX)docs-$(FMT).tar > $(ARCHIVEPREFIX)docs-$(FMT).tar.gz

bz2: tar
    bzip2 -9 -k $(ARCHIVEPREFIX)docs-$(FMT).tar

xz: tar
    xz -9 -k $(ARCHIVEPREFIX)docs-$(FMT).tar

# The number of LaTeX runs is quite conservative, but I don't expect it
# to get run often, so the little extra time won't hurt.
%.dvi: %.tex
    $(LATEX) $(LATEXOPTS) '$<'
    $(LATEX) $(LATEXOPTS) '$<'
    $(LATEX) $(LATEXOPTS) '$<'
    -$(MAKEINDEX) -s python.ist '$(basename $<).idx'
    $(LATEX) $(LATEXOPTS) '$<'
    $(LATEX) $(LATEXOPTS) '$<'

%.pdf: %.tex
    $(PDFLATEX) $(LATEXOPTS) '$<'
    $(PDFLATEX) $(LATEXOPTS) '$<'
    $(PDFLATEX) $(LATEXOPTS) '$<'
    -$(MAKEINDEX) -s python.ist '$(basename $<).idx'
    $(PDFLATEX) $(LATEXOPTS) '$<'
    $(PDFLATEX) $(LATEXOPTS) '$<'

%.ps: %.dvi
    dvips '$<'

clean:
    rm -f *.log *.ind *.aux *.toc *.syn *.idx *.out *.ilg *.pla *.ps *.tar *.tar.gz *.tar.bz2 *.tar.xz $(ALLPDF) $(ALLDVI)

.PHONY: all all-pdf all-dvi all-ps clean zip tar gz bz2 xz
.PHONY: all-pdf-ja

This is the 1.5.6 Makefile, for pdflatex. Then make latexpdf should work without Latexmk like earlier.

I could not use a template file here because https://github.com/sphinx-doc/sphinx/blob/master/sphinx/builders/latex.py#L224 does not have the context. Hence I rendered the file directly for latex_engine = pdflatex.

[tk0miya]

Yes, we decided to depend on latexmk since 1.6 release. So this is intended change.

But missing make.bat might be a bug.

[NormanDunbar]

Good Evening All,

many thanks for a swift response. I can confirm that the workaround explained by jfbu here works like a charm.

I do have a windows version of make on my PC so I am able to run it with the generated makefiles manually as I need to use mingw32-make rather than make, but that's ok.

Kind regards,
Norm.

[tk0miya]

But missing make.bat might be a bug.

I noticed Sphinx did not bundled make.bat ever. So this is not a bug. It's an improvement.
(Sphinx requires extra GNU make command and latexmk command now)

I think this is closable. Any comment?

[jfbu]

It would be nice to get confirmation that on Windows with Latexmk installed, the latex Makefile works as expected and make latexpdf at top repertory works. Perhaps this can be done with appveyor?

And we should update documentation to say Latexmk command is required on all platforms for make latexpdf.

I doubt it is 100% Windows compatible especially for the case of Japanese documents because I was told that syntax such as

$makeindex = 'rm -f %D; mendex -U -f -d %B.dic -s python.ist %S || echo "mendex exited with error code $? (ignoring)" && : >> %D';

in latexmkjarc would not work on Windows. I think situation is much less problematic for non-Japanese, but their might be problems with quotes in filenames: but this was already case with older latex Makefile, hence is not new problem.

[jfbu]

I have updated docs at 350f58a. I changed a posteriori 1.6b1 entry in CHANGES, I hope this is fine. +1 for close as soon as someone confirms make latexpdf actually works on Windows with required GNU make and Latexmk.

[shimizukawa]

I tried 'make latexpdf' on Windows, but I got another error. It's filed as #3817.

[shimizukawa]

@jfbu I tried make latexpdf with sphinx/doc on Windows.

(Sphinx stable HEAD (and 1.6.1) still have #3817 issue, however I bypassed it temporary) to check latexpdf target.)

I doubt it is 100% Windows compatible especially for the case of Japanese documents because I was told that syntax such as ... in latexmkjarc would not work on Windows.

Indeed. It failed.

make[1]: ディレクトリ `C:/Project/sphinx-dev/sphinx/doc/_build/latex' に入ります
extractbb 'agogo.png'
extractbb 'alabaster.png'
extractbb 'bizstyle.png'
extractbb 'classic.png'
extractbb 'haiku.png'
extractbb 'more.png'
extractbb 'nature.png'
extractbb 'pyramid.png'
extractbb 'scrolls.png'
extractbb 'sphinx.png'
extractbb 'sphinx_rtd_theme.png'
extractbb 'sphinxdoc.png'
extractbb 'traditional.png'
extractbb 'translation.png'
for f in *.pdf; do extractbb "$f"; done
f の使い方が誤っています。
make[1]: *** [sphinx.pdf] エラー 255
make[1]: ディレクトリ `C:/Project/sphinx-dev/sphinx/doc/_build/latex' から出ます```

It means

for f in *.pdf; do extractbb "$f"; done
The syntax of the command is incorrect
make[1]: *** [sphinx.pdf] Error 255

[jfbu]

I do not know offhand mechanism for Sphinx detects platform but the simplest fix for 1.6.2 would be to get Sphinx on Windows to use the old Makefile_t which was in sphinx/texinputs.

At least then we recover former situation and drop Latexmk dependency on Windows. I will research implementation but can not test...

[jfbu]

@shimizukawa can you give a look at #3819 ?

[jfbu]

@NormanDunbar we are going to incorporate the workaround as default behaviour with 1.6.2 release, because even with Latexmk correctly installed on Windows, the Makefile and Latexmk support files do not use Windows compatible shell syntax. So with 1.6.2 you won't need the workaround.

[tk0miya]

3819 is merged now!

[NormanDunbar]

Many thanks.