Tinytex: Feature request: extended prebuilt version

[AlfonsoMuskedunder]

TinyTeX has the very useful - for users behind a firewall - command tinytex:::install_prebuilt() which allows the installation of core TinyTeX.

The same underlying problem carries over to the ex-post installation of packages not included in the core installation, among them extremely popular packages such as beamer.

Technically proficient users might be able to apply appropriate proxy settings, but it is my understanding that even that might not suffice, as per issue https://github.com/yihui/tinytex/issues/50.

I'd like to ask to consider the possibility of providing a somewhat extended build with select, very popular packages that could be installed analogous to install_prebuilt(). (Or maybe even the combination of install_prebuilt() and install_yihui_pkgs()?)

Disclaimer: I posted a similar question over at the RStudio community a while ago: https://community.rstudio.com/t/tinytex-install-missing-latex-packages-behind-firewall/62303

---

By filing an issue to this repo, I promise that

• [x] I have fully read the issue guide at https://yihui.org/issue/.
• [x] I have provided the necessary information about my issue.
  
  
  
  • If I'm asking a question, I have already asked it on Stack Overflow or RStudio Community, waited for at least 24 hours, and included a link to my question there.
  
  
• [x] I have learned the Github Markdown syntax, and formatted my issue correctly.

I understand that my issue may be closed if I don't fulfill my promises.

[yihui]

Sure. I just took your suggestion and installed those additional packages in the prebuilt version. Thanks!

[AlfonsoMuskedunder]

That's awesome, can't wait to try it on Monday. Thank you!

[AlfonsoMuskedunder]

@yihui A big thank you again for extending the prebuilt version. I just reinstalled it at work, and now beamer et al. are there.

However, compiling a pdf still consistently fails, with the error:

! LaTeX Error: File `epstopdf-base.sty' not found.
...
!  ==> Fatal error occurred, no output PDF file produced!

I suppose that's a package that's a dependency to one of the packages loaded by default? (I cannot figure out which...)

If it is not too much to ask: Could you make sure that the prebuilt version is self-contained, i.e. that one can compile a simple (in the sense that it does not require any packages not yet included in the prebuilt version) article or a beamer presentation 'out of the box'?

[yihui]

@AlfonsoMuskedunder My intention was indeed to make the basic installation of TinyTeX able to compile common R Markdown documents. There have been some infrastructure changes in CTAN in the past few months, and I think I need to reexamine which packages I need to include in pkgs-custom.txt. I'll do this later. Thanks for the report!

[yihui]

Done. The epstopdf-pkg package is included in the basic installation now: https://github.com/yihui/tinytex/blob/c0fc376dea2364c0a6d251e23d798af255410d69/tools/pkgs-custom.txt#L12

However, currently there exists an unfortunate bug #210, which means tinytex:::install_prebuilt() won't work until it is resolved by TeX Live maintainers...

[yihui]

FYI the prebuilt version is back now.

[AlfonsoMuskedunder]

Hi @yihui , can I ask if you changed something with regard to the servers where the prebuilt versions are stored?

In the past, when using tinytex:::install_prebuilt(), it consistently succeeded on the second attempt (meaning that the message versuche URL 'https://ci.appveyor.com/api/projects/yihui/tinytex/artifacts/TinyTeX.zip' appeared twice).

Now downloading fails entirely. This is the output:

> tinytex:::install_prebuilt()
versuche URL 'https://ci.appveyor.com/api/projects/yihui/tinytex/artifacts/TinyTeX.zip'
versuche URL 'https://ci.appveyor.com/api/projects/yihui/tinytex/artifacts/TinyTeX.zip'
versuche URL 'https://ci.appveyor.com/api/projects/yihui/tinytex/artifacts/TinyTeX.zip'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:05:00 --:--:--     0
curl: (28) Connection timed out after 300784 milliseconds
Fehler in download.file(url, output, ..., method = method) : 
  'curl' call had nonzero exit status
Zusätzlich: Warnmeldungen:
1: In download.file(url, output, ..., method = method) :
  URL 'https://ci.appveyor.com/api/projects/yihui/tinytex/artifacts/TinyTeX.zip': status was 'Couldn't connect to server'
2: In download.file(url, output, ..., method = method) :
  cannot open URL 'https://ci.appveyor.com/api/projects/yihui/tinytex/artifacts/TinyTeX.zip': HTTP status was '400 Bad Request'
3: In download.file(url, output, ..., method = method) :
  cannot open URL 'https://ci.appveyor.com/api/projects/yihui/tinytex/artifacts/TinyTeX.zip': HTTP status was '400 Bad Request'

Now I'm wondering if something has changed compared with a month or two ago?

[yihui]

@AlfonsoMuskedunder For now, you need the dev version of tinytex: remotes::install_github('yihui/tinytex'). I plan to make a new release to CRAN soon.

[yihui]

tinytex 0.23 is on CRAN now.

[AlfonsoMuskedunder]

_Edit: Apologies, I studied the release notes only after posting here. I downloaded the zip file myself and everything worked! :tada:
Again, thank you very much for tackling and solving this whole issue!_

@yihui I updated tinytex to 0.23. Alas, the install command aborts immediately without an attempt at downloading the zip file:

> tinytex:::install_prebuilt()
Warnmeldung:
In extract(path, exdir = path.expand(dirname(default_inst))) :
  Fehler 1 während des Extrahierens aus Zipfile

[yihui]

That's weird... Can this function download the zip file successfully?

tinytex:::download_installer('TinyTeX.zip')

The zip file should be about 154Mb instead of a few bytes.

[AlfonsoMuskedunder]

Not at the office; will try on Monday! :crossed_fingers:

[yihui]

No problem. It will be even better if you could tell me the output of the code below.

# test unzip()
tinytex:::download_installer('TinyTeX.zip')
str(unzip('TinyTeX.zip'))
list.files('TinyTeX')

# test untar()
tinytex:::download_installer('TinyTeX.tar.gz')
str(untar('TinyTeX.tar.gz'))
list.files('.TinyTeX')

# clean up files extracted
unlink(c('TinyTeX', '.TinyTeX'), recursive = TRUE)

[AlfonsoMuskedunder]

I finally got around to running the above code. Before running it, I tried tinytex:::install_prebuilt(), with the same error as above in https://github.com/yihui/tinytex/issues/207#issuecomment-631274996. Here's the output to your code:
```{r}

test unzip()

tinytex:::download_installer('TinyTeX.zip')
versuche URL 'https://yihui.org/tinytex/TinyTeX.zip'
versuche URL 'https://yihui.org/tinytex/TinyTeX.zip'
Content type 'application/octet-stream' length 163464830 bytes (155.9 MB)
downloaded 155.9 MB

[1] 0
Warnmeldung:
In download.file(url, output, ..., method = method) :
URL 'https://yihui.org/tinytex/TinyTeX.zip': status was 'Couldn't connect to server'

str(unzip('TinyTeX.zip'))
chr [1:16083] "./TinyTeX/install-tl" "./TinyTeX/install-tl-windows.bat" "./TinyTeX/LICENSE.CTAN" "./TinyTeX/LICENSE.TL" ...
list.files('TinyTeX')
[1] "bin" "install-tl" "install-tl-windows.bat" "LICENSE.CTAN"
[5] "LICENSE.TL" "release-texlive.txt" "texmf-config" "texmf-dist"
[9] "texmf-local" "texmf-var" "texmf.cnf" "texmfcnf.lua"
[13] "tl-tray-menu.exe" "tlpkg"

test untar()

tinytex:::download_installer('TinyTeX.tar.gz')
versuche URL 'https://yihui.org/tinytex/TinyTeX.tar.gz'
versuche URL 'https://yihui.org/tinytex/TinyTeX.tar.gz'
Content type 'application/octet-stream' length 130670117 bytes (124.6 MB)
downloaded 124.6 MB

[1] 0
Warnmeldung:
In download.file(url, output, ..., method = method) :
URL 'https://yihui.org/tinytex/TinyTeX.tar.gz': status was 'Couldn't connect to server'

str(untar('TinyTeX.tar.gz'))
/usr/bin/tar: .TinyTeX/texmf-var/fonts/map/dvips/updmap/psfonts.map: Cannot create symlink to 'psfonts_t1.map': No such file or directory
/usr/bin/tar: .TinyTeX/bin/x86_64-linux/extractbb: Cannot create symlink to 'xdvipdfmx': No such file or directory
/usr/bin/tar: .TinyTeX/bin/x86_64-linux/texlua: Cannot create symlink to 'luatex': No such file or directory
/usr/bin/tar: .TinyTeX/bin/x86_64-linux/dvipdfm: Cannot create symlink to 'xdvipdfmx': No such file or directory
/usr/bin/tar: .TinyTeX/bin/x86_64-linux/texluac: Cannot create symlink to 'luatex': No such file or directory
/usr/bin/tar: .TinyTeX/bin/x86_64-linux/lualatex: Cannot create symlink to 'luahbtex': No such file or directory
/usr/bin/tar: .TinyTeX/bin/x86_64-linux/texhash: Cannot create symlink to 'mktexlsr': No such file or directory
/usr/bin/tar: .TinyTeX/bin/x86_64-linux/inimf: Cannot create symlink to 'mf': No such file or directory
/usr/bin/tar: .TinyTeX/bin/x86_64-linux/mktexfmt: Cannot create symlink to 'fmtutil': No such file or directory
/usr/bin/tar: .TinyTeX/bin/x86_64-linux/man: Cannot create symlink to '../../texmf-dist/doc/man': No such file or directory
/usr/bin/tar: Exiting with failure status due to previous errors
int 2
Warnmeldung:
In untar("TinyTeX.tar.gz") :
‘tar.exe -xf "TinyTeX.tar.gz"’ returned error code 2
list.files('.TinyTeX')
[1] "bin" "install-tl" "LICENSE.CTAN" "LICENSE.TL" "release-texlive.txt"
[6] "texmf-config" "texmf-dist" "texmf-local" "texmf-var" "texmf.cnf"
[11] "texmfcnf.lua" "tlpkg"

clean up files extracted

unlink(c('TinyTeX', '.TinyTeX'), recursive = TRUE)

Remarkably, _after_ running your code, the "regular" prebuilt command succeeded:
```{r}
> tinytex:::install_prebuilt()
Use of uninitialized value in bitwise or (|) at C:\Users\xyz\AppData\Roaming\TinyTeX\texmf-dist\scripts\texlive\tlmgr.pl line 1497.
texhash: Updating C:/Users/xyz/AppData/Roaming/TinyTeX/texmf-local/ls-R...
texhash: Updated C:/Users/xyz/AppData/Roaming/TinyTeX/texmf-local/ls-R.
texhash: Updating C:/Users/xyz/AppData/Roaming/TinyTeX/texmf-config/ls-R...
texhash: Updated C:/Users/xyz/AppData/Roaming/TinyTeX/texmf-config/ls-R.
texhash: Updating C:/Users/xyz/AppData/Roaming/TinyTeX/texmf-var/ls-R...
texhash: Updated C:/Users/xyz/AppData/Roaming/TinyTeX/texmf-var/ls-R.
texhash: Updating C:/Users/xyz/AppData/Roaming/TinyTeX/texmf-dist/ls-R...
texhash: Updated C:/Users/xyz/AppData/Roaming/TinyTeX/texmf-dist/ls-R.
texhash: Done.
...xyz\AppData\Roaming\TinyTeX\bin\win32\runscript.tlu:935: ...xyz\AppData\Roaming\TinyTeX\bin\win32\runscript.tlu:858: no appropriate script or program found: updmap~1
Note that it needs some time to create caches,
especially if there are many large font files.
Wait with patience.

Font directories:
    C:/Windows/fonts
    C:/Users/appveyor/AppData/Roaming/TinyTeX/texmf-dist/fonts/opentype
    C:/Users/appveyor/AppData/Roaming/TinyTeX/texmf-dist/fonts/truetype
C:/Windows/fonts: caching, new cache contents: 985 fonts, 0 dirs
C:/Users/appveyor/AppData/Roaming/TinyTeX/texmf-dist/fonts/opentype: skipping, no such directory
C:/Users/appveyor/AppData/Roaming/TinyTeX/texmf-dist/fonts/truetype: skipping, no such directory
C:/Users/xyz/AppData/Roaming/TinyTeX/texmf-var/fonts/cache: cleaning cache directory
C:\Users\xyz\AppData\Roaming\TinyTeX\bin\win32\fc-cache.exe: succeeded

[yihui]

@AlfonsoMuskedunder Great! I think we are all set now. Previously I was using 7-zip to create the zip archive, and it seems that R had problems with unzipping the archive. Then I switched to PowerShell to create the zip archive, and according to your testing, it can be successfully unzip()ped in R. Many thanks for the help with testing!

[AlfonsoMuskedunder]

@yihui I hope I don't get on your nerves by now, but can I ask you to consider extending the prebuilt version one step further, by including those LaTeX packages loaded by the kableExtra package? I suppose the relevance of that package is clear - you devote an entire subsection to it in your R Markdown Cookbook 😉.

I'd love to request the necessary changes to tools/pkgs-yihui.txt myself, as asked for in the latest release notes, but since I can't install LaTeX packages to begin with, I cannot figure out the full set of LaTeX packages required. The compilation process stops when missing pdflscape.sty, but I'd be surprised if there weren't additional missing packages.

I'd understand if you didn't want to bloat your list of packages - but I thought it would not hurt to ask.

[yihui]

@AlfonsoMuskedunder Done!