Nixpkgs: texlive: xelatex doesn't see fonts

[jbaum98]

Issue description

On Darwin, texlive does not system or nixpkgs installed fonts.

Steps to reproduce

```test.tex
\documentclass{minimal}
\usepackage{fontspec}
setmainfont{Latin Modern Sans}
\begin{document}
Hello
\end{document}

```bash
$ nix-shell --pure -p "(texlive.combine { inherit (texlive) scheme-basic xetex xetex-def lm fontspec euenc; })" lmodern which --command 'xelatex fontspec-test.tex'

Technical details

• System: OSX 10.12.4
• Nix version: 1.11.6
• Nixpkgs version: 17.09pre104446.09c91f5

[vbgl]

XeLaTeX from nixpkgs, on darwin, uses fontconfig (maybe this is an issue that can be addressed).

I don’t know how to compile your document in a pure shell, but in an impure environment you can:

• install fontconfig and the desired fonts (with nix-env -iA nixpkgs.fontconfig nixpkgs.lmodern);
• make fontconfig willing to use our impure setting: export FONTCONFIG_FILE=$HOME/.nix-profile/etc/fonts/fonts.conf
• update the font cache fc-cache -f --verbose
• check that fc-match "Latin Modern Sans" returns something appropriate (e.g. lmsans10-regular.otf: "Latin Modern Sans" "10 Regular")
• run XeLaTeX in a nix shell nix-shell -p "(texlive.combine { inherit (texlive) scheme-basic xetex xetex-def fontspec euenc; })" --command 'xelatex fontspec-test.tex'

[jbaum98]

My main goal is to be able to create the output PDF with only nix-build, so I guess I'm looking for a solution that works in a pure environment. I don't see why this shouldn't be possible in theory, as in it feels like this it is consistent with the "nixpkgs philosophy" that this would be a thing you could do.

[vbgl]

I’m not sure how the nixpkgs philosophy is compatible with the fontconfig philosophy. Your problem boils down to the fact that the following does not work as we may expect; even on NixOS:

nix-shell --pure -p lmodern fontconfig --command 'fc-match "Latin Modern Sans"'

BTW you can use nix to resolve the fonts and do things like the following in your TeX file:

\setmainfont[Path=/nix/store/2hk1ji0yvszs6nzmx4vk6l7kgqxy3gdn-lm-2.004/fonts/opentype/public/lm/]{lmsans10-regular.otf}

[jbaum98]

Disclaimer: I know almost nothing about fontconfig. However, the way I see it is I want my PDF to be built as a pure function of its inputs, which include a specific set of latex packages and some fonts. Why shouldn't that work?

About that last example, that would almost be what I'm looking for except it seems really ugly that I would have to a) hardcode the value to the location in the nix store and b) if it somehow got updated that path would change and it would break? But if there was some reliable way to get the path to the nix store where my font is, that would be awesome. I was thinking about somehow setting environment variables from default.nix, but how would I get them in the tex file? I guess if I needed to I could have a preConfigure script that replaces some marker text, but that would still be non-ideal... curious if you have better ideas.

[vcunat]

It might work with FONTCONFIG_FILE = pkgs.makeFontsConf { ... }. See nixpkgs for non-TeX examples, but I sadly don't remember the state of fontconfig support in our texlive. EDIT: I think this isn't Darwin-specific but completely general.

[dtzWill]

I can confirm the FONTCONFIG_FILE with makeFontConf, I've been using this to automate xelatex docs for months, seems to do the trick at least for my case. I also find it helpful to add that to the shellHook for use while editing and incremental builds.

[dtzWill]

Snippet, although unsure of the accuracy of my comment just a note to myself :).

let
  # grab fonts we need, xetex wants them installed as system fonts
  # There's a "fira" package in texlive, but I can't seem to get it to work with xetex or luatex,
  # possibly a bug in how nixpkgs packages them (there's a TODO about font visibility).
  fonts = makeFontsConf { fontDirectories = [ freefont_ttf fira fira-mono ]; };
in with import ./src.nix;
stdenv.mkDerivation {
  name = "allvm-slides-${version}";
  inherit version;
  src = fetchgit srcinfo;

  buildInputs = [ tex ghostscript pandoc ];

  preBuild = ''
    export FONTCONFIG_FILE=${fonts}
  '';

[jbaum98]

Amazing! I will post here my minimal working solution. Should there be something about this in the documentation for texlive on nixpkgs?

{ nixpkgs ? import <nixpkgs> {} }:
with nixpkgs.pkgs;
stdenv.mkDerivation {
  name = "textest";
  src = ./.;
  buildInputs = [
    (texlive.combine {
      inherit (texlive) scheme-basic xetex xetex-def fontspec euenc;
    })
  ];

  FONTCONFIG_FILE = makeFontsConf { fontDirectories = [ lmodern ]; };

  buildPhase = ''
    xelatex test.tex
  '';

  installPhase = ''
    mkdir -p $out
    cp test.pdf $out
  '';
}

\documentclass{minimal}
\usepackage{fontspec}
\setmainfont{Latin Modern Roman}
\begin{document}
Hello
\end{document}

_Sent from my LGE Nexus 5X using FastHub_

[vcunat]

It could be a subsection in the manual or perhaps a tutorial in the cookbook.

[srivathsanmurali]

@jbaum98 Hi, I tried your solution. And got this error

error: while evaluating the attribute ‘nativeBuildInputs’ of the derivation ‘textest’ at /home/sri/github/CV/default.nix:4:3: cannot coerce a function to a string, at /home/sri/github/CV/default.nix:4:3
Any idea why I would get that?

Thanks

[dtzWill]

@srivathsanmurali confirmed. Put parenthesis around the texlive.combine { ... } bit to fix it.

[srivathsanmurali]

@dtzWill Great. That worked :D

[jbaum98]

I've updated my previous answer with this fix.

_Sent from my LGE Nexus 5X using FastHub_

[loskutov]

Is there a way to use the built-in system fonts as well? It appears to break the purity but that's still pretty useful...

[vcunat]

I would expect that to work already – if you don't define FONTCONFIG_FILE, the system one is used normally, and that should pick any fonts the system sees. (I didn't test this and may easily be wrong.)

[loskutov]

Well, in my case it doesn't :(

% fc-match "Times New Roman"
Times New Roman.ttf: "Times New Roman" "Regular"

% xelatex demo.tex
This is XeTeX, Version 3.14159265-2.6-0.99996 (Web2C 2016/NixOS.org) (preloaded format=xelatex)

...

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!
! fontspec error: "font-not-found"
! 
! The font "Times New Roman" cannot be found.
! 
! See the fontspec documentation for further information.
! 
! For immediate help type H <return>.
!...............................................

[arianvp]

The mentioned method only seems to work if mentioning a font by Name, not by filename

\setmathfont{latinmodern-math.otf}

does not work.
whilst

\setmathfont{Latin Modern Math}

does work

Many LaTex packages like (unicode-math) use this way of setting fonts, which is still broken

[rnhmjoj]

I used to build a book with pandoc and xelatex and it worked ok (note that I'm not building it in a pure environment). I haven't touched it in a while and now I found out it produces a PDF with no fonts.
I'm not sure what changed.

[jbaum98]

What do you mean by a PDF with no fonts? I build this derivation

{ stdenv, fetchgit, texlive, pandoc, makeFontsConf, lmodern }:
stdenv.mkDerivation rec {
  name = "${pname}-${version}";
  pname = "category-theory-for-programmers";
  version = "0.4.0";

  src = fetchgit {
    url = "https://maxwell.ydns.eu/git/rnhmjoj/category-theory-for-programmers";
    rev = "v${version}";
    sha256 = "0z5bn5147b3r1937cd4ai5fmy44dcxjpwbr45v8mkfbkqpki0lg5";
  };

  buildInputs = [
    (texlive.combine {
      inherit (texlive) scheme-full;
    })
    pandoc
  ];

  FONTCONFIG_FILE = makeFontsConf { fontDirectories = [ lmodern ]; };

  patches = [ ./cat-theory.patch ];

  buildPhase = ''
    make all
  '';

  installPhase = ''
    mkdir -p $out
    cp ${pname}.pdf ${pname}.epub $out/
  '';
}

with this patch because pandoc options have changed

diff --git a/makefile b/makefile
index c07ffc0..6f1fefe 100644
--- a/makefile
+++ b/makefile
@@ -12,10 +12,10 @@ $(title).pdf:
    pandoc                          \
      $(src)/init/preface.html      \
      $(chapters)                   \
-     --smart                       \
+     --from html+smart             \
      --toc                         \
      --number-section              \
-     --latex-engine=xelatex        \
+     --pdf-engine=xelatex        \
      --data-dir=$(root)/pandoc     \
      -H $(root)/pandoc/options.tex \
      -o $(root)/$(title).pdf
@@ -25,10 +25,10 @@ $(title).epub:
    pandoc                          \
      $(src)/init/preface.html      \
      $(chapters)                   \
-     --smart                       \
+     --from html+smart             \
      --toc                         \
      --number-section              \
-     --epub-stylesheet=$(root)/pandoc/style.css  \
+     --css=$(root)/pandoc/style.css              \
      --epub-cover-image=$(root)/pandoc/cover.png \
      --epub-embed-font='$(src)/fonts/*.woff'     \
      -o $(root)/$(title).epub

and I get a pdf that looks fine, but it produces a lot of warnings about missing symbols.Does it look right? Do you know what the fonts are supposed to be?

The warnings look like this:

[WARNING] Missing character: There is no ∘ in font [lmroman9-regular]:mapping=tex-text;!
[WARNING] Missing character: There is no ∘ in font [lmroman9-regular]:mapping=tex-text;!
[WARNING] Missing character: There is no ∘ in font [lmmono9-regular]:!
[WARNING] Missing character: There is no ∷ in font [lmmono9-regular]:!
[WARNING] Missing character: There is no ∘ in font [lmmono9-regular]:!
[WARNING] Missing character: There is no ∘ in font [lmmono9-regular]:!
[WARNING] Missing character: There is no ∘ in font [lmmono9-regular]:!
[WARNING] Missing character: There is no ∘ in font [lmmono9-regular]:!
[WARNING] Missing character: There is no ∘ in font [lmmono9-regular]:!
[WARNING] Missing character: There is no ∘ in font [lmmono9-regular]:!
[WARNING] Missing character: There is no ∘ in font [lmmono9-regular]:!
[WARNING] Missing character: There is no ∘ in font [lmmono9-regular]:!
[WARNING] Missing character: There is no ⊥ in font [lmroman9-regular]:mapping=tex-text;!
[WARNING] Missing character: There is no ∘ in font [lmmono9-regular]:!
[WARNING] Missing character: There is no λ in font [lmroman9-regular]:mapping=tex-text;!
[WARNING] Missing character: There is no ∘ in font [lmmono9-regular]:!
[WARNING] Missing character: There is no ∘ in font [lmmono9-regular]:!
[WARNING] Missing character: There is no ∘ in font [lmroman9-italic]:mapping=tex-text;!
[WARNING] Missing character: There is no ∘ in font [lmroman9-italic]:mapping=tex-text;!
[WARNING] Missing character: There is no ∘ in font [lmroman9-italic]:mapping=tex-text;!
[WARNING] Missing character: There is no ∘ in font [lmmono9-regular]:!
[WARNING] Missing character: There is no ∘ in font [lmmono9-regular]:!
[WARNING] Missing character: There is no ∘ in font [lmmono9-regular]:!
...

[rnhmjoj]

@jbaum98 The PDF you produced looks exactly like mine: this probably mean there is an issue with my PDF viewer or maybe my NixOS configuration. Here's what I see:

[jbaum98]

Hmm so maybe the PDF being produce doesn't have the fonts embedded, but I have the fonts installed globally so its good for me.

[jbaum98]

I used pdffonts to check if the fonts are embedded, and it seems like they are...

name                                 type              emb sub uni prob object ID
------------------------------------ ----------------- --- --- --- ---- ---------
LRFKQY+LMRoman12-Bold-Identity-H     CID Type 0C       yes yes yes           5  0
UQFJKX+LMRoman9-Bold-Identity-H      CID Type 0C       yes yes yes           7  0
YQVNRT+LMRoman17-Regular-Identity-H  CID Type 0C       yes yes yes          15  0
YSFRZR+LMRoman9-Regular-Identity-H   CID Type 0C       yes yes yes          20  0
MZLYGF+LMRoman10-Regular-Identity-H  CID Type 0C       yes yes yes          55  0
VEULQF+LMMono9-Regular-Identity-H    CID Type 0C       yes yes yes         192  0
PDBHLQ+LMRoman9-Italic-Identity-H    CID Type 0C       yes yes yes         301  0
DQRRCR+LMRoman6-Regular-Identity-H   CID Type 0C       yes yes yes         318  0
WDNXEU+LMRoman10-Bold-Identity-H     CID Type 0C       yes yes yes         476  0
OQQEKL+LMRoman7-Italic-Identity-H    CID Type 0C       yes yes yes         508  0
AITGDP+LMMono10-Regular-Identity-H   CID Type 0C       yes yes yes         617  0
HCSPIJ+LMRoman10-Italic-Identity-H   CID Type 0C       yes yes yes         619  0
THBSFY+LMRoman7-Regular-Identity-H   CID Type 0C       yes yes yes         974  0
FVLADZ+LMMonoLt10-Bold-Identity-H    CID Type 0C       yes yes yes        1011  0
PBFUDK+LMRoman6-Bold-Identity-H      CID Type 0C       yes yes yes        1434  0

So I'm not so sure what's going on.

[rnhmjoj]

I tried opening it with a different PDF viewer (evince) and the fonts are displayed correctly so it must be something with zathura. I noticed when I scroll the PDF zathura floods the console with

warning: cannot draw text since font and size not set
error: name too long

I'll look into and maybe report it if it's not known already. Sorry for the noise.

[baracoder]

(triage) @jbaum98 any updates on this issue? From what I understand generating the pdf works now and the fonts are included, right?

[jbaum98]

Yes, for me this issue is resolved, but it seems like @rnhmjoj is having some other issues.

[rnhmjoj]

I reported the issue to zathura and mupdf. It could be NixOS related but it's certainly a different issue so let's close this.

[unode]

Hi guys, sorry to bump an old issue but this is the only place where I found a working example and an explanation on why fonts were not being found in the context of nix-shell.

I'm trying to use fonts that should ship with texlive.combined.scheme-full inside nix-shell in a docker nixos/nix based image and they fail with:

(/nix/store/ab73wanp32wcpmgkjcpd9vjy09cjcs9g-texlive-combined-full-2018/share/texmf/tex/latex/unicode-math/unicode-math-table.tex)))kpathsea:make_tex: Invalid filename `TeX Gyre Pagella', contains ' '

! Package fontspec Error: The font "TeX Gyre Pagella" cannot be found.

I was able to create a working environment by adding:

FONTCONFIG_FILE = pkgs.makeFontsConf { fontDirectories = pkgs.texlive.tex-gyre.pkgs; };

however due to the way texlive fonts are packaged I need to append .pkgs to tex-gyre and work with the entire list.

Question is, should this be necessary in the first place given that I'm using texlive.combined.scheme-full and if yes, would it make sense to have a target for texlive fonts that is easier to refer to with makeFontsConf?

The derivation now looks like:

 pkgs.stdenv.mkDerivation {
   name = "pandocEnv";
   buildInputs = with pkgs; [
     texlive.combined.scheme-full
     pandoc
   ];
   FONTCONFIG_FILE = pkgs.makeFontsConf { fontDirectories = pkgs.texlive.tex-gyre.pkgs; };
} 

[veprbl]

@unode I suppose the internals of the texlive packages were not designed to be a part of the public API, so they are not very convenient to use. The .pkgs suffix is probably there in case we need to add some other attributes with information about the packages. It's important to note that we do want to separate different parts of packages (doc, bin, run, source), so an additional code would be needed in any case if you want to address a specific component. Perhaps a better API would use attrsets instead of lists, but, otherwise, this doesn't seem to be an issue.

If you have some specific ideas about how to improve the situation, feel free to open a new issue.