Overleaf: Not being able to build full image (with scheme-full)

[ferdn4ndo]

Steps to Reproduce

Create a custom build of overleaf with additional packages, as the files:

docker-compose.yaml:
(showing only the diff output from the original file, using diff docker-compose.yaml <(curl -s https://raw.githubusercontent.com/overleaf/overleaf/master/docker-compose.yml) -d)

7,9c7
<         build:
<             context: .
<             dockerfile: Dockerfile
---
>         image: sharelatex/sharelatex
113c111
<             - mongodbdata:/data/db
---
>             - ~/mongo_data:/data/db
150,153d147
<
< volumes:
<   mongodbdata:
<     external: true

So basically i just remove the original link to the dockerhub image and instead tell it do build from a Dockerfile located at the same folder. I also create a volume to store the mongo database (you will need to create this volume to replicate the code as-is)

The Dockerfile:

FROM sharelatex/sharelatex:latest

RUN tlmgr install scheme-full 

ENTRYPOINT ["/sbin/my_init"]

As the Starting Guide suggests.

However, this is what happens:

$ docker-compose up --build
Building sharelatex
Step 1/2 : FROM sharelatex/sharelatex:latest
 ---> 0867310d1151
Step 2/2 : RUN tlmgr install scheme-full
 ---> Running in c853dd06b6c4

tlmgr: verification of checksum for http://osl.ugr.es/CTAN/systems/texlive/tlnet failed: cryptographic signature verification of
  /tmp/tyyWAtINWB/_FpK99S9os
against
  http://osl.ugr.es/CTAN/systems/texlive/tlnet/tlpkg/texlive.tlpdb.sha512.asc
failed. Output was:
gpg: Signature made Fri 01 May 2020 11:58:00 PM UTC
gpg:                using RSA key 4CE1877E19438C70
gpg: BAD signature from "TeX Live Distribution <[email protected]>" [ultimate]

Please try from a different mirror and/or wait a few minutes
and try again; usually this is because of transient updates.
If problems persist, feel free to report to [email protected].

ERROR: Service 'sharelatex' failed to build: The command '/bin/sh -c tlmgr install scheme-full' returned a non-zero code: 1

So after a bit of googling, I ended up getting a mirror from the CTAN official list that was close to me (tested three others with same output). Then I tweaked my Dockerfile:

FROM sharelatex/sharelatex:latest

RUN tlmgr install scheme-full --repository=ftp://linorg.usp.br/CTAN/

ENTRYPOINT ["/sbin/my_init"]

And the output changed:

$ docker-compose up --build
Building sharelatex
Step 1/2 : FROM sharelatex/sharelatex:latest
 ---> 0867310d1151
Step 2/2 : RUN tlmgr install scheme-full --repository "ftp://linorg.usp.br/CTAN/"
 ---> Running in dc9039f01aa7

/usr/local/texlive/2019/bin/x86_64-linux//tlmgr: TLPDB::from_file could not initialize from: ftp://linorg.usp.br/CTAN//tlpkg/texlive.tlpdb
/usr/local/texlive/2019/bin/x86_64-linux//tlmgr: Maybe the repository setting should be changed.
/usr/local/texlive/2019/bin/x86_64-linux//tlmgr: More info: https://tug.org/texlive/acquire.html
ERROR: Service 'sharelatex' failed to build: The command '/bin/sh -c tlmgr install scheme-full --repository "ftp://linorg.usp.br/CTAN/"' returned a non-zero code: 1

Expected Behaviour

I should be able to have the image with scheme-full installed

Observed Behaviour

The hardcoded repository doesn't work anymore and the new one seems not to be compatible

Context

I'd like to have my custom docker image with the full image

Technical Info

Provided above.

Analysis

Provided above.

[jdleesmiller]

Unfortunately, this is an upstream TeXLive issue, based on mailing list traffic. They say it should clear up in a few days. A workaround (also from the mailing list) is to update with --verify-repo=none.

[ferdn4ndo]

@jdleesmiller, the proposed solution didn't work:

$ docker-compose up --build
Building sharelatex
Step 1/2 : FROM sharelatex/sharelatex:latest
 ---> 0867310d1151
Step 2/2 : RUN tlmgr install scheme-full --verify-repo=none
 ---> Running in 29fe3016786d

tlmgr: Local TeX Live (2019) is older than remote repository (2020).
Cross release updates are only supported with
  update-tlmgr-latest(.sh/.exe) --update
See https://tug.org/texlive/upgrade.html for details.
ERROR: Service 'sharelatex' failed to build: The command '/bin/sh -c tlmgr install scheme-full --verify-repo=none' returned a non-zero code: 1

The first reported attempt (with a different repo) is also failing with the same error.

[oli-ver]

@jdleesmiller I think this issue needs to be reopened. Just like @ferdn4ndo stated the upgrade to Latex 2020 right now does not allow to install packages or updates anymore. Do you perhaps have a hint for a workaround?

[jdleesmiller]

It looks to me like the error in https://github.com/overleaf/overleaf/issues/731#issuecomment-625569169 is different:

tlmgr: Local TeX Live (2019) is older than remote repository (2020).

I think you're probably looking for the solution here (changing the year to 2019): https://tex.stackexchange.com/questions/429714/tlmgr-remote-repository-is-newer-than-local-2017-2018

[ferdn4ndo]

THERE'S A SOLUTION!

or at least it seems so

RUN wget http://mirror.ctan.org/systems/texlive/tlnet/update-tlmgr-latest.sh && \
    chmod +x update-tlmgr-latest.sh && ./update-tlmgr-latest.sh && \
    tlmgr install scheme-full --verify-repo=none

---

Adapting the answer from nikhilweee at tex.stackexchange.com/questions/429714, I tried:

FROM sharelatex/sharelatex:latest
RUN tlmgr option repository ftp://tug.org/historic/systems/texlive/2019/tlnet-final
RUN tlmgr install scheme-full --verify-repo=none

which did output:

Building sharelatex
Step 1/3 : FROM sharelatex/sharelatex:latest
 ---> 0867310d1151
Step 2/3 : RUN tlmgr option repository ftp://tug.org/historic/systems/texlive/2019/tlnet-final
 ---> efad2c59f73b
Step 3/3 : RUN tlmgr install scheme-full --verify-repo=none
 ---> Running in 39fc199721e2
TeX Live 2019 is frozen forever and will no
longer be updated.  This happens in preparation for a new release.

If you're interested in helping to pretest the new release (when
pretests are available), please read https://tug.org/texlive/pretest.html.
Otherwise, just wait, and the new release will be ready in due time.
===============================================================================
tlmgr itself needs to be updated.
Please do this via either
  tlmgr update --self
or by getting the latest updater for Unix-ish systems:
  http://mirror.ctan.org/systems/texlive/tlnet/update-tlmgr-latest.sh
and/or Windows systems:
  http://mirror.ctan.org/systems/texlive/tlnet/update-tlmgr-latest.exe
Then continue with other updates as usual.
===============================================================================
tlmgr: Terminating; please see warning above!
tlmgr: package repository ftp://tug.org/historic/systems/texlive/2019/tlnet-final (not verified: gpg unavailable)
Service 'sharelatex' failed to build: The command '/bin/sh -c tlmgr install scheme-full --verify-repo=none' returned a non-zero code: 255
Failed to deploy 'Compose: docker-compose.yaml': `docker-compose` process finished with exit code 1

I did then try:

FROM sharelatex/sharelatex:latest
RUN tlmgr update --self --verify-repo=none
RUN tlmgr install scheme-full --verify-repo=none

which outputed:

Building sharelatex
Step 1/3 : FROM sharelatex/sharelatex:latest
 ---> 0867310d1151
Step 2/3 : RUN tlmgr update --self --verify-repo=none
 ---> Running in 2b65a70fd682

tlmgr: Local TeX Live (2019) is older than remote repository (2020).
Cross release updates are only supported with
  update-tlmgr-latest(.sh/.exe) --update
See https://tug.org/texlive/upgrade.html for details.
Service 'sharelatex' failed to build: The command '/bin/sh -c tlmgr update --self --verify-repo=none' returned a non-zero code: 1
Failed to deploy 'Compose: docker-compose.yaml': `docker-compose` process finished with exit code 1

Which leaded me to the code:

FROM sharelatex/sharelatex:latest
RUN wget http://mirror.ctan.org/systems/texlive/tlnet/update-tlmgr-latest.sh && chmod +x update-tlmgr-latest.sh && ./update-tlmgr-latest.sh
RUN tlmgr install scheme-full --verify-repo=none

And it successfully started installing the packages from scheme-full

NOTE: I don't know if it's fully working as it's still downloading and installing the packages (14 hours left according to the package manager), but I'll update here ASAP