Docker-stacks: Downloaded packages doesn't remain installed after container update

[schizophrene]

What docker image you are using?

jupyter/all-spark-notebook:latest

What complete docker command do you run to launch the container (omitting sensitive values)?

I launch containers from OMV GUI, so it would be something like docker run -it --rm -p 58888:8888 jupyter/all-spark-notebook:latest with these binds :

• "/docker/jupyter/config:/home/jovyan/.jupyter:rw"
• "/docker/jupyter/work:/home/jovyan/work:rw"

What steps do you take once the container is running to reproduce the issue?

1. Run the container
2. Run Watchtower or Ouroboros containers, that auto-update containers when an update is available
3. Dowload some packages via pip, (ipympl or mysql-connectors, for example)
4. Wait for an auto-update of Jupyter container

What do you expect to happen?

Downloaded packages remain installed.

What actually happens?

Downloaded packages do not stay installed.

I'm not the most comfortable with Docker, although I'm doing pretty well.
I did not find where packages downloaded by users are installed, to try to create a bind.

At the moment, Ouroboros is set to ignore Jupyter updates, and I maintain a list of downloaded packages for re-download if I update manually.

Does anyone have an idea, or a solution?

Greetings

[ericdill]

First, some definitions. A dockerfile is used as a textual specification of what to put into your docker image. A docker image is the result of "building" a dockerfile. A docker container is one running instance of an image. When you use Watchtower or Ouroboros you are pulling an immutable image from a docker registry somewhere (likely the one here: https://hub.docker.com/r/jupyter/all-spark-notebook).

When you run a docker image it becomes a running container. Anything you do inside of that container is transient and will not be saved.

To recap: Dockerfile becomes an image when it is built using docker build. An image becomes a container at runtime by using docker run.

If you want to add your own libraries to the docker images that you get from docker-stacks then you need to make your own Dockerfile that starts from, in your case, jupyter/all-spark-notebook:latest and then you need to layer in your pip packages (ipympl or mysql-connectors).
Once you have your Dockerfile, you'll need to docker build it and docker push it somewhere so that you can docker run it in your infrastructure.

So, to make it a little more concrete:

First, you'll create a dockerfile starting from the all-spark-notebook that docker-stacks provides. Give it a name in its own directory. Something like:my_all-spark-notebook.Dockerfile. Its contents should be this:

FROM jupyter/all-spark-notebook:latest

RUN pip install ipympl mysql-connectors

Then you'd need to build it:
docker build -t schizophrene/all-spark-notebook /path/to/dockerfile/directory

And then you'd need to push it (or something like this, I'm not sure my syntax is 100% correct)
docker push schizophrene/all-spark-notebook

Once you've pushed your image up to docker hub then you can run it in your infrastructure with Watchtower or Ouroboros.

Hopefully this helps!

[schizophrene]

Thanks @ericdill for these explanations.

I know that Docker offers a lot of functionalities that I doesn't use for now, and that's a lack in my knowledges.
I'm used to use OpenMediaVault Docer GUI to launch containers, and I've never created or custom any container. But I will.

I close this "issue", I understand that I need to work a little more ;)