Docker-stacks: Using NB_UID produces error due to unwriteable home directory

[harschware]

I may be doing this all wrong, but I'm trying to run the all-spark-notebook with a custom user and I get:

[root@sandbox sparkall] docker run --name sparkall --user root -e NB_USER=centos -e NB_GROUP=centos -e NB_UID=$(id -u centos) -e NB_GID=$(id -g centos) -p 8888:8888 -v work:/home/centos/work jupyter/all-spark-notebook Set username to: centos Setting CWD to /home/centos/ Set centos GID to: 1000 Execute the command: jupyter notebook /opt/conda/lib/python3.6/site-packages/IPython/paths.py:68: UserWarning: IPython parent '/home/centos' is not a writable location, using a temp directory. " using a temp directory.".format(parent)) Traceback (most recent call last): File "/opt/conda/bin/jupyter-notebook", line 6, in <module> sys.exit(notebook.notebookapp.main()) File "/opt/conda/lib/python3.6/site-packages/jupyter_core/application.py", line 267, in launch_instance return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs) File "/opt/conda/lib/python3.6/site-packages/traitlets/config/application.py", line 657, in launch_instance app.initialize(argv) File "<decorator-gen-7>", line 2, in initialize File "/opt/conda/lib/python3.6/site-packages/traitlets/config/application.py", line 87, in catch_config_error return method(app, *args, **kwargs) File "/opt/conda/lib/python3.6/site-packages/notebook/notebookapp.py", line 1290, in initialize super(NotebookApp, self).initialize(argv) File "<decorator-gen-6>", line 2, in initialize File "/opt/conda/lib/python3.6/site-packages/traitlets/config/application.py", line 87, in catch_config_error return method(app, *args, **kwargs) File "/opt/conda/lib/python3.6/site-packages/jupyter_core/application.py", line 243, in initialize self.migrate_config() File "/opt/conda/lib/python3.6/site-packages/jupyter_core/application.py", line 169, in migrate_config migrate() File "/opt/conda/lib/python3.6/site-packages/jupyter_core/migrate.py", line 240, in migrate ensure_dir_exists(env['jupyter_config']) File "/opt/conda/lib/python3.6/site-packages/ipython_genutils/path.py", line 167, in ensure_dir_exists os.makedirs(path, mode=mode) File "/opt/conda/lib/python3.6/os.py", line 220, in makedirs mkdir(name, mode) PermissionError: [Errno 13] Permission denied: '/home/centos/.jupyter'

I think the problem might be here: https://github.com/jupyter/docker-stacks/blob/master/base-notebook/start.sh#L20 since the home directory was moved but then permissions were not adjusted?

[parente]

Root cause: The -v parameter to docker creates a volume mount at container start time. Because /home/centos does not yet exist at the time, docker creates all of the folders leading up to the mount point of /home/centos/work and makes them root owned. Later, the mv command in start.sh never executes because /home/centos already exists due to the mount point creation.

[dsevilla]

Related to this, when you change the UID and GID of the jovyan user using an environment variable, actually the user is modified to hold those UID and GID, but permissions over /home/jovyan and /opt/conda are not granted again after the UID/GID change, so you cannot, for example, install packages with pip anymore unless you specify the GRANT_SUDO and use sudo for the installation of the packages.

Unsure if I should open a new bug report.

[harschware]

Just a status update that I tried the command posted above on the latest jupyter/all-spark-notebook to see if anything may have changed. same error. I just wanted to see if any commits since the issue was filed had any effects.

[GrahamDumpleton]

@dsevilla Your issue may have been addressed by https://github.com/jupyter/docker-stacks/pull/555. Try again.

[parente]

It's likely that PR #555 resolved this issue so I'm closing it out. If the problem persists, feel free to open a new issue or PR.

[harschware]

Unfortunately this bug was not resolved with PR#555

[GrahamDumpleton]

@harschware What permissions does the directory /home/centos have. Override the start up command when using docker run and then use ls -las on the directory.

[harschware]

@GrahamDumpleton I've responded to you by sharing the output in the open PR #598. Thanks.