repo2docker hangs on local install

[psychemedia]

Running latest repo2docker fresh from pypi:

FROM buildpack-deps:artful

# Set up locales properly
RUN apt-get update && \
    apt-get install --yes --no-install-recommends locales && \
    apt-get purge && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

RUN echo "en_US.UTF-8 UTF-8" > /etc/locale.gen && \
    locale-gen

ENV LC_ALL en_US.UTF-8
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US.UTF-8

# Use bash as default shell, rather than sh
ENV SHELL /bin/bash

# Set up user
ARG NB_USER
ARG NB_UID
ENV USER ${NB_USER}
ENV HOME /home/${NB_USER}

RUN adduser --disabled-password \
    --gecos "Default user" \
    --uid ${NB_UID} \
    ${NB_USER}
WORKDIR ${HOME}

RUN apt-get update && \
    apt-get install --yes --no-install-recommends \
       less \
       npm \
       && apt-get purge && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

RUN apt-get update && \
    apt-get install --yes \
       python3 \
       python3-dev \
       python3-venv \
       && apt-get purge && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*
EXPOSE 8888

# Almost all environment variables
ENV APP_BASE /srv
ENV VENV_PATH ${APP_BASE}/venv
ENV NB_PYTHON_PREFIX ${VENV_PATH}
# Special case PATH
ENV PATH ${VENV_PATH}/bin:${PATH}
# If scripts required during build are present, copy them

COPY python/requirements.frozen.txt /tmp/requirements.frozen.txt
RUN mkdir -p ${VENV_PATH} && \
chown -R ${NB_USER}:${NB_USER} ${VENV_PATH}

USER ${NB_USER}
RUN python3 -m venv ${VENV_PATH}

RUN pip install --no-cache-dir -r /tmp/requirements.frozen.txt && \
jupyter nbextension enable --py widgetsnbextension --sys-prefix && \
jupyter serverextension enable --py jupyterlab --sys-prefix && \
jupyter serverextension enable nteract_on_jupyter --sys-prefix



# Copy and chown stuff. This doubles the size of the repo, because
# you can't actually copy as USER, only as root! Thanks, Docker!
USER root
COPY src/ ${HOME}
RUN chown -R ${NB_USER}:${NB_USER} ${HOME}

# Run assemble scripts! These will actually build the specification
# in the repository into the image.
RUN apt-get update && \
apt-get install --yes --no-install-recommends pandoc texlive-latex-base texlive-latex-recommended texlive-science texlive-latex-extra texlive-fonts-recommended dvipng ghostscript && \
apt-get purge && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*

USER ${NB_USER}
RUN pip3 install --no-cache-dir -r "requirements.txt"


# Container image Labels!
# Put these at the end, since we don't want to rebuild everything
# when these change! Did I mention I hate Dockerfile cache semantics?


# We always want containers to run as non-root
USER ${NB_USER}

RUN ./postBuild

Using PythonBuildPack builder
Step 1/33 : FROM buildpack-deps:artful
 ---> 15aabadeaee6
Step 2/33 : RUN apt-get update &&     apt-get install --yes --no-install-recommends locales &&     apt-get purge &&     apt-get clean &&     rm -rf /var/lib/apt/lists/*
 ---> Using cache
 ---> a503cac6c9ac
Step 3/33 : RUN echo "en_US.UTF-8 UTF-8" > /etc/locale.gen &&     locale-gen
 ---> Using cache
 ---> 2f62ac8d042a
Step 4/33 : ENV LC_ALL en_US.UTF-8
 ---> Using cache
 ---> 5246ca660498
Step 5/33 : ENV LANG en_US.UTF-8
 ---> Using cache
 ---> 46d5ee905d85
Step 6/33 : ENV LANGUAGE en_US.UTF-8
 ---> Using cache
 ---> 25ca381c1ccb
Step 7/33 : ENV SHELL /bin/bash
 ---> Using cache
 ---> 2f3e4d95e5a6
Step 8/33 : ARG NB_USER
 ---> Using cache
 ---> d0da0c76273f
Step 9/33 : ARG NB_UID
 ---> Using cache
 ---> 23f9d9d773c6
Step 10/33 : ENV USER ${NB_USER}
 ---> Using cache
 ---> 996af037e629
Step 11/33 : ENV HOME /home/${NB_USER}
 ---> Using cache
 ---> 09634520cc8c
Step 12/33 : RUN adduser --disabled-password     --gecos "Default user"     --uid ${NB_UID}     ${NB_USER}
 ---> Running in 14bb574a0353
Adding user `ajh59' ...
Adding new group `ajh59' (494230618) ...
Adding new user `ajh59' (494230618) with group `ajh59' ...
Creating home directory `/home/ajh59' ...
Copying files from `/etc/skel' ...

and then it just hangs. It also seems to break my docker install by using up all memory, though I can't see where. I wonder if the copy could be infinitely stuck somehow?

[psychemedia]

Maybe this?

https://access.redhat.com/solutions/59174

[minrk]

Hrm. Does this still happen with master? What's your docker info?

[psychemedia]

Yep... I destroyed everything - docker system prune -a, uninstalled docker edge and put in the latest docker stable release with 4GB memory, upgraded to repo2docker from master and same thing happened; when the skel copy got out of hand causing laptop fan to go into overdrive, I ctrl-C'd out and tried again which gave me an out of memory error.

[minrk]

What's your host OS? And docker info output?

[psychemedia]

Mac 10.13.3
Docker version 17.12.0-ce, build c97c6d6

 docker info
Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 0
Server Version: 17.12.0-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 89623f28b87a6004d4b785663257362d1658a729
runc version: b2567b37d7b75eb4cf325b77297b140ea686ce8f
init version: 949e6fa
Security Options:
 seccomp
  Profile: default
Kernel Version: 4.9.60-linuxkit-aufs
Operating System: Docker for Mac
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 4.095GiB
Name: linuxkit-025000000001
ID: [REMOVED]
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 21
 Goroutines: 39
 System Time: 2018-02-09T12:49:59.552984963Z
 EventsListeners: 2
HTTP Proxy: docker.for.mac.http.internal:3128
HTTPS Proxy: docker.for.mac.http.internal:3129
Registry: https://index.docker.io/v1/
Labels:
Experimental: true
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

[minrk]

That's very strange indeed. My environment is quite similar to yours (I am on Edge like you were):

diff yourinfo myinfo

1d0
<  docker info
6,7c5,6
< Images: 0
< Server Version: 17.12.0-ce
---
> Images: 202
> Server Version: 18.02.0-ce-rc2
22,23c21,22
< containerd version: 89623f28b87a6004d4b785663257362d1658a729
< runc version: b2567b37d7b75eb4cf325b77297b140ea686ce8f
---
> containerd version: 9b55aab90508bd389d7654c4baf173a981477d55
> runc version: 9f9c96235cc97674e935002fc3d78361b696a69e
28c27
< Kernel Version: 4.9.60-linuxkit-aufs
---
> Kernel Version: 4.9.75-linuxkit-aufs
33c32
< Total Memory: 4.095GiB
---
> Total Memory: 1.952GiB
39,41c38,40
<  File Descriptors: 21
<  Goroutines: 39
<  System Time: 2018-02-09T12:49:59.552984963Z
---
>  File Descriptors: 22
>  Goroutines: 41
>  System Time: 2018-02-09T15:32:18.53410775Z
50a50
> 

What repo are you trying to build and what exact command are you running? If it's a local directory, how big is it?

[psychemedia]

Local directory with an apt file to install pandoc and some Latex bits. Nothing I haven't run before... It was just a rebuild with a new notebook, which I've since got rid of.

[minrk]

what's in the apt.txt? I wonder if that's relevant.

[psychemedia]

I'm pretty sure I also tried with config files and just a dummy notebook and had the same issue.
Does the copy over pull in hidden files? Maybe there's something in a .git it doesn't like?

[minrk]

What's your id -u?

[minrk]

Also just in case, what is python -c 'import os; print(os.geteuid())' in case it's different for some reason.

[psychemedia]

python -c 'import os; print(os.geteuid()) returns same as id -u

[psychemedia]

Looking at the log:

Adding user `ajh59' ...
Adding new group `ajh59' (494230618) ...
Adding new user `ajh59' (494230618) with group `ajh59' ...
Creating home directory `/home/ajh59' ...
Copying files from `/etc/skel' ...

I'm now wondering: is this step trying to copy all my files - ~/ajh59/* rather than just the files in the current directory?

FWIW, there is a space in the path name of the folder I'm building in... I've noticed that with some other things spaces on Macs cause things to break. Will check to see if a simple path makes a difference... (It doesn't...)

[willingc]

@psychemedia Are you still having issues with this?

[psychemedia]

Yes... I don't know if it's something to do with my mac environment. I had an o/s update and different envts seem to be all over the place...

[willingc]

@psychemedia I'll give it a shot on my mac. Thanks.

[choldgraf]

friendly follow-up on this one :-) @psychemedia or @willingc , is the bug still reproducible?

[psychemedia]

I've just had my mac o/s reinstalled. Will give it another go tomorrow

[psychemedia]

Same behaviour - just gets stuck on the /etc/skel copy step. Could there be anything in my environment that is messing things up? Is there a way I can easily log what files it's copying to terminal or a host logfile to see if the copy is working correctly? It should just be copying over files from the cloned repo or local directory, right?

[psychemedia]

Realising I have no idea how repo2docker works at all. For instance, why are my user details on host copied asa new use into the image?

 ---> Running in 4a81344302fc
Adding user `ajh59' ...
Adding new group `ajh59' (494230618) ...
Adding new user `ajh59' (494230618) with group `ajh59' ...
Creating home directory `/home/ajh59' ...

[willingc]

Hi @psychemedia,

Can you try this workflow? This will create a virtual environment, clone this repo and install from master, and then run repo2docker locally to build and serve my willingc/ThinkDSP repo.

python3 -m venv myrepoenv
source myrepoenv/bin/activate
mkdir r2d2test
cd r2d2test
git clone https://github.com/jupyter/repo2docker.git
cd repo2docker/
pip install -e .

repo2docker https://github.com/willingc/ThinkDSP

For comparison, here's the pastebin from the run on my Mac: https://pastebin.com/qahXKFR0

Please let me know how it goes. Thanks.

[psychemedia]

As before.... everything runs fine then it hangs on Copying files from /etc/skel ...

How long is that step supposed to take? My machine fan goes in to meltdown for a bit and then it just sits on that step... 10 mins? 15? Is there an easy way to log what it's doing around there?

If I ctrl-c out (I appreciate the break might be hit and miss) I get:

^CTraceback (most recent call last):
  File "/Users/ajh59/myrepoenv/bin/repo2docker", line 11, in <module>
    load_entry_point('jupyter-repo2docker', 'console_scripts', 'repo2docker')()
  File "/Users/ajh59/r2d2test/repo2docker/repo2docker/__main__.py", line 7, in main
    f.start()
  File "/Users/ajh59/r2d2test/repo2docker/repo2docker/app.py", line 616, in start
    self.build_memory_limit, build_args):
  File "/Users/ajh59/r2d2test/repo2docker/repo2docker/buildpacks/base.py", line 375, in build
    container_limits=limits
  File "/Users/ajh59/myrepoenv/lib/python3.6/site-packages/docker/api/client.py", line 301, in _stream_helper
    for chunk in json_stream(self._stream_helper(response, False)):
  File "/Users/ajh59/myrepoenv/lib/python3.6/site-packages/docker/utils/json_stream.py", line 66, in split_buffer
    for data in stream_as_text(stream):
  File "/Users/ajh59/myrepoenv/lib/python3.6/site-packages/docker/utils/json_stream.py", line 22, in stream_as_text
    for data in stream:
  File "/Users/ajh59/myrepoenv/lib/python3.6/site-packages/docker/api/client.py", line 307, in _stream_helper
    data = reader.read(1)
  File "/Users/ajh59/myrepoenv/lib/python3.6/site-packages/urllib3/response.py", line 384, in read
    data = self._fp.read(amt)
  File "/Users/ajh59/anaconda3/lib/python3.6/http/client.py", line 449, in read
    n = self.readinto(b)
  File "/Users/ajh59/anaconda3/lib/python3.6/http/client.py", line 483, in readinto
    return self._readinto_chunked(b)
  File "/Users/ajh59/anaconda3/lib/python3.6/http/client.py", line 578, in _readinto_chunked
    chunk_left = self._get_chunk_left()
  File "/Users/ajh59/anaconda3/lib/python3.6/http/client.py", line 546, in _get_chunk_left
    chunk_left = self._read_next_chunk_size()
  File "/Users/ajh59/anaconda3/lib/python3.6/http/client.py", line 506, in _read_next_chunk_size
    line = self.fp.readline(_MAXLINE + 1)
  File "/Users/ajh59/anaconda3/lib/python3.6/socket.py", line 586, in readinto
    return self._sock.recv_into(b)
KeyboardInterrupt
(myrepoenv) mcl267208:repo2docker ajh59$ 

[betatim]

Could you post the exact command you run and from which directory you run it?

I am wondering if we somehow pick up the wrong (and far too large) build context. Triggered by you mentioning that the path has a space in it. Depending on how you quote that it might or might not get passed on properly. This is a bit of a shot in the dark but out of good ideas.

[psychemedia]

@betatim I followed https://github.com/jupyter/repo2docker/issues/223#issuecomment-384289185 exactly. No commands more or less, other than ctrl-c to break from the hang.

Thinks... hang on, I'll try and run it again in a path I've checked that guarantees no spaces...
... ah, I did, the above code was run in: /Users/ajh59/r2d2test/repo2docker

[psychemedia]

printenv shows the following are set:

TERM_PROGRAM
SHELL
TERM
TMPDIR
Apple_PubSub_Socket_Render
TERM_PROGRAM_VERSION
TERM_SESSION_ID
USER
SSH_AUTH_SOCK
VIRTUAL_ENV
PATH
PWD
LANG
XPC_FLAGS
PS1
XPC_SERVICE_NAME
SHLVL
HOME
LOGNAME
DISPLAY
SECURITYSESSIONID
_
OLDPWD

[psychemedia]

Could there be a Jupyter setting I've mangled somewhere? I've done reinstalls of packages but not cleared out .jupyter config files. Is there a complete list of Jupyter config file locations and I could poke through those - or just see if mv-ing them out of the way helps.

[willingc]

@psychemedia The whole process on my mac took about 10ish minutes.

One difference that I had was I have TERMSESSIONID but no SECURITYSESSIONID.

A few questions:

• do you have the firewall on or off?
• if on does it permit docker to connect?

FYI. There were changes with High Sierra related to SKEL but the [docs]
(https://support.apple.com/en-us/HT208019) are slim.

I'm going to rerun with debug mode enabled to see if that sheds any additional light.

[willingc]

Well, I don't think the firewall is it. I enabled mine and it still works. Does the account that you work from on the Mac have admin privileges. If you could pastebin the entire output from when you run the above repo2docker example, we may see something that we didn't before. Thanks @psychemedia

[psychemedia]

Paste here (good for just a week): https://pastebin.com/d6JsjxnJ

No firewall; Sophos running in part - for now docker is giving me an Error writing to output file - write (28: No space left on device) Error writing to file - write (28: No space left on device) (as I'd experienced previously), which requires things like:

docker system prune -a
docker volume rm `docker volume ls -q -f dangling=true` 
rm ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2

and then a Docker restart in order to reset things.

Just tried running repo2docker under sudo, but that gives:

Step 12/39 : RUN adduser --disabled-password     --gecos "Default user"     --uid ${NB_UID}     ${NB_USER}
 ---> Running in b0d0279cd236
adduser: The user `root' already exists.

and fails.

As for privileges - yep, my account is set up to run with pretty full privileges.

I don't know if there's a High Sierra permissions thing I need to set somewhere following o/s update? I know each time I do update I get a slew of permission lock downs on installing things that I have to unset.

The way that Docker.qcow2 grows, I'm pretty sure that the copy command is just trying to copy a tonne of stuff into the container? And my ~/ path leads to a whole tonne of files. If I get a chance tomorrow, I'll try to run things in a new user account and see if that helps.

[DominikGlodzik]

I get the same problem on MacOS 10.13.4.

Any advice from those that successfully solved it, highly appreciated.

[willingc]

@DominikGlodzik What output or errors do you get if you run through the steps in this example: https://github.com/jupyter/repo2docker/issues/223#issuecomment-384289185

[DominikGlodzik]

Thank you for your response Carol.

I tried the example you suggested, and the process hangs (for an hour now) at:
"Step 4/33 : RUN echo "en_US.UTF-8 UTF-8" > /etc/locale.gen && locale-gen
---> Running in 4d6870ce7cc4
Generating locales (this might take a while)...
en_US.UTF-8... done
Generation complete.
"

I appreciate any suggestions.

[DominikGlodzik]

I am still struggling with this problem. Any ideas how to use repo2docker on my mac are welcome :)

[psychemedia]

I'm still stuck on the issue too, gone to a simpler method of using azure.notebooks configs for Jupyter notebook envts for now, though it's much less flexible

[willingc]

@psychemedia Are you seeing the exact same output as @DominikGlodzik? One of the difficulties is that we can't seem to replicate what you are seeing. Any insights on how your macs may be configured differently than standard would be helpful. Thanks.

[psychemedia]

@willingc My issue is the hang on Copying files from/etc/skel' ...` which I'm pretty sure is a result of the copy trying to copy everything from homedir, given the way the size of the container explodes.

[DominikGlodzik]

Hello again,

maybe it helps, but I asked my colleagues to run your example and they get an error similar to mine. One is on MacOS 10.11.6, another 10.13.4. We are still all in one organisation, so maybe there are issues with how our laptops are set up. I will also try on my home computer.

Dominik

[psychemedia]

I am running on a company machine, but with Admin privileges. Some bits of the account are locked out though - eg changing the password requires connecting to company network and inheriting password changes that way (a process that itself seems a bit hit and miss...).

repo2docker had worked for me prior to a combined update of Mac o/s and repo2docker so it was hard to identify which update caused the issue.

[willingc]

Thanks @DominikGlodzik for the additional detail. This leads me to believe it is not strictly an operating system version but some sort of configuration difference.

@psychemedia Thanks for the input. Just curious, does the old version of repo2docker work on your current Mac OS?

[psychemedia]

I did try rolling back repo2docker several times but got errors because of
not-found Linux packages (there had been an LTS update that seemed to break
one of the base containers?)

What version of of repo2docker should I try? And is there a repo I can try
to pull into it that is know to work with that version?

On 29 May 2018 at 17:55, Carol Willing [e-mail] wrote:

Thanks @DominikGlodzik https://github.com/DominikGlodzik for the
additional detail. This leads me to believe it is not strictly an operating
system version but some sort of configuration difference.

@psychemedia https://github.com/psychemedia Thanks for the input. Just
curious, does the old version of repo2docker work on your current Mac OS?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/jupyter/repo2docker/issues/223#issuecomment-392852064,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAFELFRtooGfF3VuV2bm83-WCXFk46W2ks5t3X2ZgaJpZM4R9c_2
.

[juanesarango]

So as @DominikGlodzik and @psychemedia mentioned, the laptops are from an organization. So the error is caused trying to create the users, as it might by trying to copy existing users configuration (that is not accesible to employees to change) inside the image.

I'm not fully aware of the internals of adduser but I just changed some options of the command, and I managed to make it work.

Changing from:
adduser --disabled-password --gecos "Default user" --uid ${NB_UID} ${NB_USER}

to a simple form, just:
useradd ${NB_USER}

I made a PR #323 just in case you agree, or someone wants to try the fork for testing.

[betatim]

@psychemedia and @juanesarango could you try if providing --uid 1000 as command-line option when you run repo2docker fixes the problem and tell us what your user ID is (run id to find out)?

By default repo2docker will use the UID of your local user when it adds the user to the image.

The fact that the patch in #323 fixes the problem suggests that UID 1000 should work. It is a bit of a mystery why it does not work with your local user's ID.

[psychemedia]

Ok - that seems to work with --user-id 1000:

Step 13/34 : RUN adduser --disabled-password     --gecos "Default user"     --uid ${NB_UID}     ${NB_USER}
 ---> Running in 7c5fd8f2ef04
Adding user `ajh59' ...
Adding new group `ajh59' (1000) ...
Adding new user `ajh59' (1000) with group `ajh59' ...
Creating home directory `/home/ajh59' ...
Copying files from `/etc/skel' ...
Removing intermediate container 7c5fd8f2ef04

Would it make sense to also add to docs a sensible set of switches that means users don't leak their own user and group names (ajh59 in the example above)?

Thanks... :-)

[betatim]

From a previous comment of yours: is 494230618 the UID you get if you don't set it to 1000 by hand?

Would it make sense to also add to docs a sensible set of switches that means users don't leak their own user and group names (ajh59 in the example above)?

What exactly do you mean? There is --user-name to set the username to what ever you want. We pick the name of the user executing repo2docker to try and minimise the chances of filesystem permission errors.

[psychemedia]

My uid from id -u is currently 494230618. My comment was along lines of: if I am creating a docker container from a local folder on my own computer, I might not want any reference at all to my user group/id being used to create containers in the repo2docker context. Just because I am happy to share contents of ./ with third parties doesn't mean it's relevant for me to share my user id/group. That sort of leakage is the sort of thing that can also cause grief with org admins ("why are you sharing your user id in public etc etc?")

[betatim]

Sounds reasonable. --user-name and --user-id are for those cases. I think we should keep the default tuned for people who are using repo2docker locally to run images, and not tuned towards people who are using it as a docker image builder.

[juanesarango]

@betatim yes that definitely worked. Thanks for your help.

repo2docker --user-id `id -u` https://github.com/willingc/ThinkDSP
works as you would expect to behave, to have your uid by default.

I also agree with @psychemedia that there should be a docs warning, or a flag to opt-out from sharing your user-id/group in the docker images you build, and may want to share. Maybe some flag such as --no-user.

If you agree, I'm happy to help to implement.

[betatim]

Maybe update the help text you see when running repo2docker --help to mirror more closely https://github.com/jupyter/repo2docker/blob/d4c9c8858a4d880dedeb79ab4b4488327e9bc087/repo2docker/app.py#L115-L154

I am a confused why repo2docker --user-idid -uhttps://github.com/willingc/ThinkDSP works for you but repo2docker https://github.com/willingc/ThinkDSP doesn't because the default behaviour is to pick the UID of the user running repo2docker which should be the same as id -u.

[juanesarango]

@betatim sorry for the confusion. You're right:

This is the same as the default behavior that doesn't work for us.
repo2docker --user-id `id -u` https://github.com/willingc/ThinkDSP

Passing a uid value of 1000 like this, does work for us.
repo2docker --user-id 1000 https://github.com/willingc/ThinkDSP

[betatim]

The next thing I'd try and establish is: why does changing the UID fix the problem? Is the default UID too big (something special about very large UIDs?) or would literally any other UID do the job? What is in the /etc/skel that is being copied?

[manics]

I can easily reproduce the problem outside of repo2docker if I create a Dockerfile consisting of just these lines from the start of repo2docker/buildpacks/base.py:
https://github.com/jupyter/repo2docker/blob/0.9.0/repo2docker/buildpacks/base.py#L15-L46

• Works:
  cat Dockerfile | docker build --build-arg NB_USER=jovyan --build-arg NB_UID=1000 -
• Hangs on /etc/skel and eat up all disk space:
  cat Dockerfile | docker build --build-arg NB_USER=jovyan --build-arg NB_UID=200000000 -

In the case of docker-for-mac docker system prune does not clean up the space, the only solution I've found is to delete the Docker VM disk image rm ~/Library/Containers/com.docker.docker/Data/vms/0/Docker.raw 🙁

Interestingly as you increase NB_UID, in steps (e.g. start from NB_UID=1000000 and increment by 1000000 each time) the time taken for the Copying files from /etc/skel ... increases.

[manics]

I might've stumbled upon the cause: https://github.com/moby/moby/issues/5419

It sounds like it depends on the capabilities of your Docker host, and whether sparse files are supported:

root@a8e6431173ab:/# adduser --gecos comment --disabled-password --uid 4294967294 test
Adding user `test' ...
Adding new group `test' (4294967294) ...
Adding new user `test' (4294967294) with group `test' ...
Creating home directory `/home/test' ...
Copying files from `/etc/skel' ...

root@a8e6431173ab:/# ls -lh /var/log/lastlog 
-rw-rw-r--. 1 root utmp 1.2T Oct  5 23:20 /var/log/lastlog

root@a8e6431173ab:/# du -h /var/log/lastlog 
40K /var/log/lastlog

In this case if your system doesn't support sparse files adduser will attempt to create a 1.2 TB file!

I've got a PR but the tests are failing: https://github.com/jupyter/repo2docker/pull/804