Podman: build hangs/crashes when useradd/usermod/groupmod with large UID

[aduzsardi]

kind bug

Description
When an instruction in dockerfile contains a large UID/GID the build will hang forever. This is similar to the docker issue that has been reported , the good thing is however is that podman/buildah wont fill up the storage/ram/swap or be cpu intensive on the host , or it will do this very slowly

Steps to reproduce the issue:

FROM nginx

ENV DEFAULT_UID=1023415920
# 1023415920

RUN usermod -u $DEFAULT_UID nginx && \
    groupmod -g $DEFAULT_UID nginx

podman build -t testimg .

Host OS:

Distributor ID: Ubuntu
Description:    Ubuntu 18.04.1 LTS
Release:    18.04
Codename:   bionic

Output of podman version:

Version:       0.10.2-dev
Go Version:    go1.10.4
OS/Arch:       linux/amd64

Output of podman info:

host:
  BuildahVersion: 1.5-dev
  Conmon:
    package: 'cri-o-1.12: /usr/lib/crio/bin/conmon'
    path: /usr/lib/crio/bin/conmon
    version: 'conmon version 1.12.1-dev, commit: '
  Distribution:
    distribution: ubuntu
    version: "18.04"
  MemFree: 86679552
  MemTotal: 1033150464
  OCIRuntime:
    package: 'runc: /usr/sbin/runc'
    path: /usr/sbin/runc
    version: 'runc version spec: 1.0.1'
  SwapFree: 0
  SwapTotal: 0
  arch: amd64
  cpus: 2
  hostname: ubuntu-bionic
  kernel: 4.15.0-34-generic
  os: linux
  uptime: 1h 40m 8.6s (Approximately 0.04 days)
insecure registries:
  registries: []
registries:
  registries:
  - docker.io
  - registry.fedoraproject.org
  - quay.io
  - registry.access.redhat.com
  - registry.centos.org
store:
  ContainerStore:
    number: 2
  GraphDriverName: overlay
  GraphOptions:
  - overlay.override_kernel_check=true
  GraphRoot: /var/lib/containers/storage
  GraphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
  ImageStore:
    number: 2
  RunRoot: /var/run/containers/storage

[rhatdan]

This is a known issue with useradd in Golang apps, could you switch to using
useradd -l
in your scripts, this will prevent the creation of sparse files, which causes huge files to get created.

[aduzsardi]

Sure , i could probably do that with useradd , however usermod / groupmod doesn't have an option for _--no-log-init_
I could probably just run a sed search & replace in /etc/passwd or /etc/group for those cases.

Do you know if any of these has been brought up in Golang issues/bugs , and if it can be fixed ?
Thank you!

[rhatdan]

Yes this is a well known issue, and has been worked on for years. Sadly it has not been fixed yet.

[rhatdan]

https://github.com/golang/go/issues/22735
https://stackoverflow.com/questions/43035271/sparse-files-are-huge-with-io-copy

[aduzsardi]

doesn't look like this is going to be fixed very soon in golang :)

[rhatdan]

@vbatts Any update on ths?

[mheon]

@rhatdan Worth noting this down in the troubleshooting guide, maybe?

[vbatts]

@rhatdan update on golang + sparse file support in archive/tar? none

[rhatdan]

@vbatts kind of what I thought, thanks.

[rhatdan]

I am going to close this, since it really can not be fixed by libpod/podman until golang fixes it, We have documented the issue.

[niklas88]

Just wanted to add that this also breaks podman build for projects that rely on working sparse file support in the build phase such as https://github.com/ad-freiburg/QLever

We use sparse files inside our database index (for very large on-disk sorts) and it's of course used in the tests which execute during docker build. I think this issue should still be open as it does break docker compatibility.

[mheon]

What Docker version are you running? Would be really interesting if they've resolved this, since this is an inherent Go limitation and both projects are using Go.

[niklas88]

@mheon uhh, I think this might have been another issue. QLever builds fine with podman now on my up to date Arch Linux and we do in fact not use a large sparse file during build (just a small one).