Runc: docker: Error response from daemon: oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:359: container init caused \"invalid argument\"".

[runelabs]

Doing a docker run causes this error with Docker 17.04.0-ce and also 17.05.0-ce-rc3

When running with 17.04.0-ce:

Containers: 52
 Running: 43
 Paused: 0
 Stopped: 9
Images: 790
Server Version: 17.04.0-ce
Storage Driver: btrfs
 Build Version: Btrfs v4.7.3
 Library Version: 101
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins: 
 Volume: local
 Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: 
containerd version: 422e31ce907fd9c3833a38d7b8fdd023e5a76e73
runc version: 9c2d8d184e5da67c95d601382adf14862e4f2228
init version: 949e6fa
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.4.0-1013-aws
Operating System: Ubuntu 17.04
OSType: linux
Architecture: x86_64
CPUs: 16
Total Memory: 120GiB
Name: ip-172-nn-nn-nn
ID: ARRL:TYYM:UI4C:6SDJ:RSBE:3NW4:WYUV:4WOV:MNWC:DSN2:MBMD:TKNO
Docker Root Dir: /srv/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false
WARNING: No swap limit support

runc version 1.0.0-rc2
commit: 9c2d8d184e5da67c95d601382adf14862e4f2228
spec: 1.0.0-rc2-dev

When running with 17.05.0-ce-rc3

Containers: 52
 Running: 4
 Paused: 0
 Stopped: 48
Images: 790
Server Version: 17.05.0-ce-rc3
Storage Driver: btrfs
 Build Version: Btrfs v4.9.1
 Library Version: 102
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 9048e5e50717ea4497b757314bad98ea3763c145
runc version: 9c2d8d184e5da67c95d601382adf14862e4f2228
init version: 949e6fa
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.4.0-1013-aws
Operating System: Ubuntu 17.04
OSType: linux
Architecture: x86_64
CPUs: 16
Total Memory: 120GiB
Name: ip-172-nn-nn-nn
ID: ARRL:TYYM:UI4C:6SDJ:RSBE:3NW4:WYUV:4WOV:MNWC:DSN2:MBMD:TKNO
Docker Root Dir: /srv/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false
WARNING: No swap limit support

runc version 1.0.0-rc2
commit: 9c2d8d184e5da67c95d601382adf14862e4f2228
spec: 1.0.0-rc2-dev

The host is running Ubuntu Zesty on AWS, some information:

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 17.04
Release:        17.04
Codename:       zesty

Linux ip-172-nn-nn-nn 4.4.0-1013-aws #22-Ubuntu SMP Fri Mar 31 15:41:31 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

dpkg -l system\*
un  system-log-daemon  <none>         <none>  (no description available)
ii  systemd            232-21ubuntu3  amd64   system and service manager
un  systemd-container  <none>         <none>  (no description available)
un  systemd-shim       <none>         <none>  (no description available)
ii  systemd-sysv       232-21ubuntu3  amd64   system and service manager - SysV links
un  systemd-ui         <none>         <none>  (no description available)

Failing command:

docker run -d --name nnnnnnnn-ci-nnnnnn-cnnnnn-448-check-if-already-nnnnnn-nn-when-loading-page --hostname nnnnnnnn-ci-nnnnnn-cnnnnn-448-check-if-already-nnnnnn-nn-when-loading-page --network nnnnnnn_nnnnnn --log-driver "gelf" --log-opt "gelf-address=udp://nnnnnn.nnnn.nnnnnn.nn:12201" --log-opt "tag={{.FullID}}" --expose 9000 nnnnnnnnn/nnnnnnnn:nnnnnn-nnnn-448-check-if-already-nnnnnn-nn-when-loading-page

This only happens on some of the containers, while others run fine. They all build fine with any Docker version, but some will not run.

The images are based on the latest node:6 (Debian Jessie) and run NodeJS with HapiJS delivering a simple Webpack bundle. The failing containers contain small variations on the served bundle, but otherwise the failing containers are the same as the running ones. Around 10% of these containers fail with this error message. The host is an AWS EC2.

[runelabs]

This was caused when exceeding getconf HOST_NAME_MAX - which is 64 - i.e --hostname argument length.

[buckyster]

This was caused when exceeding getconf HOST_NAME_MAX - which is 64 - i.e --hostname argument length.

Sheesh, I ran into this too. Talk about a vague error message!

[crosbymichael]

I created a PR to set a better error message for some of these syscalls that happen during container init and especially for hostname.

https://github.com/opencontainers/runc/pull/1796/files#diff-50f0c740b61fd97d8cea9da8eedc18ccR93

Hopefully this helps in the future @buckyster