Moby: 'docker save' produces unrunnable images [exec: "/bin/bash": stat /bin/bash: no such file or directory]

[nisanharamati]

Images imported from files created with docker save <image> cannot be run. Instead they return the error

Error response from daemon: Cannot start container 2862b...: exec: "/bin/bash": stat /bin/bash: no such file or directory

build -> run -> export -> import -> run works just fine, though.

Possibly related to #6465 and #5157.

To reproduce:

Dockerfile

FROM ubuntu:trusty
MAINTAINER nisan haramati [email protected]

# simple image to demonstrate corruption in 'docker save IMAGE'
# desired behaviour:
#   create image using docker build
#   save to a tar using docker save, then gzip --best
#   run built image in interactive with tty, execute /bin/bash
#   export that image to tar, then gzip --best
#   cat corrupt.save.tar.gz | docker import - corrupt:save
#   cat corrupt.export.tar.gz | docker import - corrupt:export

# lets actually install something, though
RUN apt-get update
RUN apt-get -y -f install nano

Build the image and create the different tar files (optionally, gzip them too)

# build the image
docker build --force-rm=true --rm=true --tag="corrupt:base" .
# run it, enter bash
docker run --rm -i -t --name corrupt corrupt:base /bin/bash

# =====
# in another terminal, export it
docker export corrupt > corrupt.export.tar
# you can exit that container now.
# save the image to a tar file as well.
docker save -o corrupt.save.tar corrupt:base

# gzip both images in place
gzip --best *.tar

# import both image files
cat corrupt.export.tar.gz | docker import - corrupt:export
cat corrupt.save.tar.gz | docker import - corrupt:save

# Now try running /bin/bash in both in interactive mode
# the exported version works fine:
docker run --rm -i -t --name export corrupt:export /bin/bash


# but the saved one 
docker run --rm -i -t --name save corrupt:save /bin/bash
# produces error:
2014/07/31 05:28:24 exec: "/bin/bash": stat /bin/bash: no such file or directory
2014/07/30 22:28:24 Error response from daemon: Cannot start container 2862bb6ffcc5c9526b3cd2df56844fae4dbe0eefdf4448c413a26dc6b4925527: exec: "/bin/bash": stat /bin/bash: no such file or directory

docker -dD log

2014/07/30 22:39:43 docker daemon: 1.1.2 d84a070; execdriver: native; graphdriver: 
[620b8579] +job serveapi(unix:///var/run/docker.sock)
[620b8579] +job initserver()
[620b8579.initserver()] Creating server
2014/07/30 22:39:43 Listening for HTTP on unix (/var/run/docker.sock)
[debug] server.go:1137 Registering GET, /events
[debug] server.go:1137 Registering GET, /images/search
[debug] server.go:1137 Registering GET, /containers/{name:.*}/export
[debug] server.go:1137 Registering GET, /info
[debug] server.go:1137 Registering GET, /images/viz
[debug] server.go:1137 Registering GET, /images/{name:.*}/json
[debug] server.go:1137 Registering GET, /containers/{name:.*}/json
[debug] server.go:1137 Registering GET, /containers/{name:.*}/top
[debug] server.go:1137 Registering GET, /containers/{name:.*}/attach/ws
[debug] server.go:1137 Registering GET, /_ping
[debug] server.go:1137 Registering GET, /containers/json
[debug] server.go:1137 Registering GET, /containers/{name:.*}/changes
[debug] server.go:1137 Registering GET, /version
[debug] server.go:1137 Registering GET, /images/json
[debug] server.go:1137 Registering GET, /images/{name:.*}/get
[debug] server.go:1137 Registering GET, /images/{name:.*}/history
[debug] server.go:1137 Registering GET, /containers/ps
[debug] server.go:1137 Registering GET, /containers/{name:.*}/logs
[debug] server.go:1137 Registering POST, /images/create
[debug] server.go:1137 Registering POST, /containers/{name:.*}/unpause
[debug] server.go:1137 Registering POST, /containers/{name:.*}/stop
[debug] server.go:1137 Registering POST, /auth
[debug] server.go:1137 Registering POST, /build
[debug] server.go:1137 Registering POST, /containers/{name:.*}/resize
[debug] server.go:1137 Registering POST, /containers/{name:.*}/copy
[debug] server.go:1137 Registering POST, /commit
[debug] server.go:1137 Registering POST, /images/load
[debug] server.go:1137 Registering POST, /images/{name:.*}/tag
[debug] server.go:1137 Registering POST, /containers/{name:.*}/restart
[debug] server.go:1137 Registering POST, /containers/{name:.*}/start
[debug] server.go:1137 Registering POST, /images/{name:.*}/push
[debug] server.go:1137 Registering POST, /containers/create
[debug] server.go:1137 Registering POST, /containers/{name:.*}/kill
[debug] server.go:1137 Registering POST, /containers/{name:.*}/pause
[debug] server.go:1137 Registering POST, /containers/{name:.*}/wait
[debug] daemon.go:779 Using graph driver aufs
[debug] daemon_aufs.go:16 Migrating existing containers
[debug] daemon.go:792 Creating images graph
[debug] graph.go:67 Restored 33 elements
[debug] daemon.go:804 Creating volumes graph
[debug] graph.go:67 Restored 6 elements
[debug] daemon.go:809 Creating repository list
[620b8579] +job init_networkdriver()
[debug] /sbin/iptables, [--wait -C POSTROUTING -t nat -s 172.17.42.1/16 ! -d 172.17.42.1/16 -j MASQUERADE]
[debug] server.go:1137 Registering POST, /containers/{name:.*}/attach
[debug] server.go:1137 Registering DELETE, /containers/{name:.*}
[debug] server.go:1137 Registering DELETE, /images/{name:.*}
[debug] server.go:1137 Registering OPTIONS, 
[debug] server.go:1227 docker group found. gid: 127
[debug] /sbin/iptables, [--wait -D FORWARD -i docker0 -o docker0 -j DROP]
[debug] /sbin/iptables, [--wait -C FORWARD -i docker0 -o docker0 -j ACCEPT]
[debug] /sbin/iptables, [--wait -C FORWARD -i docker0 ! -o docker0 -j ACCEPT]
[debug] /sbin/iptables, [--wait -C FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT]
[debug] /sbin/iptables, [--wait -t nat -D PREROUTING -m addrtype --dst-type LOCAL -j DOCKER]
[debug] /sbin/iptables, [--wait -t nat -D OUTPUT -m addrtype --dst-type LOCAL ! --dst 127.0.0.0/8 -j DOCKER]
[debug] /sbin/iptables, [--wait -t nat -D OUTPUT -m addrtype --dst-type LOCAL -j DOCKER]
[debug] /sbin/iptables, [--wait -t nat -D PREROUTING -j DOCKER]
[debug] /sbin/iptables, [--wait -t nat -D OUTPUT -j DOCKER]
[debug] /sbin/iptables, [--wait -t nat -F DOCKER]
[debug] /sbin/iptables, [--wait -t nat -X DOCKER]
[debug] /sbin/iptables, [--wait -t nat -N DOCKER]
[debug] /sbin/iptables, [--wait -t nat -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER]
[debug] /sbin/iptables, [--wait -t nat -A OUTPUT -m addrtype --dst-type LOCAL ! --dst 127.0.0.0/8 -j DOCKER]
[620b8579] -job init_networkdriver() = OK (0)
2014/07/30 22:39:43 WARNING: Your kernel does not support cgroup swap limit.
2014/07/30 22:39:43 Local (127.0.0.1) DNS resolver found in resolv.conf and containers can't use it. Using default external servers : [8.8.8.8 8.8.4.4]
[debug] daemon.go:336 Loaded container 241aa67b2021cf901d799d4a169c5145d90a7b69b9575204724f2fe034f0ee92
[debug] daemon.go:336 Loaded container e073b59ef999c1a65616d5b76f243bd93f18771dbdfecbd337ac1a0d9687a689
[620b8579.initserver()] Creating pidfile
[620b8579.initserver()] Setting up signal traps
[620b8579] -job initserver() = OK (0)
[620b8579] +job acceptconnections()
[620b8579] -job acceptconnections() = OK (0)
[debug] server.go:1022 Calling POST /containers/create
2014/07/30 22:39:51 POST /v1.13/containers/create?name=save
[620b8579] +job create(save)
[620b8579] -job create(save) = OK (0)
[debug] server.go:1022 Calling POST /containers/{name:.*}/attach
2014/07/30 22:39:51 POST /v1.13/containers/19ff06eb1e12ba01b47004c056e7ff26e4749d093ee8ec5ca0747ed39c3daecb/attach?stderr=1&stdin=1&stdout=1&stream=1
[620b8579] +job container_inspect(19ff06eb1e12ba01b47004c056e7ff26e4749d093ee8ec5ca0747ed39c3daecb)
[620b8579] -job container_inspect(19ff06eb1e12ba01b47004c056e7ff26e4749d093ee8ec5ca0747ed39c3daecb) = OK (0)
[620b8579] +job attach(19ff06eb1e12ba01b47004c056e7ff26e4749d093ee8ec5ca0747ed39c3daecb)
[debug] attach.go:22 attach: stdin: begin
[debug] attach.go:59 attach: stdout: begin
[debug] attach.go:97 attach: stderr: begin
[debug] attach.go:143 attach: waiting for job 1/3
[debug] server.go:1022 Calling POST /containers/{name:.*}/start
2014/07/30 22:39:51 POST /v1.13/containers/19ff06eb1e12ba01b47004c056e7ff26e4749d093ee8ec5ca0747ed39c3daecb/start
[620b8579] +job start(19ff06eb1e12ba01b47004c056e7ff26e4749d093ee8ec5ca0747ed39c3daecb)
[620b8579] +job allocate_interface(19ff06eb1e12ba01b47004c056e7ff26e4749d093ee8ec5ca0747ed39c3daecb)
[620b8579] -job allocate_interface(19ff06eb1e12ba01b47004c056e7ff26e4749d093ee8ec5ca0747ed39c3daecb) = OK (0)
[error] container.go:475 Error running container: exec: "/bin/bash": stat /bin/bash: no such file or directory
[620b8579] +job release_interface(19ff06eb1e12ba01b47004c056e7ff26e4749d093ee8ec5ca0747ed39c3daecb)
[620b8579] -job release_interface(19ff06eb1e12ba01b47004c056e7ff26e4749d093ee8ec5ca0747ed39c3daecb) = OK (0)
[error] container.go:522 19ff06eb1e12ba01b47004c056e7ff26e4749d093ee8ec5ca0747ed39c3daecb: Error closing terminal: invalid argument
[debug] attach.go:76 attach: stdout: end
[debug] attach.go:114 attach: stderr: end
[debug] attach.go:148 attach: job 1 completed successfully
[debug] attach.go:143 attach: waiting for job 2/3
[debug] attach.go:148 attach: job 2 completed successfully
[debug] attach.go:143 attach: waiting for job 3/3
[debug] server.go:2344 Closing buffered stdin pipe
[debug] attach.go:49 attach: stdin: end
[debug] attach.go:148 attach: job 3 completed successfully
[debug] attach.go:150 attach: all jobs completed successfully
[620b8579] -job attach(19ff06eb1e12ba01b47004c056e7ff26e4749d093ee8ec5ca0747ed39c3daecb) = OK (0)
[620b8579] +job release_interface(19ff06eb1e12ba01b47004c056e7ff26e4749d093ee8ec5ca0747ed39c3daecb)
[620b8579] -job release_interface(19ff06eb1e12ba01b47004c056e7ff26e4749d093ee8ec5ca0747ed39c3daecb) = OK (0)
[error] container.go:522 19ff06eb1e12ba01b47004c056e7ff26e4749d093ee8ec5ca0747ed39c3daecb: Error closing terminal: invalid argument
Cannot start container 19ff06eb1e12ba01b47004c056e7ff26e4749d093ee8ec5ca0747ed39c3daecb: exec: "/bin/bash": stat /bin/bash: no such file or directory
[620b8579] -job start(19ff06eb1e12ba01b47004c056e7ff26e4749d093ee8ec5ca0747ed39c3daecb) = ERR (1)
[error] server.go:1048 Error making handler: Cannot start container 19ff06eb1e12ba01b47004c056e7ff26e4749d093ee8ec5ca0747ed39c3daecb: exec: "/bin/bash": stat /bin/bash: no such file or directory
[error] server.go:90 HTTP Error: statusCode=500 Cannot start container 19ff06eb1e12ba01b47004c056e7ff26e4749d093ee8ec5ca0747ed39c3daecb: exec: "/bin/bash": stat /bin/bash: no such file or directory
2014/07/30 22:40:00 Received signal 'interrupt', starting shutdown of docker...
[debug] daemon.go:890 starting clean shutdown of all containers...

uname -a

Linux 63m67 3.13.0-32-generic #57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

docker version

Client version: 1.1.2
Client API version: 1.13
Go version (client): go1.2.1
Git commit (client): d84a070
Server version: 1.1.2
Server API version: 1.13
Go version (server): go1.2.1
Git commit (server): d84a070

docker -D info

Containers: 0
Images: 11
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Dirs: 11
Execution Driver: native-0.2
Kernel Version: 3.13.0-32-generic
Debug mode (server): false
Debug mode (client): true
Fds: 10
Goroutines: 21
EventsListeners: 0
Init Path: /usr/bin/docker
Sockets: [unix:///var/run/docker.sock]
WARNING: No swap limit support

[nisanharamati]

Turns out that the error arises from using docker import instead of docker load.
The saved images work correctly when loaded, rather than imported.

[bitterloa]

@nisanharamati your comment on using docker load instead of docker import SAVED ME. I was having an extremely hard time trying to restore a docker image. And using docker LOAD did the trick. Why, why, why is docker import shown in the official documentation if it just mangles the images??

[thaJeztah]

docker import is the counterpart to docker export, and docker load the counterpart of docker save. They both have their function, but you cannot mix them