Packer: Packer 1.4.0 builds Ubuntu Docker image that cannot run /bin/bash, /bin/sh

[njgraham]

Note that this problem doesn't happen when I use Packer version 1.3.5 following the exact same steps (as far as I can tell - I did double-check).

Use the minimal template below and run the following with packer version 1.4.0:

$ packer build packer.json
...
==> Builds finished. The artifacts of successful builds are:
--> docker: Imported Docker image: sha256:1078de9e6ed859500f3551279a760b38c056078ac139d8af8f0dca5217901030
--> docker: Imported Docker image: my-test-repository:my-test-tag

Then, try to run /bin/bash or /bin/sh with a container from the resulting image - both fail:

$ docker run -it --rm my-test-repository:my-test-tag /bin/sh                                                                                                                     
/bin/sh: 1: /bin/sh: Syntax error: "(" unexpected

$ docker run -it --rm my-test-repository:my-test-tag /bin/bash                                                                                                                   
/bin/bash: 3: /bin/bash: Syntax error: Unterminated quoted string

Rerun the same packer command with version 1.3.5. Then, bash works:

$ docker run -it --rm my-test-repository:my-test-tag /bin/bash                                                                                                                   
root@2b6741a0cc3d:/# 

Requested details

Packer version: 1.4.0

> $ packer --version                                                                                                                                                                                                  
1.4.0

Host platform: LinuxMint x64:

Distributor ID: LinuxMint
Description:    Linux Mint 19.1 Tessa
Release:    19.1
Codename:   tessa

Debug log output: https://gist.github.com/njgraham/216761a042988abade4dc22a3cd13978

Simple packer template used in the above examples:

{
  "builders": [{
    "type": "docker",
    "image": "ubuntu:18.04",
    "commit": true
  }],
  "provisioners":
  [
    {
      "type": "shell",
      "inline": [
        "touch /test-file"
      ]
    }
  ],
  "post-processors": [
    [
      {
        "type": "docker-tag",
        "repository": "my-test-repository",
        "tag": "my-test-tag"
      }
    ]
  ]
}

[njgraham]

I was able to reproduce a very similar symptom with version 1.3.5 if I use a shell other than bash (I picked the official Alpine image). Specifically, using the following line in the docker builder:

"run_command": ["-d", "-i", "-t", "--entrypoint=/bin/ash", "--", "{{.Image}}"]

...and the following in the inline shell provisioner:

"inline_shebang": "/bin/ash -e"

Then, after building like before:

$ docker run -it --rm my-test-repository:my-test-tag /bin/ash
/bin/ash: line 1: ELF: not found
/bin/ash: line 2: p: not found
...lots more complaints...

If I switch to a base container with bash installed (frolvlad/alpine-bash) and remove the above docker/shell arguments it works as expected.

I thought this might be relevant as I seem to recall seeing some discussion about how 1.4.0 defaults to sh rather than bash.

[SwampDragons]

We did change it for the 1.4.0 release: https://github.com/hashicorp/packer/pull/7106/commits/2718a5ed62a1b21a106c9a69742d6b763d092935

[SwampDragons]

if I use a run_command of ["-d", "-i", "-t", "{{.Image}}", "/bin/sh"] it seems to work fine. I'm not an expert on how docker reads entrypoints so I'm going to need to do a docs dive on this one.

[azr]

Hello there, recently the default entrypoint has changed for docker images, it's now /bin/sh, so when you run docker run -it --rm my-test-repository:my-test-tag "/bin/sh", it does this:

❯ /bin/sh /bin/sh
/bin/sh: /bin/sh: cannot execute binary file

You can run sh or bash by running docker run -it --rm my-test-repository:my-test-tag -c "/bin/sh" :

❯ docker run -it --rm my-test-repository:my-test-tag -c "/bin/sh"
# ls
bin   dev  home  lib64  mnt  packer-files  root  sbin  sys        tmp  var
boot  etc  lib   media  opt  proc          run   srv   test-file  usr
# exit

❯ docker run -it --rm my-test-repository:my-test-tag -c "/bin/bash"
root@2209dffa207c:/# ls
bin   dev  home  lib64  mnt  packer-files  root  sbin  sys        tmp  var
boot  etc  lib   media  opt  proc          run   srv   test-file  usr
root@2209dffa207c:/# exit

Another way to run bash could be to just pass the entrypoint:

❯ docker run -it --rm --entrypoint=/bin/bash my-test-repository:my-test-tag      
root@ab767ff84d6d:/# ls
bin   dev  home  lib64  mnt  packer-files  root  sbin  sys        tmp  var
boot  etc  lib   media  opt  proc          run   srv   test-file  usr
root@ab767ff84d6d:/# exit

The simplest form is to run sh:

❯ docker run -it --rm my-test-repository:my-test-tag      
# ls
bin   dev  home  lib64  mnt  packer-files  root  sbin  sys        tmp  var
boot  etc  lib   media  opt  proc          run   srv   test-file  usr
# exit

[azr]

One thing we could do here would be to change the default entrypoint from [/bin/sh] to [/bin/sh -c] so that you can run: docker run -it --rm my-test-repository:my-test-tag /bin/sh.

What do you think @rickard-von-essen, @SwampDragons ?

[njgraham]

Ah, -c as pointed out by @azr clears it up for me! Thank you!

> $ docker run -it --rm my-test-repository:my-test-tag -c /bin/ash                                                                                                                                                    
/ # 

[SwampDragons]

@azr I think we should add some documentation about this and then close, what do you think?

[azr]

there it is !

[rlees85]

I'm not sure if this is relevant but it seems now packer is changing the entrypoint from the parent images to just "/bin/sh" in my opinion this is a huge step backwards/bug.

[azr]

Hey @rlees85, because the docker builder uses command lines, the entrypoint is defaulted to what entrypoint was used in the command line.
From docker's point of view, I think it makes sense because if you could build a docker image with that entrypoint there's a very high chance it's still there after ! 🙂

It used to be /bin/bash so this a bit better as /bin/sh is a more spread shell.

You can follow the instructions in this comment to fix it !

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.