Pact-js: Pact exits with code 127 when run in docker alpine

[matusnovak]

I am trying to run pact-js within a docker (more details below). Pact runs without a problem when run directly on the host machine. However, to do my integration testing, I need to run pact within the docker container. When I create a provider and run .setup() method, pact will try to run but immediately exits with error code 127. The docker image I am using is mhart/alpine-node:9.4.0

Software versions

• OS: Linux 4.13.0-32-generic x86_64
• Consumer Pact library: [email protected]
• Node Version: 9.4.0

Expected behaviour

Run pact and create an interaction

Actual behaviour

Pact won't run and reports:

> npm run test
[2018-02-26T10:49:30.491Z]  INFO: [email protected]/236 on 6fca4e44b197: 
    Creating Pact Server with options: 
    host = 127.0.0.1,
    port = 3001,
    log = /srv/logs/pact.log,
    dir = /srv/pacts,
    spec = 2,
    ssl = false,
    sslcert = false,
    sslkey = false,
    cors = true,
    pactFileWriteMode = overwrite
[2018-02-26T10:49:30.639Z]  INFO: [email protected]/236 on 6fca4e44b197: Created './platforms/linux-x64/bin/pact-mock-service service --host '127.0.0.1' --port '3001' --log '/srv/logs/pact.log' --pact_dir '/srv/pacts' --pact_specification_version '2' --cors 'true' --pact-file-write-mode 'overwrite'' process with PID: 246
[2018-02-26T10:49:30.678Z]  WARN: [email protected]/236 on 6fca4e44b197: Pact exited with code 127.
[2018-02-26T10:49:30.679Z]  INFO: [email protected]/236 on 6fca4e44b197: Removing Pact with PID: 246

No logs are produced. However, when I try to run pact-mock-service service manually through the shell, I get error ....linux-x64/lib/ruby/bin.real/ruby: No such file or directory. The entire output:

> node_modules/@pact-foundation/pact-standalone/platforms/linux-x64/bin/pact-mock-service service --host '127.0.0.1' --port '3001' --log '/srv/logs/pact.log' --pact_dir '/srv/pacts' --pact_specification_version '2' --cors 'true' --pact-file-write-mode 'overwrite'
/srv/node_modules/@pact-foundation/pact-standalone/platforms/linux-x64/lib/ruby/bin/ruby: line 6: /srv/node_modules/@pact-foundation/pact-standalone/platforms/linux-x64/lib/ruby/bin.real/ruby: No such file or directory

What is interesting is that the ruby in bin.real exists and can be executed:

> stat /srv/node_modules/@pact-foundation/pact-standalone/platforms/linux-x64/lib/ruby/bin.real/ruby
  File: /srv/node_modules/@pact-foundation/pact-standalone/platforms/linux-x64/lib/ruby/bin.real/ruby
  Size: 2852648     Blocks: 5576       IO Block: 4096   regular file
Device: 34h/52d Inode: 153         Links: 1
Access: (0755/-rwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2015-07-14 13:19:16.000000000
Modify: 2015-07-14 13:19:16.000000000
Change: 2018-02-26 12:16:17.000000000

However, when I change the docker image from mhart/alpine-node:9.4.0 to node:9.6.0 it won't produce the error and everything works. The same problem happens even if I run https://hub.docker.com/_/alpine/ image and install nodejs manually.

My question is why does the error happen on the alpine docker image?

Steps to reproduce

1. Clone https://github.com/matusnovak/pact-cucumber-example.git
2. Build the docker image: docker build . -t pact-cucumber-example
3. Run the image: docker run --rm -it pact-cucumber-example /bin/sh
4. Then when the shell appears, run the pact tests: npm run test
5. You will get the error message: Pact exited with code 127
6. Exit docker
7. Replace FROM mhart/alpine-node:9.4.0 to FROM node:9.6.0 inside of Dockerfile
8. Build and run the tests inside of the docker again
9. Now it works

Relevant log files

No logs are produced when the error happens.

[mefellows]

Hi @matusnovak, thanks for taking the time to provide a detailed bug report.

I'm almost certain it is missing some key libraries (like musl or glibc). Take a look at https://github.com/pact-foundation/pact-mock-service-docker/blob/master/Dockerfile, which should contain the dependencies required to run Ruby in Alpine.

If that fixes your problems (it should) then I'll update the docs to help others going forward.

[bethesque]

I'm a bit confused by the failures, because the "standalone" should be completely standalone. I didn't think it relied on anything in the host. But I'm not an expert in Travelling Ruby, just a user.

[bethesque]

I'll do some googling for this error in the travelling ruby issues.

[mefellows]

"standalone" != native binary. For it to be portable across different OS's (even *nix variants), it needs to be able to depend on some minimal libraries to get stuff done which abstract low-level APIs (things like open, read, write, malloc, printf, getaddrinfo, dlopen, pthread_create, crypt, login, exit).

I believe Alpine uses the musl libc, which is what is provided in the referenced image, which is why it works there.

[bethesque]

Ok, that makes sense.

[bethesque]

So, we really need a centralised place to put the alpine instructions, because we'll have to duplicate them for every wrapper language otherwise. How about we put something on the standalone wiki, and link to it from all the wrappers?

[bethesque]

https://github.com/pact-foundation/pact-ruby-standalone/wiki/Using-the-pact-ruby-standalone-with-Alpine-Linux-Docker

Not exactly sure what we need, but I've deleted the bits we definitely don't need. Do we just want the line: RUN apk add --no-cache --virtual build-dependencies build-base ?

[mefellows]

I think that's about right
https://github.com/pact-foundation/pact-ruby-standalone/wiki/Using-the-pact-ruby-standalone-with-Alpine-Linux-Docker

As an aside, the benefit of doing this:

COPY Gemfile /app/
COPY Gemfile.lock /app/

Means that it will only trigger a docker rebuild if those files change. If given the chance, i'd probably refactor the image to look more like (not tested):

RUN apk add --no-cache --virtual build-dependencies build-base && \
    gem install bundler --no-ri --no-rdoc && \
    apk del build-dependencies build-base

COPY Gemfile /app/
COPY Gemfile.lock /app/

RUN  cd /app; bundle install && \
    gem uninstall bundler && \
    apk del build-dependencies build-base && \
    rm -r ~/.bundle/ /usr/local/bundle/cache

this would create some small speed enhancements :)

[matusnovak]

@mefellows

You were right, it was missing glibc. Alpine has smaller version uglibc, but the ruby binary inside of node_modules/@pact-foundation/pact-standalone/platforms/linux-x64/lib/ruby is looking for glibc.

Following the following instructions, I was able to run pact normally.

# Source: https://github.com/sgerrand/alpine-pkg-glibc
> apk --no-cache add ca-certificates wget
> wget -q -O /etc/apk/keys/sgerrand.rsa.pub https://raw.githubusercontent.com/sgerrand/alpine-pkg-glibc/master/sgerrand.rsa.pub
> wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.27-r0/glibc-2.27-r0.apk
> apk add glibc-2.27-r0.apk

The glibc adds another ~86MB into the container, but for my use I do not really mind, since the pact is only used during testing on CircleCI and never leaves into production.

Thanks for the help!

Edit:

Just for the clarification, I am using mhart/alpine-node:9.4.0, then install bash apk add bash, then I have installed glibc using the steps above, then npm install with [email protected], and all works without additional dependencies needed.

For some reason, you need to install bash too, otherwise you will still get exit error 127 (No such file or directory).

[mefellows]

Thanks @matusnovak, I'll update our readme to point at Beth's dude and we'll get the docs updated with this info.

As for bash, it's probably in the shebang for the runner and perhaps isn't a real requirement - I'm doubtful we rely on any bash-isms, and it could probably be converted to standard shell.

Either way, it's not a big deal or a priority so we'll close this one off - thanks!

[bethesque]

I think there were some slight tweaks to resolving the real path under sh compared to bash.

[shavo007]

just got this error on CI with the same docker image mhart/alpine-node:10.5.0

what was the solution?

[shavo007]

based on @matusnovak comment, i added the below steps to my docker file and ran the CI build and it works now.

#alpine does not come with bash :(
RUN apk add --no-cache bash git zip curl wget ca-certificates

# Source: https://github.com/sgerrand/alpine-pkg-glibc
RUN wget -q -O /etc/apk/keys/sgerrand.rsa.pub https://raw.githubusercontent.com/sgerrand/alpine-pkg-glibc/master/sgerrand.rsa.pub
RUN wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.27-r0/glibc-2.27-r0.apk

RUN apk add glibc-2.27-r0.apk

[shavo007]

Update: This url has changed now.

See https://github.com/sgerrand/alpine-pkg-glibc#please-note

[bethesque]

Does this not work? https://github.com/pact-foundation/pact-ruby-standalone/wiki/Using-the-pact-ruby-standalone-with-Alpine-Linux-Docker

[bethesque]

What's the standalone version now @mefellows? 1.54.4 has the changes to make the scripts sh compatible.

[mefellows]

Pact Node is now at 6.19.11 which contains the latest binaries, and latest Pact is 6.0.1.

To the OP, Pact 4.x.x is deprecated and we're no longer supporting upgrades.

[mefellows]

FYI Have added a section on Docker to https://docs.pact.io/docker/