Yarn: Running jest from yarn fails with "/bin/sh: 1: jest: not found" in certain circumstances

[davidmichaelkarr]

I don't honestly believe I'm dealing with a bug with yarn here, but I've been beating on this problem for WEEKS now, and I'm just getting nowhere.

I'm working on a project that builds a docker image built with Yarn 1.3.2 and then uses that in a Jenkins pipeline to run some build steps, including "jest".

This Jenkins pipeline script is used in a multibranch pipeline job to build the master and release branches, and this works perfectly fine. I recently started setting up a pull request build using an ordinary pipeline, but using the same Jenkinsfile.

There's a point in the script that runs "yarn --verbose test:coverage -u" inside the docker container, using standard Docker pipeline step syntax. Again, this works perfectly fine in the multibranch build for the master or release branches.

In that working multibranch pipeline job, I see output like this:
+ yarn --verbose test:coverage -u yarn run v1.3.2 warning package.json: No license field verbose 0.317 Checking for configuration file ".../.npmrc". verbose 0.317 Found configuration file ".../.npmrc". ... verbose 0.323 current time: 2018-11-15T23:42:46.484Z $ jest --env=jsdom --coverage -u [BABEL] Note: The code generator ...

In the "package.json" file, we have the following line:
"test:coverage": "jest --env=jsdom --coverage",

When this is run in the pull request build, I instead see this:
+ yarn --verbose test:coverage -u yarn run v1.3.2 warning package.json: No license field verbose 0.325 Checking for configuration file ".../.npmrc". verbose 0.325 Found configuration file ".../.npmrc". ... verbose 0.332 current time: 2018-11-15T23:42:53.120Z $ jest --env=jsdom --coverage -u /bin/sh: 1: jest: not found verbose 0.4 Error: Command failed with exit code 127.

I emphasize, we're running yarn in a docker image. The same image is used in both jobs. I don't understand what is different between the multibranch job and plain pipeline job that makes the multibranch job able to find "jest" from "yarn", but when running from the plain pipeline job, the attempt to find "jest" fails.

Is there some other diagnostic I can add to this that can give me a clue? I don't know what other background information I can provide that would help.

[davidmichaelkarr]

If it helps, here is the elided Dockerfile that is used to build the image we use:

`FROM jenkinsci/slave:latest

Install NodeJS, set it up, install gulp and bower

USER root

ENV http_proxy ...
ENV https_proxy ...
ENV no_proxy localhost,127.0.0.1,...

RUN
apt-get update && apt-get install apt-transport-https &&
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - &&
echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list &&
curl -sL https://deb.nodesource.com/setup_8.x -o setup_node.sh &&
bash setup_node.sh &&
apt-get install nodejs &&
npm config set proxy ... &&
npm config set https-proxy ... &&
npm config set registry .../nexus/repository/npm-group/ &&
npm install -g gulp bower &&
apt-get install yarn &&
apt-get clean

USER jenkins`

[davidmichaelkarr]

I thought to add additional diagnostics, comparing the output between the multibranch build and the PR build. I tried "env", "yarn info jest", and "yarn list".

I found differences in the "env" output, but some would be expected. I didn't see any significant differences. The outputs from the two "yarn info jest" calls were identical. The outputs from the two "yarn list" calls were NOT identical, and I think that's curious. Many of the differences were just small version number variances, but there were several blocks of dependencies in the multibranch output that were not in the PR build output. Many of the differences reference internal package names, but some appeared to be public packages. I didn't see "jest" mentioned in any of the differences.

[davidmichaelkarr]

I posted the following on StackOverflow describing this problem. In that posting, I also shared the output from the "yarn install" execution, which might provide a clue: https://stackoverflow.com/questions/53268194/running-jest-from-yarn-in-jenkinsfile-gets-jest-not-found-from-one-branch-su .

[bpatram]

I managed to run into a similar issue here. I was using the node:12.14.0-alpine docker image trying to run yarn test. This would result in a similar error as @davidmichaelkarr reported: /bin/sh: jest: not found.

In my case, it seems to be related to a quirk/bug when running yarn from sh (or maybe a PATH issue?). Fortunately I was able to switch to using bash. The issue went away after changing my shell. Not ideal, but a workaround nevertheless!

Additions I made to my Dockerfile:

RUN apk add --no-cache bash
SHELL [ "/bin/bash", "-c"]

# then run yarn commands

I haven't tested this, but you could probably wrap your commands like this: /bin/bash -c 'yarn command-here' if you wanted to avoid the SHELL call.