Docker-node: 'npm install' in a Dockerfile is not installing all my dependencies.

[AnatoleLucet]

So it's pretty strange, i don't have any error, my missing module is listed in my package.json in my container, but the module is not in my node_modules folder.

Here's my Dockerfile :

FROM node:8-alpine
RUN mkdir /app
WORKDIR /app
COPY package.json /app
RUN apk add --no-cache ffmpeg opus pixman cairo pango giflib ca-certificates \
    && apk add --no-cache --virtual .build-deps python g++ make gcc .build-deps curl git pixman-dev cairo-dev pangomm-dev libjpeg-turbo-dev giflib-dev \
    && npm install \
    && apk del .build-deps
COPY . /app
CMD ["npm", "start"]

Here's my docker-compose.yml :

version: '3'
services:
  app:
    build: .
    volumes:
      - ./:/app
      - /app/node_modules
    restart: always
    env_file:
      - .env
    links:
      - mongo
    deploy:
      restart_policy:
        condition: on-failure
        delay: 5s
        max_attempts: 3

  mongo:
    image: mongo
    volumes:
      - ~/data:/data/db
    restart: always

  adminmongo:
    image: mrvautin/adminmongo
    ports:
      - 1234:1234
    environment:
      - HOST=0.0.0.0

And my dependencies package.json :

"dependencies": {
    "cleverbot-node": "^0.3.11",
    "discord.js": "github:discordjs/discord.js",
    "enmap": "^4.5.0",
    "googleapis": "^36.0.0",
    "jsonfile": "^5.0.0",
    "mongoose": "^5.3.15",
    "node-gyp": "^3.8.0",
    "node-opus": "^0.3.1",
    "nodemon": "^1.18.9",
    "request": "^2.88.0",
    "xml-js": "^1.6.8",
    "ytdl-core-discord": "^1.0.2"
  },

Just to precise the missing module is "jsonfile" but this is probably because it is the last i've installed.
As i've said when i do a docker-compose build i don't get any error but the 'building' of the container is abnormally long.

[chorrell]

I think this might be caused by a combination of WORKDIR /app and your later steps.

WORKDIR sets the working directory of all subsequent steps, so package.json /app effectively copies to /app/app.

Try something like this (note that WORKDIR will create directories that do not already exist):

FROM node:8-alpine
WORKDIR /app
COPY package.json .
RUN apk add --no-cache ffmpeg opus pixman cairo pango giflib ca-certificates \
    && apk add --no-cache --virtual .build-deps python g++ make gcc .build-deps curl git pixman-dev cairo-dev pangomm-dev libjpeg-turbo-dev giflib-dev \
    && npm install \
    && apk del .build-deps
COPY . .
CMD ["npm", "start"]

[AnatoleLucet]

@chorrell Hoo ok, this worked thanks you i understand now.

[twitu]

I faced a similar problem when using docker-compose, however I do not understand this solution completely. Can you please explain why the COPY package.json . and subsequent COPY . /app is required, when the README for this repo states

Docker Compose example copies your current directory (including node_modules) to the container. It assumes that your application has a file named package.json defining start script.

Thanks for your help.

[vagnervst]

Just sharing my two cents here in case anyone is facing a similar scenario:

I have a volume to share source code between host and container using docker-compose. This ended up deleting the node_modules folder on the container (because as a first time cloner, I didn't install the dependencies on the host machine), so that caused my modules to not be found.

The solution I found was to include node_modules on my volumes

volumes:
  - /sillymistake/node_modules

[bharath0208]

I am running node version v13.14.0 and npm v 6.14.4 inside my docker container. The base image is node:13-alpine. When i run npm install inside my docker pod it only installs 185 packages whereas when I do npm install in my local using the same package.json file it installs around 1733 packages. Any idea on why this is happening @chorrell @AnatoleLucet @twitu @vagnervst @paolomainardi

[AnatoleLucet]

@bharath0208 can you please give us more info (Dockerfile, docker-compose.yml, command(s) used, etc...)?
Do you also copy the package-lock.json / yarn.lock in you container before installing the deps? COPY package*.json .

[bharath0208]

I have verified that the files like package.json are there in the correct location. I don't copy the package-lock.json but I do copy the yarn.lock.

I even tried with and without copying package-lock.json but no help. Should i also try without copying yarn.lock file as well. @anatlole

FROM node:latest
#FROM ubuntu:18.04 
#RUN apt-get -y update
#RUN apt-get -y install nodejs
#RUN apt-get -y install npm
#RUN apt-get -y install vim
#RUN apt-get -y install curl

WORKDIR /app
COPY frontend ./frontend
COPY backend .
COPY backend/app .
COPY nodeInstall.sh .
COPY .npmrc /root
COPY ibmca.pem .
COPY redis.cert .
RUN npm install
RUN npm script build
CMD [ "node", "server.js" ]

@ AnatoleLucet

[AnatoleLucet]

@bharath0208 why would you use both yarn and npm? You should copy lock files to your container.
By what I can see, you have some kinds of monorepo with both your backend & frontend. It seems like you copy them to your container in some strange locations (your probleme may come from here).

But anyway, I don't really understand what you're trying to do. It would probably be more appropriated to ask the same question on StackOverflow or on Docker's forum.
If you do so, give a repo that reproduce your issue if it's possible. If it isn't possible, give the more information you can, like your above Dockerfile, your docker-compose.yml if you use docker-compose, the command you execute to build your image, your package.json location (you aren't copying it explicitly in your Dockerfile so I assume it's in a subfolder), and anything else you can think about...

[bharath0208]

@AnatoleLucet Yes we follow monorepo style wherein if you see my docker file I copy all the files from frontend & backend to my workdir. The frontend has package.json,webpack.config.js,webpack.dev.js so what I am doing here is basically installing my npm packages and then trying to build it. Once I have the dist files created I would serve it to my backend and start my backend server. The command which I use to run my docker file is docker build -t "frontend:0.2" . We gitignore package-lock.json & yarn.lock
these files from getting pushed into our repo so these are not copied into the container as well. The package.json is already present in frontend so it gets copied and I can see it inside my container.

[AnatoleLucet]

@bharath0208 then I guess you may try to RUN cd ./fontend before installing the deps?

[bharath0208]

@AnatoleLucet I did try that but still no luck. Also, I am trying to exec into my docker pod and do npm i. Please see the snippet When i do npm i in my docker it installs only 183 packages whereas when I do it on my localhost it installs more than 1883 packages.

[AnatoleLucet]

@bharath0208 maybe your local npm's version isn't the same has the one in your container, and since you're not using a lock file npm is installing more or less dependencies.

From npm's doc about package-lock.json:

Describe a single representation of a dependency tree such that teammates, deployments, and continuous integration are guaranteed to install exactly the same dependencies.

Anyway, I do recommend you to use a lock file, pushing it to the remote repo for your coworkers, and try copying it in your container. This may solve your issue.

[bharath0208]

@AnatoleLucet i got it working actually there was a tag which says devDependencies in package.json. And my env variable was set to prod because of which it was not installing certain dependencies.

[Angelina1984]

@bharath0208 how did you final docker file ended up looking like?
I am facing the same issue. I want to create docker image with dependencies during Install stage.
Then, call that docker image in Test stage. Is this doable?

Dockerfile

FROM node:12
WORKDIR /app
COPY package.json .
COPY . .
RUN npm install

Jenkinsfile

String MY_TEST_DOCKER_IMAGE = 'my-artifactory.com/reponame/mymfetestcontainer:latest'
buildContainerVersion = "1.1"
imageName = "reponame/mymfetestcontainer"

stage('Install') {
    steps {
           script {
                 docker.withRegistry('https://my-artifactory.com', 'svc_d_artifactory'){
                       def customImage = docker.build("${imageName}:${buildContainerVersion}")
                       customImage.push()
                       customImage.push("latest")
                       // delete image from local to cleanup your server space
                       sh "docker rmi --force \$(docker images -q ${customImage.id} | uniq)"
                  }
            }
      }
}
stage('Unit') {
     agent {
            docker {
                 image MY_TEST_DOCKER_IMAGE
                args '-u root'
            }
     }
     steps {
            sh 'npm run test:unit'
     }
}