Incubator-superset: docker-compose failed to build

[lyy8510a]

Step 16/22 : USER superset
---> Running in 696d6fc0cd39
Removing intermediate container 696d6fc0cd39
---> 1552fd127697
Step 17/22 : RUN cd superset/assets && npm ci && npm run build && rm -rf node_modules
---> Running in 83c241f02c64
/bin/sh: 1: npm: not found
ERROR: Service 'superset' failed to build: The command '/bin/sh -c cd superset/assets && npm ci && npm run build && rm -rf node_modules' returned a non-zero code: 127

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the label #bug to this issue, with a confidence of 0.95. Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[zhouyao1994]

I just run this sucess by folowing step.may be that wil help you.

1. install ubuntu 18.04 in virtual box
2. install docker and docker-compose
3. git clone the superset source code
4. run docker-compose run -e SUPERSET_LOAD_EXAMPLES=yes --rm superset ./docker-init.sh

please write more detail.thank you.

[vdmitriyev]

@lyy8510a I had the same issue. The advice of the @zhouyao1994 helped me. Actually, I was not doing anything else, other than applying the commands suggested by @zhouyao1994.

• However, in my case by the very first installation of the VM I used not that much resources. My VM - Ubuntu 18.04 LTS, installed via vagrant on Windows host.
• Then I added more resources to the VM (RAM - 4 Gb, CPU - 2x), but still experienced the same problem.
• Then I increased size of the swap to 1GB, as it was suggested here.
• But even after adding more resources, the installation didn't succeed.
• Then I restarted VM, stopped all containers, and removed them via sudo docker system prune and made sure that all previous containers were removed.
• Then navigated to the <YOUR-PATH>/incubator-superset/contrib/docker/ and run sudo docker-compose run -e SUPERSET_LOAD_EXAMPLES=yes --rm superset ./docker-init.sh (as suggested by @zhouyao1994). And after waiting for a while and typing in admin login the installation was successfully completed!
• As the last step, I run the containers manually via sudo docker-compose up

I hope it helps you.

[zhouyao1994]

Hi @vdmitriyev @lyy8510a

• You should do some init oprations when you first run the docker compose .After that just run docker-compose up the superset will be run.
• there has some tips ,in document

I hope it helps you.

[vdmitriyev]

Now it could be that the issue no longer exists and it's not a bug anymore (didn't check it yet on my setup). The reason is that the docker deployment was changed by the developers in this commit - https://github.com/apache/incubator-superset/commit/a44635e3098fd0836df6d81f1cfb2a3bed2da91e

[richdevboston]

I am attempting to install Superset from the Docker container on Windows 10. I get the errors below. It repeatedly fails at Step 11/32 with a message

"ERROR: Service 'superset' failed to build: The command '/bin/sh -c cd /app/superset/assets && npm run build && rm -rf node_modules' returned a non-zero code: 1"

Here is the complete run log:

PS C:\users\lysak\incubator-superset> docker-compose run -e SUPERSET_LOAD_EXAMPLES=yes --rm superset ./docker-init.sh
Creating incubator-superset_postgres_1 ... done                                                                                                                                        Creating incubator-superset_redis_1    ... done                                                                                                                                        Building superset
Step 1/32 : ARG PY_VER=3.6.9
Step 2/32 : FROM python:${PY_VER} AS superset-py
 ---> 5bf410ee7bb2
Step 3/32 : RUN mkdir /app         && apt-get update -y         && apt-get install -y --no-install-recommends             build-essential             default-libmysqlclient-dev             libpq-dev         && rm -rf /var/lib/apt/lists/*
 ---> Using cache
 ---> ab9bca097c76
Step 4/32 : COPY ./requirements.txt /app/
 ---> Using cache
 ---> 7fb9af974dfc
Step 5/32 : RUN cd /app         && pip install --no-cache -r requirements.txt
 ---> Using cache
 ---> de369d00ea55

Step 6/32 : FROM node:10-jessie AS superset-node
 ---> 48b88f03a78a
Step 7/32 : RUN mkdir -p /app/superset/assets
 ---> Using cache
 ---> a4c90f0692c5
Step 8/32 : COPY ./superset/assets/package* /app/superset/assets/
 ---> Using cache
 ---> 48144fc78100
Step 9/32 : RUN cd /app/superset/assets         && npm ci
 ---> Using cache
 ---> 17ef2a24d08a
Step 10/32 : COPY ./superset/assets /app/superset/assets
 ---> Using cache
 ---> 524aabe08d04
Step 11/32 : RUN cd /app/superset/assets         && npm run build         && rm -rf node_modules
 ---> Running in 649d97f7d7c7

> [email protected] build /app/superset/assets
> cross-env NODE_OPTIONS=--max_old_space_size=8192 NODE_ENV=production webpack --mode=production --colors --progress

clean-webpack-plugin: /app/superset/assets/dist has been removed.
Starting type checking service...
Using 1 worker with 2048MB memory limit
 92% chunk asset optimization OptimizeCssAssetsWebpackPluginBrowserslist: caniuse-lite is outdated. Please run next command `npm update`
 92% chunk asset optimization TerserPluginnpm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] build: `cross-env NODE_OPTIONS=--max_old_space_size=8192 NODE_ENV=production webpack --mode=production --colors --progress`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2019-12-06T03_31_57_536Z-debug.log
ERROR: Service 'superset' failed to build: The command '/bin/sh -c cd /app/superset/assets         && npm run build         && rm -rf node_modules' returned a non-zero code: 1

[richdevboston]

This is still an open issue. I just tried it again today on a different Windows 10 Professional machine with a fresh Docker Toolbox installation that uses VirtualBox and it fails there too. Same error as above.

It failed again when I tried doubling the amount of RAM and swap space available for VirtualBox during the build process.

[vdmitriyev]

@richdevboston I see, I didn't try it on Windows 10 yet (at least the latest variant of the deployment), but in my case I used vagrant and I also was forced to increase the size of the hard-disk of the virtualbox to get the service up and running. It's because the initialization phase is quite greedy for the resources.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.

[jcrodrigue]

I have experienced the same exact issue last night trying to set it up using Docker on windows 10

[vdmitriyev]

@jcrodrigue what setup do you have? It could happen because of some random docker issues on windows.

[jcrodrigue]

Dell Lattitude 7490
Windows 10. I feel this may be due to a windows Docker issue because I managed to successfully install it with the docker image on my Macbook pro without any issues.

[vdmitriyev]

@jcrodrigue in my case on Windows with vagrant I had to remove all files related to Superset from shared with my host machine (in this case windows) folder. The issue was that the shared folder cannot be used to create and use symlinks, which are used by Superset installation or so.

[arunp-motorq]

Running into the same issue on windows 10. Am getting

No such file or directory
ERROR: Service 'superset' failed to build: The command '/bin/sh -c /frontend-mem-nag.sh && cd /app/superset-frontend && npm ci' returned a non-zero cod
e: 127

[jcrodrigue]

Yeah so maybe it's not possible to do it on Windows 10. I don't know.

[tlightsky]

Running into the same issue on windows 10. Am getting

No such file or directory
ERROR: Service 'superset' failed to build: The command '/bin/sh -c /frontend-mem-nag.sh && cd /app/superset-frontend && npm ci' returned a non-zero cod
e: 127

same problem here in Windows 10

[tlightsky]

is there any walk around?

[vdmitriyev]

1. Setup/Environment:
   
   
   
   • Windows 10 + Docker Desktop + ~ 6 GB of RAM given to Docker (can be configured in settings of the docker)
   
   
2. Problem:
   
   
   
   • The script that is used to check whether there is enough RAM memory provided to the docker is called frontend-mem-nag.sh. It is initially git-cloned to WINDOWS system and by default gets file-endings for WINDOWS system and then the script runs on LINUX system and fails. More information mentioned here - Docker for Windows: Dealing With Windows Line Endings
   
   
3. (version 1 - no code changes required) Workaround solution for Windows host:
   # all commands executed in cmd/terminal docker system prune -a git clone https://github.com/apache/incubator-superset --config core.autocrlf=input <... navigate to cloned git-repo ...> cd docker docker-compose up
4. (version 2 - code changes required) Workaround solution for Windows host:
   
   
   
   • Make sure no previous docker images are existing
     
     
     
     
     
     • Stop everything and run docker system prune -a
     
     
     
   
   
   • Find the following line in the file "Dockerfile"
     
     COPY ./superset-frontend/package* /app/superset-frontend/
   
   
   • Insert following commands after the line:
     
     bash RUN apt-get update -y RUN apt-get install -y apt-file RUN apt-file update -y RUN apt-get install -y dos2unix RUN dos2unix /frontend-mem-nag.sh
     
     P.S. @tlightsky I hope it will help you.
   
   

[tlightsky]

1. Setup/Environment:

   • Windows 10 + Docker Desktop + ~ 6 GB of RAM given to Docker (can be configured in settings of the docker)

2. Problem:

   • The script that is used to check whether there is enough RAM memory provided to the docker is called frontend-mem-nag.sh. It is initially git-cloned to WINDOWS system and by default gets file-endings for WINDOWS system and then the script runs on LINUX system and fails. More information mentioned here - Docker for Windows: Dealing With Windows Line Endings
3. (version 1 - no code changes required) Workaround solution for Windows host:
   # all commands executed in cmd/terminal docker system prune -a git clone https://github.com/apache/incubator-superset --config core.autocrlf=input <... navigate to cloned git-repo ...> cd docker docker-compose up

4. (version 2 - code changes required) Workaround solution for Windows host:

   • Make sure no previous docker images are existing

     • Stop everything and run docker system prune -a

   • Find the following line in the file "Dockerfile"
     COPY ./superset-frontend/package* /app/superset-frontend/

   • Insert following commands after the line:

   RUN apt-get update -y 
   RUN apt-get install -y apt-file
   RUN apt-file update -y 
   RUN apt-get install -y dos2unix
   RUN dos2unix /frontend-mem-nag.sh
   

P.S. @tlightsky I hope it will help you.

many thanks for your detailed explanation,
I'll try out on my Win 10

[tlightsky]

1. Setup/Environment:

   • Windows 10 + Docker Desktop + ~ 6 GB of RAM given to Docker (can be configured in settings of the docker)

2. Problem:

   • The script that is used to check whether there is enough RAM memory provided to the docker is called frontend-mem-nag.sh. It is initially git-cloned to WINDOWS system and by default gets file-endings for WINDOWS system and then the script runs on LINUX system and fails. More information mentioned here - Docker for Windows: Dealing With Windows Line Endings
3. (version 1 - no code changes required) Workaround solution for Windows host:
   # all commands executed in cmd/terminal docker system prune -a git clone https://github.com/apache/incubator-superset --config core.autocrlf=input <... navigate to cloned git-repo ...> cd docker docker-compose up

4. (version 2 - code changes required) Workaround solution for Windows host:

   • Make sure no previous docker images are existing

     • Stop everything and run docker system prune -a
   • Find the following line in the file "Dockerfile"
     COPY ./superset-frontend/package* /app/superset-frontend/
   • Insert following commands after the line:

   RUN apt-get update -y 
   RUN apt-get install -y apt-file
   RUN apt-file update -y 
   RUN apt-get install -y dos2unix
   RUN dos2unix /frontend-mem-nag.sh
   

P.S. @tlightsky I hope it will help you.

many thanks for your detailed explanation,
I'll try out on my Win 10

it works, thanks!

[jcrodrigue]

Great! Would this make it to the documentation to help windows users setup in docker?

[vdmitriyev]

Great! Would this make it to the documentation to help windows users setup in docker?

I think, we need to ask one of the maintainers @mistercrunch

[jcrodrigue]

Bummer looks like this is unrelated but another issue has come up
ERROR: error pulling image configuration: Get https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sha256/87/87856cc39862cec77541d68382e4867d7ccb29a85a17221446c857ddaebca916/data?verify=1587082069-3PtcJC%2F8OtRmHd4VA5wMNH2OxnQ%3D: x509: certificate has expired or is not yet valid This may not be windows specific

[vdmitriyev]

Bummer looks like this is unrelated but another issue has come up
ERROR: error pulling image configuration: Get https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sha256/87/87856cc39862cec77541d68382e4867d7ccb29a85a17221446c857ddaebca916/data?verify=1587082069-3PtcJC%2F8OtRmHd4VA5wMNH2OxnQ%3D: x509: certificate has expired or is not yet valid This may not be windows specific

It looks like a docker client problem. The issue was discussed as one of the docker's issues here - https://github.com/docker/machine/issues/4046

Possible solution could be (as suggested in the answers to aforementioned issue - https://github.com/docker/machine/issues/4046)

docker-machine regenerate-certs --client-certs default

[brij100]

you should follow below command & it might be help you guys!

docker pull preset/superset

docker run -d -p 8080:8080 --name superset preset/superset

docker exec -it superset superset fab create-admin \
--username admin \
--firstname Superset \
--lastname Admin \
--email [email protected] \
--password admin

docker exec -it superset superset db upgrade

docker exec -it superset superset load_examples

docker exec -it superset superset init

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.