Incubator-superset: Fresh Docker-Compose Local Execution Fails

Created on 21 Dec 2019  路  7Comments  路  Source: apache/incubator-superset

Tried to get Superset up and running on my local macOS machine with docker-compose based on the docs (https://superset.apache.org/installation.html#start-with-docker) and it fails to authenticate to the backend Postgres database and therefore won't start.

Expected results

I expect that when I run docker-compose up after a fresh pull of the repo, it would run and allow me to access SuperSet locally

Actual results

Fails with the following error:

docker-compose up
Starting incubator-superset_db_1    ... done
Starting incubator-superset_redis_1 ... done
Starting incubator-superset_superset-node_1   ... done
Starting incubator-superset_superset-init_1   ... done
Starting incubator-superset_superset_1        ... done
Starting incubator-superset_superset-worker_1 ... done
Attaching to incubator-superset_redis_1, incubator-superset_db_1, incubator-superset_superset-init_1, incubator-superset_superset-node_1, incubator-superset_superset_1, incubator-superset_superset-worker_1
db_1               | 
db_1               | PostgreSQL Database directory appears to contain a database; Skipping initialization
db_1               | 
redis_1            | 1:C 21 Dec 03:13:02.783 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis_1            |                 _._                                                  
redis_1            |            _.-``__ ''-._                                             
redis_1            |       _.-``    `.  `_.  ''-._           Redis 3.2.12 (00000000/0) 64 bit
redis_1            |   .-`` .-```.  ```\/    _.,_ ''-._                                   
redis_1            |  (    '      ,       .-`  | `,    )     Running in standalone mode
redis_1            |  |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
redis_1            |  |    `-._   `._    /     _.-'    |     PID: 1
redis_1            |   `-._    `-._  `-./  _.-'    _.-'                                   
redis_1            |  |`-._`-._    `-.__.-'    _.-'_.-'|                                  
redis_1            |  |    `-._`-._        _.-'_.-'    |           http://redis.io        
redis_1            |   `-._    `-._`-.__.-'_.-'    _.-'                                   
redis_1            |  |`-._`-._    `-.__.-'    _.-'_.-'|                                  
redis_1            |  |    `-._`-._        _.-'_.-'    |                                  
redis_1            |   `-._    `-._`-.__.-'_.-'    _.-'                                   
redis_1            |       `-._    `-.__.-'    _.-'                                       
redis_1            |           `-._        _.-'                                           
redis_1            |               `-.__.-'                                               
redis_1            | 
redis_1            | 1:M 21 Dec 03:13:02.785 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_1            | 1:M 21 Dec 03:13:02.785 # Server started, Redis version 3.2.12
redis_1            | 1:M 21 Dec 03:13:02.785 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
redis_1            | 1:M 21 Dec 03:13:02.785 * DB loaded from disk: 0.000 seconds
redis_1            | 1:M 21 Dec 03:13:02.785 * The server is now ready to accept connections on port 6379
db_1               | 2019-12-21 03:13:03.007 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
db_1               | 2019-12-21 03:13:03.007 UTC [1] LOG:  listening on IPv6 address "::", port 5432
superset-init_1    | 
superset-init_1    | ######################################################################
superset-init_1    | 
superset-init_1    | 
superset-init_1    | Init Step 1/4 [Starting] -- Setting up admin user
superset-init_1    | 
superset-init_1    | 
superset-init_1    | ######################################################################
superset-init_1    | 
db_1               | 2019-12-21 03:13:03.013 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1               | 2019-12-21 03:13:03.038 UTC [26] LOG:  database system was shut down at 2019-12-21 03:12:45 UTC
db_1               | 2019-12-21 03:13:03.055 UTC [1] LOG:  database system is ready to accept connections
superset_1         |  * Serving Flask app "superset.app:create_app()" (lazy loading)
superset_1         |  * Environment: development
superset_1         |  * Debug mode: on
superset_1         |  * Running on http://0.0.0.0:8088/ (Press CTRL+C to quit)
superset_1         |  * Restarting with stat
superset-worker_1  | INFO:root:logging was configured successfully
db_1               | 2019-12-21 03:13:07.524 UTC [33] FATAL:  password authentication failed for user "superset"
db_1               | 2019-12-21 03:13:07.524 UTC [33] DETAIL:  Role "superset" does not exist.
db_1               |    Connection matched pg_hba.conf line 95: "host all all all md5"
superset-worker_1  | ERROR:flask_appbuilder.security.sqla.manager:DB Creation and initialization failed: (psycopg2.OperationalError) FATAL:  password authentication failed for user "superset"
superset-worker_1  | 
superset-worker_1  | (Background on this error at: http://sqlalche.me/e/e3q8)
superset-worker_1  | Loaded your LOCAL configuration at [/app/pythonpath/superset_config.py]
superset_1         | INFO:root:logging was configured successfully
db_1               | 2019-12-21 03:13:08.053 UTC [34] FATAL:  password authentication failed for user "superset"
db_1               | 2019-12-21 03:13:08.053 UTC [34] DETAIL:  Role "superset" does not exist.
db_1               |    Connection matched pg_hba.conf line 95: "host all all all md5"
superset_1         | ERROR:flask_appbuilder.security.sqla.manager:DB Creation and initialization failed: (psycopg2.OperationalError) FATAL:  password authentication failed for user "superset"
superset_1         | 
superset_1         | (Background on this error at: http://sqlalche.me/e/e3q8)
superset-init_1    | INFO:root:logging was configured successfully
db_1               | 2019-12-21 03:13:08.292 UTC [35] FATAL:  password authentication failed for user "superset"
db_1               | 2019-12-21 03:13:08.292 UTC [35] DETAIL:  Role "superset" does not exist.
db_1               |    Connection matched pg_hba.conf line 95: "host all all all md5"
superset-init_1    | ERROR:flask_appbuilder.security.sqla.manager:DB Creation and initialization failed: (psycopg2.OperationalError) FATAL:  password authentication failed for user "superset"
superset-init_1    | 
superset-init_1    | (Background on this error at: http://sqlalche.me/e/e3q8)
superset-init_1    | Loaded your LOCAL configuration at [/app/pythonpath/superset_config.py]
superset_1         |  * Debugger is active!
superset_1         |  * Debugger PIN: 215-042-490
incubator-superset_superset-init_1 exited with code 1
superset_1         | INFO:root:logging was configured successfully
db_1               | 2019-12-21 03:13:10.122 UTC [36] FATAL:  password authentication failed for user "superset"
db_1               | 2019-12-21 03:13:10.122 UTC [36] DETAIL:  Role "superset" does not exist.
db_1               |    Connection matched pg_hba.conf line 95: "host all all all md5"
superset_1         | ERROR:flask_appbuilder.security.sqla.manager:DB Creation and initialization failed: (psycopg2.OperationalError) FATAL:  password authentication failed for user "superset"
superset_1         | 
superset_1         | (Background on this error at: http://sqlalche.me/e/e3q8)
Killing incubator-superset_superset-node_1    ... done
Killing incubator-superset_superset-worker_1  ... done
Killing incubator-superset_superset_1         ... done
Killing incubator-superset_db_1               ... done
Killing incubator-superset_redis_1            ... done

Screenshots

N/A

How to reproduce the bug

git clone https://github.com/apache/incubator-superset/
cd incubator-superset
docker-compose up

Environment

(please complete the following information):

  • superset version: Unknown, running in docker. Git commit is commit 36c6f4ca3adfb237e1a4931563b782a4d7e7932a
  • python version: Docker Compose
  • node.js version: Docker Compose
  • npm version: Docker Compose

Checklist

Make sure these boxes are checked before submitting your issue - thank you!

  • [x ] I have checked the superset logs for python stacktraces and included it here as text if there are any.
  • [x ] I have reproduced the issue with at least the latest released version of superset.
  • [ x] I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context

N/A

#bug .docker
Source
picture rvictory
👍1

Most helpful comment

Issue-Label Bot is automatically applying the label #bug to this issue, with a confidence of 0.78. Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

picture issue-label-bot[bot] on 21 Dec 2019
👍2

All 7 comments

Issue-Label Bot is automatically applying the label #bug to this issue, with a confidence of 0.78. Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

issue-label-bot[bot] picture issue-label-bot[bot] on 21 Dec 2019
👍2

+1 same issue here

gwaldvogel picture gwaldvogel on 21 Dec 2019

I have found a temporary workaround:
Modify docker/.env, change:

# database engine specific environment variables
# change the below if you prefers another database engine
DATABASE_PORT=5432
DATABASE_DIALECT=postgresql
POSTGRES_DB=${DATABASE_DB}
POSTGRES_USER=${DATABASE_USER}
POSTGRES_PASSWORD=${DATABASE_PASSWORD}

to

# database engine specific environment variables
# change the below if you prefers another database engine
DATABASE_PORT=5432
DATABASE_DIALECT=postgresql
POSTGRES_DB=superset
POSTGRES_USER=superset
POSTGRES_PASSWORD=superset

Looks like the variables do not properly work in this situation. Beware: You also have to manually delete the db_home docker volume for the changes to have any effect.

gwaldvogel picture gwaldvogel on 21 Dec 2019

Thank you for reporting. I can replicate the issue. Created a temporary fix, tried to retain the variable indirection but it does not seem possible.

dpgaspar picture dpgaspar on 22 Dec 2019

I do not think this is an issue. If you use docker-compose.yml file, you should at least look into what you are using in there, what are the parameters expected.

nabinkhadka picture nabinkhadka on 22 Dec 2019

Fixed with #8882

dpgaspar picture dpgaspar on 22 Dec 2019

I'm currently facing the same problem and I've the correct .env file. The only think I changed was the port in the db (inside the docker-compose.yml to 5432:5432).
@dpgaspar, could you reopen this issue?

OS: Ubuntu 18.04.3 LTS x86_64

docker-compose.yml

  db:
    env_file: docker/.env
    image: postgres:10
    restart: unless-stopped
    ports:
      - "5432:5432"
    volumes:
      - db_home:/var/lib/postgresql/data

ENV file:

# database configurations (do not modify)
DATABASE_DB=superset
DATABASE_HOST=db
DATABASE_PASSWORD=superset
DATABASE_USER=superset

# database engine specific environment variables
# change the below if you prefers another database engine
DATABASE_PORT=5432
DATABASE_DIALECT=postgresql
POSTGRES_DB=superset
POSTGRES_USER=superset
POSTGRES_PASSWORD=superset

Error LOG:
redis_1 | 1:M 27 Jan 18:54:34.032 * The server is now ready to accept connections on port 6379 db_1 | selecting dynamic shared memory implementation ... posix db_1 | creating configuration files ... ok db_1 | running bootstrap script ... ok db_1 | performing post-bootstrap initialization ... ok db_1 | syncing data to disk ... ok db_1 | db_1 | Success. You can now start the database server using: db_1 | db_1 | pg_ctl -D /var/lib/postgresql/data -l logfile start db_1 | db_1 | db_1 | WARNING: enabling "trust" authentication for local connections db_1 | You can change this by editing pg_hba.conf or using the option -A, or db_1 | --auth-local and --auth-host, the next time you run initdb. db_1 | waiting for server to start....2020-01-27 18:54:38.348 UTC [46] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" db_1 | 2020-01-27 18:54:38.830 UTC [47] LOG: database system was shut down at 2020-01-27 18:54:36 UTC db_1 | 2020-01-27 18:54:38.854 UTC [46] LOG: database system is ready to accept connections db_1 | done db_1 | server started db_1 | CREATE DATABASE db_1 | db_1 | db_1 | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/* db_1 | db_1 | 2020-01-27 18:54:39.936 UTC [46] LOG: received fast shutdown request db_1 | waiting for server to shut down....2020-01-27 18:54:40.263 UTC [46] LOG: aborting any active transactions db_1 | 2020-01-27 18:54:40.266 UTC [46] LOG: worker process: logical replication launcher (PID 53) exited with exit code 1 db_1 | 2020-01-27 18:54:40.266 UTC [48] LOG: shutting down db_1 | 2020-01-27 18:54:40.495 UTC [46] LOG: database system is shut down db_1 | done db_1 | server stopped db_1 | db_1 | PostgreSQL init process complete; ready for start up. db_1 | db_1 | 2020-01-27 18:54:40.590 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432 db_1 | 2020-01-27 18:54:40.590 UTC [1] LOG: listening on IPv6 address "::", port 5432 db_1 | 2020-01-27 18:54:40.624 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" db_1 | 2020-01-27 18:54:40.707 UTC [64] LOG: database system was shut down at 2020-01-27 18:54:40 UTC db_1 | 2020-01-27 18:54:40.732 UTC [1] LOG: database system is ready to accept connections db_1 | 2020-01-27 18:54:44.357 UTC [71] FATAL: password authentication failed for user "postgres" db_1 | 2020-01-27 18:54:44.357 UTC [71] DETAIL: Role "postgres" does not exist. db_1 | Connection matched pg_hba.conf line 95: "host all all all md5" superset-init_1 | INFO:root:logging was configured successfully superset_1 | * Serving Flask app "superset.app:create_app()" (lazy loading) superset_1 | * Environment: development superset_1 | * Debug mode: on superset_1 | * Running on http://0.0.0.0:8088/ (Press CTRL+C to quit) superset_1 | * Restarting with stat superset_1 | INFO:root:logging was configured successfully superset_1 | * Debugger is active! superset_1 | * Debugger PIN: 131-988-229

SWoto picture SWoto on 27 Jan 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

ylkjick532428 picture ylkjick532428  路  3Comments
gbrian picture gbrian  路  3Comments
josephtyler picture josephtyler  路  3Comments
kalimuthu123 picture kalimuthu123  路  3Comments
tmccartan picture tmccartan  路  3Comments