Incubator-superset: Configure to use Google BigQuery

[k-hashimoto]

Hi,

I started to use superset and try to load data on BigQuery to superset on docker container. But I could not. Give me advices?

Cheers,
Kimiaki

• [x ] I have checked the superset logs for python stacktraces and included it here as text if any
• [ x] I have reproduced the issue with at least the latest released version of superset
• [x ] I have checked the issue tracker for the same issue and I haven't found one similar

Superset version

"version": "0.25.5"

Expected results

Can connect to bigquery on "Test Connection" in SQLAlchemy URI.

Actual results

ERROR: {"error": "Connection failed!\n\nThe error message returned was:\nCan't load plugin: sqlalchemy.dialects:bigquery"}

Steps to reproduce

1. docker run -d -p 8088:8088 amancevice/superset
2. docker exec -it CONTAINER ID bash
3. (on running container) pip3 install pybigquery
4. docker exec -it CONTAINER ID superset-init
5. After that, access to locahost:8088 and login, move to "Sources -> Database -> Add a new record"
6. Entry URL into text box("SQLAlchemy URI"). In my case bigquery://XXXX
   XXXX mean project id of BQ.

[mistercrunch]

You need to pip install pybigquery

[k-hashimoto]

Instead of pip3 install pybigquery ?

[mistercrunch]

Oh idk how this environment is setup, but you need pybigquery in your Python environment.

[k-hashimoto]

This is short update. I built docker container to install pybigquery. I modified original dockerfile to build on my laptop. Modified version is here.

I could build and run container, create login user successfully and try again. But I got other error message when I pressed "Test Connection" button.

ERROR: {"error": "Connection failed!\n\nThe error message returned was:\n(setuptools 33.1.1 (/usr/lib/python3/dist-packages), Requirement.parse('setuptools>=34.0.0'), {'google-api-core'})"}

[aaronbannin]

@k-hashimoto that looks like pybigquery also requires google-api-core, which seems logical. Have you tried creating a venv with a basic connection script? This would help you identify all dependancies. Be sure to query and fetch data to ensure everything is flowing.

As you mentioned, you need to include all dependancies in the Dockerfile. Another approach would be setting up a venv with pip install superset.

[k-hashimoto]

@aaronbannin According to your advise, I modified my Dockerfile to use venv(latest version is here) and success to login BQ from superset. Thanks!