Incubator-superset: Dockerfile does not install browser and web driver, preventing emailer from working

[Fingerzam]

Cannot use emailer feature using a container built with contrib/docker/Dockerfile.

Expected results

Emailing dashboards and charts to work with a container built with the provided Dockerfile.

Actual results

Emailer logs about missing browser and/or driver:

 [2019-05-07 14:58:30,786: ERROR/ForkPoolWorker-1] Task email_reports.send[1ee0a6a2-fa7d-47d2-86dc-8bba91da4066] raised unexpected: WebDriverException("'geckodriver' executable needs to be in PATH. ",)
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/selenium/webdriver/common/service.py", line 76, in start
    stdin=PIPE)
  File "/usr/local/lib/python3.6/subprocess.py", line 729, in __init__
    restore_signals, start_new_session)
  File "/usr/local/lib/python3.6/subprocess.py", line 1364, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'geckodriver': 'geckodriver'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/celery/app/trace.py", line 382, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/celery/app/trace.py", line 641, in __protected_call__
    return self.run(*args, **kwargs)
  File "/home/superset/superset/tasks/schedules.py", line 392, in schedule_email_report
    deliver_dashboard(schedule)
  File "/home/superset/superset/tasks/schedules.py", line 219, in deliver_dashboard
    driver = create_webdriver()
  File "/home/superset/superset/tasks/schedules.py", line 168, in create_webdriver
    driver = driver_class(**kwargs)
  File "/usr/local/lib/python3.6/site-packages/selenium/webdriver/firefox/webdriver.py", line 164, in __init__
    self.service.start()
  File "/usr/local/lib/python3.6/site-packages/selenium/webdriver/common/service.py", line 83, in start
    os.path.basename(self.path), self.start_error_message)
selenium.common.exceptions.WebDriverException: Message: 'geckodriver' executable needs to be in PATH. 

Screenshots

If applicable, add screenshots to help explain your problem.

How to reproduce the bug

1. build a container with contrib/docker/Dockerfile
2. set required settings for emailer
3. click Manage
4. click Dashboard Emails
5. make an email schedule (e.g. crontab 5 12 * * *) , check Send Test Email and add your email to Test Email Recipients and Recipients
6. click save
7. check worker logs for errors

Environment

• superset version v0.32.0rc2
• python, node and npm versions as in the Dockerfile

Checklist

Make sure these boxes are checked before submitting your issue - thank you!

• [X] I have checked the superset logs for python stacktraces and included it here as text if there are any.
• [X] I have reproduced the issue with at least the latest released version of superset.
• [X] I have checked the issue tracker for the same issue and I haven't found one similar.

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the label #bug to this issue, with a confidence of 0.91. Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[Fingerzam]

This issue can be fixed by adding

RUN apt-get update && apt-get install -y firefox-esr

RUN wget https://github.com/mozilla/geckodriver/releases/download/v0.24.0/geckodriver-v0.24.0-linux64.tar.gz
RUN tar -x geckodriver -zf geckodriver-v0.24.0-linux64.tar.gz -O > /usr/bin/geckodriver
RUN chmod +x /usr/bin/geckodriver
RUN rm geckodriver-v0.24.0-linux64.tar.gz

to the Dockerfile

[mistercrunch]

I'd suggest layering it in another image to avoid bloating the main image.

[Fingerzam]

It seems to add 400 MB. Current image is 1.64 GB and with these additions ends up at 2.04GB

[mistercrunch]

Wow that's pretty big. I made a Superset image for internal use that is < 200mb

[theikkila]

Does that internal image contain also the webdriver?
Emailer is nowadays feature of Superset so probably we should include the webdriver and browser in some distribution for users.

It could be another version or there could be "lite" version for those users that are not using the emailer-feature

[mistercrunch]

It does not but it could be layered on top of another derived image. Generally it'd be great to have a lightweight base image that is built upon to create heavier builds with more assumptions.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.

[PRamia]

Wow that's pretty big. I made a Superset image for internal use that is < 200mb

Hi,

could you share that image with the community? i am looking for something like that.

[klazaj]

@Fingerzam If I add those lines I get the error: Task email_reports.send[eda46911-df86-4927-b9bf-56c09e3aa277] raised unexpected: WebDriverException('Reached error page: about:neterror?e=connectionFailure&u=http%3A//0.0.0.0%3A8080/superset/welcome&c=UTF-8&f=regular&d=Firefox%20can%E2%80%99t%20establish%20a%20connection%20to%20the%20server%20at%200.0.0.0%3A8080.', None, None)

[aleksarias]

I got the same issue as @klazaj

[dextermapalya]

I get a similar message when I try to schedule the emails.
Is there any fix for it. I am using incubator-superset dockerfile

[2020-07-28 12:05:49,662: ERROR/ForkPoolWorker-1] Task email_reports.send[ebfc5f8a-7eb0-42e2-a2b1-831bb1f2faa4] raised unexpected: PermissionError(13, 'Permission denied')
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/celery/app/trace.py", line 412, in trace_task
R = retval = fun(args, kwargs)
File "/app/superset/app.py", line 115, in __call__
return task_base.__call__(self, *args, *kwargs)
File "/usr/local/lib/python3.6/site-packages/celery/app/trace.py", line 704, in __protected_call__
return self.run(args, *kwargs)
File "/app/superset/tasks/schedules.py", line 518, in schedule_email_report
schedule.deliver_as_group,
File "/app/superset/tasks/schedules.py", line 289, in deliver_dashboard
driver = create_webdriver()
File "/app/superset/tasks/schedules.py", line 227, in create_webdriver
driver = driver_class(kwargs)
File "/usr/local/lib/python3.6/site-packages/selenium/webdriver/firefox/webdriver.py", line 163, in __init__
log_path=service_log_path)
File "/usr/local/lib/python3.6/site-packages/selenium/webdriver/firefox/service.py", line 44, in __init__
log_file = open(log_path, "a+") if log_path is not None and log_path != "" else None
PermissionError: [Errno 13] Permission denied: 'geckodriver.log'