Docker-airflow: Airflow variables set via command line don't appear in webserver's UI

[dfdx]

Scenario 1:

1. Run Airflow docker image using docker-compose file similar to the example docker-compose-LocalExecutor (the only difference is FERNET_KEY). This including running default entrypoint.sh including airflow scheduler and airflow webserver commands.
2. Run airflow initdb in the running container.
3. Set a few Airflow variables in the running container.

Scenario 2:

1. Run Airflow docker image without scheduler and webserver.
2. Run airflow initdb.
3. Set a few Ariflow variables in the running container.
4. Start scheduler and webserver in the running container.

In both scenarios webserver starts fine, but when I navigate to Admin -> Variables, I never see my variables. Also any job started from webserver's UI complain that my variables don't exist, although tasks launched using airflow run ... directly (again, from the running container) finish just fine.

All in all it looks like webserver is disconnected from the rest of Airflow. Is it a bug or a misconfiguration on my side?

[PedramNavid]

I'm having a similar issue. I used entrypoint.sh to add variables. If I run airflow variables on the webserver container, I get an error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    context)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
    cursor.execute(statement, parameters)
sqlite3.OperationalError: no such table: log

But I'm using docker-compose with LocalExecutor so it shouldn't be reading sqlite. I think what's happening is the variables we set when airflow runs is not set for when I run docker exec causing weird issues like this. Not sure how to work around that though.

[dfdx]

Almost nailed it down. It turns out whatever you export with entrypoint.sh - including variables like AIRFLOW__CORE__EXECUTOR and AIRFLOW__CORE__SQL_ALCHEMY_CONN - aren't available when you just run docker exec [...] /bin/bash. Neither airflow.cfg is updated. As a result, the following:

docker exec airflow_webserver_1 bash -l -c "airflow variables -s key value"

are executed in a different context, with SequentialExecutor and SQLite instead of Local executor and PostgreSQL.

@PedramNavid thanks, without your commented I wouldn't pay attention to difference in database in use :)

[dfdx]

Fixed by adding these environment variables to environment section of docker compose file. The full definition looks like this now:

      version: '2.1'
      services:
          postgres:
              image: postgres:9.6
              environment:
                  - POSTGRES_USER=airflow
                  - POSTGRES_PASSWORD=airflow
                  - POSTGRES_DB=airflow

          webserver:
              image: puckel/docker-airflow:1.10.1
              restart: always
              depends_on:
                  - postgres
              environment:
                  - LOAD_EX=y
                  - EXECUTOR=Local
                  - FERNET_KEY="<generated fernet key>"
                  # the following will override defaults in airflow.cfg
                  # so we could run `airflow` commands later
                  - POSTGRES_HOST=postgres
                  - POSTGRES_PORT=5432
                  - POSTGRES_PASSWORD=airflow
                  - POSTGRES_DB=airflow
                  - AIRFLOW__CORE__SQL_ALCHEMY_CONN=postgresql+psycopg2://airflow:airflow@postgres:5432/airflow
                  - AIRFLOW__CORE__EXECUTOR=LocalExecutor
              volumes:
                 # had to use absolute path to make it work with ansible inline defintion
                  - /home/ubuntu/dags:/usr/local/airflow/dags
                  # Uncomment to include custom plugins
                  # - ./plugins:/usr/local/airflow/plugins
              ports:
                  - "8080:8080"
              # command: webserver   # I start webserver manually later, but it might be enough to uncomment this line 
              healthcheck:
                  test: ["CMD-SHELL", "[ -f /usr/local/airflow/airflow-webserver.pid ]"]
                  interval: 30s
                  timeout: 30s
                  retries: 3

I then run the following on the host machine to set Airflow variables inside the running container:

docker exec airflow_webserver_1 bash -l -c "airflow variables -s {{ item }}"

[PedramNavid]

This looks related to #290.. adding the FERNET_KEY explicitly in the config should fix that problem as well..

[SolbiatiAlessandro]

This looks related to #290.. adding the FERNET_KEY explicitly in the config should fix that problem as well..

Regarding that issue for me setting

AIRFLOW__CORE__FERNET_KEY="<generated-fernet-key>"

worked, instead of FERNET_KEY