Airflow: .airflowigore ignores files with "latest" in the name

Created on 16 Apr 2020  路  3Comments  路  Source: apache/airflow

Apache Airflow version:
1.10.10

Environment:

  • Ubuntu 19.10 (5.3.0-46-generic)
  • Python 3.7.5

What happened:

I ran into a strange issue with Airflow today.
I created a DAG whose file name ended with "_latest.py", and the dag did not show up after running airflow list_dags.
After replacing "_latest.py" with something else ("_2.py") the dag showed up.

When i removed .airflowignore everything works as expected. Both dags show up.
Files were not in any of the folders listed in the .airflowignore.

Content of .airflowignore does not mention anything about "latest".
It just has these two lines:

test
venv/

bug

All 3 comments

Thanks for opening your first issue here! Be sure to follow the issue template!

I'm willing to bet it's because test is matching in latest (latest)! If you have a test folder, try changing test in your .airflowignore to test/ or making your test search more specific.
More info on how the .airflowignore works: https://airflow.apache.org/docs/stable/concepts.html#airflowignore

You would win that bet :)

Thanks for the help.

I obviously was not aware that .airflowignore is so eager (i.e. did not read the manual :))
It's a bit unexpected that a pattern like test would exclude _latest.
Thought it would be more like .gitignore and require something like *test*.

In any case, it is documented and it behaves as documented.

Was this page helpful?
0 / 5 - 0 ratings