Charts: [stable/airflow] Optionally mount `requirements.txt` for `entrypoint.sh` consumption

[raphaelsampaio]

Is this a request for help?: Maybe

---

Is this a BUG REPORT or FEATURE REQUEST? (choose one):
FEATURE REQUEST

Version of Helm and Kubernetes:
not relevant

Which chart:
stable/airflow

What happened:
There is no way -- except from modifying the chart -- to mount a requirements.txt in the default Docker image (puckel/docker-airflow).

What you expected to happen:
Maybe it would be valuable to modify the chart and add an option in values.yaml to mount requirements.txt in the default Docker image so that the entrypoint.sh would run pip install --user -r /requirements.txt. If that was available, users who choose to deploy DAGs via a shared persistent volume would still be able to install any Python dependencies needed by their DAGs (for example kubernetes for the KubernetesPodOperator to work).

How to reproduce it (as minimally and precisely as possible):
Just install Airflow on a Kubernetes cluster with any DAGs needing some Python dependency which is not installed (for example a DAG using a KubernetesPodOperator without the kubernetes lib). The DAG will fail and you'll have to modify the chart to run pip install or switch from the default image to another one with Python dependencies pre-installed.

Anything else we need to know:
Of course there are workarounds to this problem, but I don't think any of them are as nice as having that capability native to the chart. If I'm missing anything and there is indeed a native way of doing that without modifying the chart, then please let me know :)

[antweiss]

Well, in fact, as I found today - it's sufficient to put a requirements.txt at the root of your dags directory and the init container will run it.

[raphaelsampaio]

Are you saying that placing a requirements.txt at the root of the dags directory works even for a shared persistent volume strategy, i.e. not only for a git-init strategy?

[gsemet]

hi. Yes place the requirements.txt in the root of your DAG folder and it will be "pip install"ed automatically. (see initcontainer docs). Feel free to enhance the README of this chart if it is not enough well described. Install is done by this script.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Any further update will cause the issue/pull request to no longer be considered stale. Thank you for your contributions.

[gsemet]

Hi. Have you found a solution, can we close this ticket?

[raphaelsampaio]

Sure @gsemet! Thanks to you and @antweiss for the help 👍

[eugenelopatkin]

Placing requirements.txt to DAGs root doesn't work with Shared Persistance Volume because /usr/local/scripts/install-requirements.sh && appends to deployment only if .Values.dags.initContainer.enabled is enabled. I think it is bug.