Airflow: Detect automatically the lack of reference to the guide in the operator descriptions

[mik-laj]

Hello,

We have a lot of guides that are fantastic.
https://airflow.readthedocs.io/en/latest/howto/operator/index.html
https://airflow.readthedocs.io/en/latest/howto/operator/gcp/index.html
However, not every person may be aware that they exist. In order to promote them, we add a reference in the class description to the guide.

Finding operators that have a guide is simple and can be based on the text pattern search.

find . -type f -name '*.rst' | xargs cat  | grep '.. _howto/operator:' | sort

In the next step we should find the operator and check the class description for reference to the guide

>>> from airflow.operators import python
>>> ':ref:`howto/operator:PythonOperator`' in python.PythonOperator.__doc__

This test should be added to the codebase::

• if you can do it without loading the python modules (text only operation), because the documentation process assumes that you don't have to have all libraries installed, then in this file:
  https://github.com/apache/airflow/blob/master/docs/build
• If you have to load Python modules, , then in this file:
  https://github.com/apache/airflow/blob/master/tests/test_project_structure.py

[ivan-afonichkin]

Hello @mik-laj 👋

I would love to start contributing to airflow. So, would you mind assigning this issue to me?

[mik-laj]

Hello @ivan-afonichkin

Fantastic. I am very happy that we have a lot of new contributors. If you have any questions, ask.

A lot of health in this difficult time,
Kamil

[ivan-afonichkin]

Hello @mik-laj, I created PR to tackle this issue using the first approach you suggested (make it directly through docs/build without importing any modules). Would you mind checking and leave some comments on PR? Thanks in advance.

https://github.com/apache/airflow/pull/9290

[mik-laj]

@ivan-afonichkin I will try to look at it on Monday. I am very happy that this has been achieved. I did not think that so many problems could be detected.