Airflow: Create guide for Dataflow operators

[mik-laj]

Description

Hello,

A guide that describes how to use Dataflow service operators would be useful.

We have an example DAG for this service, so the guide should not be a big challenge.

If anyone is interested in this task, I am willing to provide all the necessary tips and information.

Other guides are available:
https://airflow.readthedocs.io/en/latest/howto/operator/index.html

All example DAGs for GCP are available in the example_dags directory.

Best regards,

Kamil

Use case / motivation

N/A

Related Issues

Missing guide for GCP: https://github.com/apache/airflow/issues/8209

[tanjinP]

I can take this on - this is a service I've worked heavily in.

[mik-laj]

@tanjinP Fantastic. I am working on this integration now with the Dataflow team, so this guide would be very helpful. I saw that you applied for 3 services, but then this service is the most important. I will be happy to share my thoughts on this integration and together with you will develop this guide. I am sure that the Dataflow team will be willing to review this guide as well. Dataflow is a hot topic right now, so I'm very happy.

[mik-laj]

@tanjinP I would be happy if you added information that asynchronous execution is recommended.
https://cloud.google.com/dataflow/docs/guides/specifying-exec-params#python_8
I would like to create sensors that will allow more efficient use of resources.