Airflow: Make extras install corresponding provider packages.

[potiuk]

When you install Aiflow with an extra, the required provider packages should be installed automatically.

For example:

pip instal apache-airflow[google] should install apache-airlfow-providers-google package.

[obarisk]

The apache-airlfow-providers-google package is not on PyPI.

[potiuk]

Yep. Not yet - we are going to release it when we get to Beta level, that's why I am implementing it in the way to work-around it and to be able to work without those packages being released (but it will also have to work in case when we install airflow for development from sources - see the discussion in #11526

[potiuk]

@kaxil - since we agreed that PyPI releases will be done in Beta, I changed the milestone to beta1 for this one.

[potiuk]

Btw. The installation from sources for providers has already been merged @obarisk . See #11529 -> by default when you install Airflow from sources with 'pip install -e .' - this works out-of-the-box, for installing via 'pip install .' you can setINSTALL_PROVIDERS_FROM_SOURCES env variable to "true" and it will install the providers together with airflow.

In Dockerfiles, when you install them from sources, all providers will be installed by default, but when you install Airflow from PyPI or binary packages, you will have to manually add all providers that are needed.

I think it really nicely balances the development vs. production needs of users.

[ashb]

I'll merge this in a moment, one final PR to fix readme of packages coming up.

[mjpieters]

I suspect that the list of provider packages is too broad; there is no apache-airflow-providers-statsd package, for example. Should I file a new bug?

[kaxil]

I suspect that the list of provider packages is too broad; there is no apache-airflow-providers-statsd package, for example. Should I file a new bug?

We have one issue filed for it here: https://github.com/apache/airflow/issues/12255