Airflow: Feature Request - Sqoop hook and operators

[syvineckruyk]

to be defined.

[cgelman]

I want to +1 this.

[chenghaibo]

I want to +1 this

[rakeshsingh]

I want to +1 this

[notifytovishal]

+1 Can this be done using https://pypi.python.org/pypi/sqoopy/ ? Can this be defined !!

[syvineckruyk]

We currently have a functioning SqoopOperator that we use internally. Currently the feature set is based on our need and may be considered rather limited. I would be interested to hear what the community is looking in terms of features from a SqoopOperator.

Currently our Sqoop Operator has the benefit of using the connections from Airflow ... so no need to to have your connection strings and credentials in your code or command parameter. It takes tool (import,export etc...), args, kwargs and templated option file as parameters.

[hyperborea]

@syvineckruyk any chance to get a sneak peak at your current implementation?

[syvineckruyk]

@hyperborea sure thing ... ill get a pr going and update here.

[mafc]

Any updates on this?

[mistercrunch]

https://github.com/apache/incubator-airflow/pull/1489