Delta: [DML] Add Python APIs for Delta DMLs and other operations

[tdas]

Other issues track the Scala/Java APIs for Merge, Delete, etc. This issue track adding Python APIs for these operations. We will be using Py4j to call into the Scala APIs.

[ronflomin]

Do you have any estimated date that you will submit a pr for merge with Python API?

[tdas]

We are a little busy with the 0.3.0 release this week. After that, maybe in a couple of weeks?
Glad to see that you are excited about this :)

[kkr78]

hoping to integrate the python API into our data pipelines.

[AFFogarty]

Is there a fork where this is being worked on?

[rahulsmahadev]

Hi @AFFogarty ,

We are working on Python API support right now and expect it to be ready in time for the next release. We do not have a fork which has the python APIs but we will be pushing our code to master as and when they are completed.

Thanks for your interest

[AFFogarty]

@rahulsmahadev Is there an ETA?

[rahulsmahadev]

By the end of next week we want to have a Pull Request out hopefully and merged shortly thereafter.

[ronflomin]

Is there an estimated time for the pr?

[tdas]

This is being done in parts across a few PRs. The first one got merged just now in commit https://github.com/delta-io/delta/commit/7e76205a93c28637072e17c1365af5a5009f999e . It only has the basic DeltaTable class but none of the actual operations. Those should come as a couple of more PRs over the next week. Eventually, we are hoping to release it in 0.4.0 by end of this month..

[Praxkv1]

@tdas @rahulsmahadev :- I am really exited about delta lake and waiting for the python support for upsert . Hope to see this features in next release.

[tdas]

Here is my PR for Python APIs for merge, update, delete - https://github.com/delta-io/delta/pull/182/files

and vacuum, history and convert to delta has already been merged - https://github.com/delta-io/delta/commit/99ad0d130f2833f2c8042a5ec90615c4e148009c

[tdas]

Python APIs for update, delete, merge has been merged in https://github.com/delta-io/delta/commit/888f78d850e84e3a4198889a18698acca82434ec

We are preparing for the 0.4.0 release.