Pipelines: Release the SDK in Pypi and automate the release process

[andrewsmartin]

Hi, is it possible to publish releases to pypi? I saw that there is https://pypi.org/project/kfp/ but the latest version is 0.1.11.

[neuromage]

Yes definitely. I uploaded the latest version (0.1.16) to PyPi just now.

Ideally, we should look to push new wheels monthly, or whenever we have major changes in SDK. Weekly is probably much too frequent. If so, we can make it the duty of the release operator. What do you think @vicaire @IronPan @Ark-kun @gaoning777 ?

[vicaire]

Should we have a push to PyPi for each release of KFP? Otherwise the PyPi release and the KFP release would be out of sync.

I agree this should be part of the task of the release operator.

[Ark-kun]

Ideally, we should look to push new wheels monthly, or whenever we have major changes in SDK. Weekly is probably much too frequent. If so, we can make it the duty of the release operator. What do you think @vicaire @IronPan @Ark-kun @gaoning777 ?

I agree that it should be a part of the release process.
I think we should push each release to PyPI. Currently I do not think we have a reason not to push some of the releases.

[neuromage]

Just had an offline discussion with @gaoning777 and @Ark-kun . We'll be looking into pushing each week's release to PyPi from here on. Let's us this issue to track the progress on automating this effort.

[andrewsmartin]

Hello! Just wondering if there are any updates on this. The latest version on pypi is 0.1.19.

[gaoning777]

It should already be updated along with the pipeline release.
If not, please file a bug. Thanks

[Ark-kun]

The instructions are in the release playbook.
The problem with automating it is passing the PyPI password to twine or authenticating some other way.