Pipelines: Better Kubernetes version support

[Bobgy]

We've got reports once a while asking us to get compatible with newer Kubernetes versions.

I think we'd better give a support table and periodically verify whether KFP is compatible with newer Kubernetes versions in addition to the stable version we are testing on.

Related to https://github.com/kubeflow/pipelines/issues/4138

[linkvt]

Hi @Bobgy, does that mean kubeflow pipelines is not following the supported Kubernetes version matrix of the whole kubeflow setup? https://www.kubeflow.org/docs/started/k8s/overview/

I would have assumed that this is the case as kubeflow pipelines is documented besides this version overview and seems to follow a similar/the same versioning.

Thanks.

[Bobgy]

Ohh, I didn't know we have a version matrix there. I'm not sure that version matrix is intensively tested for the cells with no known issues. e.g. https://github.com/kubeflow/pipelines/issues/4138 reported an incompatibility with Kubernetes 1.18.

[linkvt]

Oh then at least the good news are, that you guys know about that matrix now :grin: .

For me as an end user this is quite confusing as it seems to me that Kubeflow Pipelines is only a part of the full kubeflow suite, so I was already confused that kubeflow _full_ is in 1.1RC and you are still in 1.0RC.

As a k8s admin this are also not very good news for me, as kubeflow held us back quite a lot with its slow support of uptodate Kubernetes versions forcing us for quite some time on unsupported versions.
I was really hoping that kubeflow pipelines as a way less complex tool compared to the full kubeflow suite was a bit less restrictive regarding this topic.

I would really appreciate at least a "Supported" status and not a "No known issues" status regarding the current officially supported kubernetes versions: https://kubernetes.io/docs/setup/release/version-skew-policy/#supported-versions

Thanks again for the efforts spent in these topics!

[Bobgy]

For me as an end user this is quite confusing as it seems to me that Kubeflow Pipelines is only a part of the full kubeflow suite, so I was already confused that kubeflow full is in 1.1RC and you are still in 1.0RC.

FYI, individual components are versioned separately from Kubeflow.

[linkvt]

@Bobgy thanks so far, is there some documentation on the latest supported k8s version for kubeflow-pipelines yet? Because we currently have no clue if we can upgrade our clusters or if KFP would be a blocker, thanks.

[Bobgy]

We are going to release KFP 1.0.1, it's supposed to support k8s 1.18 now.

[Bobgy]

Multiple people have tried 1.18 and we fixed reported problem, verified they work.

There is no automated testing on 1.18 yet.

This is status quo

[Bobgy]

We have been having quite some issues with Kubernetes version support recently, it will be ideal if we could set up periodic tests for supported Kubernetes versions and monitor them over time.

Off top of my head, I think we should support 1.14~latest versions.

With the periodic test matrix set up, we can update a Kubernetes version compatibility table for major versions we support to further communicate the information.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.