Pipelines: KFP should support containerd runtime on Kube

Created on 14 Jan 2019 · 9Comments · Source: kubeflow/pipelines

KubeFlow pipeline currently only works on Docker runtime since the pipeline output artifacts are copied with "docker cp" command. We need support for containerd runtime

arebackend arecomponents aredocs help wanted kinfeature lifecyclstale prioritp1

Source

animeshsingh

Most helpful comment

Hey @animeshsingh, with new process namespace sharing (PNS) I got argo to run with containerd on k3s, no docker involved. You can use it by setting a property in the config map as described in #901 here

Does this solve your problem?

Duske on 10 Jun 2019

👍2

All 9 comments

Could you give more context on why it's needed?

We rely on argo to copy the artifact files. However, only docker executor currently supports file operations in argo (https://github.com/argoproj/argo/tree/master/workflow/executor). There is an argo FR to get rid of docker cp and it seems that argo team had a plan but still WIP.

hongye-sun on 18 Jan 2019

@hongye-sun we are moving towards containerd in IBM Cloud IKS, and hence this becomes key
https://www.ibm.com/blogs/bluemix/2019/01/ibm-cloud-kubernetes-service-supports-containerd/

animeshsingh on 18 Jan 2019

Hi folks - coming back to this. Given we have phased out support for docker in IBM Kubernetes service, this is becoming a blocker. Anyway to take this forward jointly with Argo?

animeshsingh on 26 Feb 2019

Thanks for filing the issue with Argo. Would you be able to drive support for containerd with both Argo and then KFP? Let me know if some help is needed.

vicaire on 8 Mar 2019

Thanks @vicaire. Definitely chasing it - there is a PR with Argo which are trying to make it work within our context, and then will be going back with what still remains opens.

animeshsingh on 8 Mar 2019

Does this solve your problem?

Duske on 10 Jun 2019

👍2

For those who have the same issue, add or change containerRuntimeExecutor: k8sapi, to configmap workflow-controller-configmap and restart deployment ml-pipeline work for pipeline version 0.1.39, and this has been fixed in argo: https://github.com/argoproj/argo/pull/1419

MingzhouZhou on 17 Jan 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.