Odo: Support Kubernetes build and deploy support in odo

[neeraj-laad]

/kind feature

Which functionality do you think we should add?

The following proposal adds support for building a container image and deploying it:
https://github.com/openshift/odo/blob/master/docs/proposals/odo-deploy.md

There is a PR open to supporting this capability on OpenShift: https://github.com/openshift/odo/pull/3478

We should add support for any Kubernetes as the target. At a high level this will involve:

• detect when we are not running on Openshift
• use buildah or kaniko for building the container image
• accept image registry credentials from the user as an argument and use it to push the built image
• support Ingress while detecting the deployed application URL

Why is this needed?

Provide generic Kubernetes support for odo deploy

[neeraj-laad]

@sbose78 @wtam2018 @jaideepr97 - Lets keep all discussions on Kubernetes path for odo deploy here.

As @EnriqueL8 has mentioned, you can use some aspects from https://github.com/EnriqueL8/odo/tree/buildah_deploy. This was the work we had started.

We started looking at buildah first , but were not sure if we could use it without privileged access, were looking to make a switch to kaniko.

[groeges]

Raising this comment as a potential change that might be needed when supporting Ingress on Kube (and/or OCP).

This was something we found when working on the odo deploy when attempting to use a deployment manifest that contained an Ingress, ie

kind: Ingress
apiVersion: networking.k8s.io/v1beta1
metadata:
  name: {{.COMPONENT_NAME}}
spec:
  rules:
    - host: {{.COMPONENT_NAME}}-{{.NAMESPACE}}-{{.ROUTE_SUFFIX}}
      http:
        paths:
          - path: /
            backend:
              serviceName: {{.COMPONENT_NAME}}
              servicePort: {{.PORT}}-tcp

The spec.rules.host seems to need to have a host value defined in order to correctly setup the remote URL.

The above format was that which seems to be used on OCP but I dont know what is required on Kube.

This may be OCP specific as I have only tried this on Kube running in my Docker Desktop (on Mac) and that Kube setup didn't need the host and just used localhost. Just highlighting this just in case it is needed on a full Kube system.

The {{.COMPONENT_NAME}} and {{.NAMESPACE}} template variables are already available within odo deploy but the {{.ROUTE_PREFIX}} is not easily obtainable (specifically from OCP - not sure about Kube).

There might be a need to include a --host (or some suitable named flag) on the odo deploy command in order to provide this info when using an Ingress.

[jaideepr97]

@neeraj-laad @EnriqueL8 @groeges @sbose78 @wtam2018

Hey folks,
At the moment, we are leaning more towards going with Buildah as the first build strategy to support for the following reasons:

• RedHat has an officially supported image on registry.redhat.io whereas this is not the case for Kaniko
• If needed we can reach out to the Buildah team to request changes/features that align with our requirements, whereas it would
  not be so with Kaniko
• Buildah is working towards supporting unprivileged builds in the foreseeable future
• Buildah has better compatibility with service accounts and it is relatively easier for buildah to read creds from the associated
  service account the pod is running as

If there isn't an immediate need for unprivileged builds in our collective vision for what odo deploy should be doing in the near future, we think starting with Buildah makes sense for now. We could work to add Kaniko support at a later stage -- and we'll continue to to work with the team to make unprivileged builds using Buildah work in the coming months

But it would be useful to get everybody's perspective, and understanding the nature of the use cases or other considerations being made on your side would help drive the decision that would be more in line with our vision

cc @Shraddhak22 @ranakan19 @reginapizza

[sbose78]

+1

We started looking at buildah first , but were not sure if we could use it without privileged access,

Note, there are few other places we are trying to get buildah unprivileged to work
https://github.com/redhat-developer/build/issues/134#issuecomment-619494883

[neeraj-laad]

Sounds good to me. Our preference was to start with buildah too. The only reason for switching from buildah to kaniko was need for privileged access. As long as we can solve that, this seems like a good plan.

[neeraj-laad]

@jaideepr97 I'm keen to hear about your plans on handling registry credentials. Have you started thinking about that yet?

[jaideepr97]

@jaideepr97 I'm keen to hear about your plans on handling registry credentials. Have you started thinking about that yet?

@neeraj-laad haven't gotten into the weeds yet, but the initial idea is that the --credentials flag in odo deploy would point to the location of the dockerconfig file with external registry credentials, and odo deploy could leverage that file to create a secret in the specified namespace before spinning up the builder pod
The Buildah container in the pod can then access this secret through a volume mount and use it to push the image to the external registry

for internal registries I suppose obtaining the builder SA's credentials shouldn't be an issue

[neeraj-laad]

@jaideepr97 you might want to assign this issue to yourself, so odo team knows this is coming from you. Perhaps even put it under right milestone etc.

[jaideepr97]

@jaideepr97 you might want to assign this issue to yourself, so odo team knows this is coming from you. Perhaps even put it under right milestone etc.

@neeraj-laad will do, thanks for all your help!

[sbose78]

/assign jaideepr97

@kadel

[Shraddhak22]

@neeraj-laad @EnriqueL8 @groeges,

Hello Folks,
After further discussion with team and research on buildah unprivileged build, we have decided to go ahead with,

1. Buildah strategy driven by BuildConfig on OpenShift clusters.
2. Kaniko dockerfile strategy for Kubernetes
   Please let us know your thoughts.

cc @sbose78 @wtam2018 @jaideepr97 @reginapizza @ranakan19

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[openshift-bot]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

[openshift-bot]

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

[openshift-ci-robot]

@openshift-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.