Odo: Publish the default Devfiles registry to support IBM Z&Power client issue.

[zhengxiaomei123]

/kind bug

What versions of software are you using?

Operating System:
Red Hat Enterprise Linux 8.2 (Ootpa)
Output of odo version:
odo v2.0.1 (30d947850)

How did you run odo exactly?

use odd binary directly on cop 4.6

Actual behavior

odo catalog list components

Odo Devfile Components:
NAME                 DESCRIPTION                            REGISTRY
java-maven           Upstream Maven and OpenJDK 11          DefaultDevfileRegistry
java-openliberty     Open Liberty microservice in Java      DefaultDevfileRegistry
java-quarkus         Upstream Quarkus with Java+GraalVM     DefaultDevfileRegistry
java-springboot      Spring Boot® using Java                DefaultDevfileRegistry
java-vertx           Upstream Vert.x using Java             DefaultDevfileRegistry
nodejs               Stack with NodeJS 12                   DefaultDevfileRegistry
python               Python Stack with Python 3.7           DefaultDevfileRegistry
python-django        Python3.7 with Django                  DefaultDevfileRegistry

Odo OpenShift Components:
NAME                                                 PROJECT       TAGS                                                                                              SUPPORTED
java                                                 openshift     [openjdk-11](https://issues.redhat.com/browse/openjdk-11)-el7,[openjdk-8](https://issues.redhat.com/browse/openjdk-8)-el7                                                                      YES
nodejs                                               openshift     latest                                                                                            YES
dotnet                                               openshift     2.1-el7,2.1-ubi8,3.1-el7,3.1-ubi8,latest                                                          NO
golang                                               openshift     1.13.4-ubi7,1.13.4-ubi8,latest                                                                    NO
httpd                                                openshift     2.4-el7,2.4-el8,latest                                                                            NO
java                                                 openshift     8,latest,openj9-11-el7,openj9-11-el8,openj9-8-el7,openj9-8-el8,[openjdk-11](https://issues.redhat.com/browse/openjdk-11)-ubi8,[openjdk-8](https://issues.redhat.com/browse/openjdk-8)-ubi8     NO
jboss-webserver53-openj9-tomcat9-rhel7-openshift     openshift     1.0,latest                                                                                        NO
nginx                                                openshift     1.14-el8,1.16-el7,1.16-el8,1.18-ubi8,latest                                                       NO
nodejs                                               openshift     10-ubi7,10-ubi8,12-ubi7,12-ubi8                                                                   NO
perl                                                 openshift     5.26-el7,5.26-ubi8,5.30-el7,latest                                                                NO
php                                                  openshift     7.2-ubi8,7.3-ubi7,7.3-ubi8,latest                                                                 NO
python                                               openshift     2.7-ubi7,2.7-ubi8,3.6-ubi8,3.8-ubi7,3.8-ubi8,latest                                               NO
ruby                                                 openshift     2.5-ubi7,2.5-ubi8,2.6-ubi7,2.6-ubi8,2.7-ubi7,latest                                               NO
wildfly                                              openshift     latest

Expected behavior

For Odo Devfile Components on IBM Z&P, only NodeJS is supported now.
For Odo OpenShift Components on IBM Z&P, donet and wildfly is not supported now.

Any logs, error output, etc?

[zhengxiaomei123]

/area system-pz

[sarveshtamba]

Output for Power:-

[root@stamba-odo-aa53-bastion-0 odo]# ./odo catalog list components
Odo Devfile Components:
NAME                 DESCRIPTION                            REGISTRY
java-maven           Upstream Maven and OpenJDK 11          DefaultDevfileRegistry
java-openliberty     Open Liberty microservice in Java      DefaultDevfileRegistry
java-quarkus         Upstream Quarkus with Java+GraalVM     DefaultDevfileRegistry
java-springboot      Spring Boot® using Java                DefaultDevfileRegistry
java-vertx           Upstream Vert.x using Java             DefaultDevfileRegistry
nodejs               Stack with NodeJS 12                   DefaultDevfileRegistry
python               Python Stack with Python 3.7           DefaultDevfileRegistry
python-django        Python3.7 with Django                  DefaultDevfileRegistry

Odo S2I Components:
NAME        PROJECT       TAGS                                                             SUPPORTED
java        openshift     8,latest,[openjdk-11](https://issues.redhat.com/browse/openjdk-11)-el7,[openjdk-8](https://issues.redhat.com/browse/openjdk-8)-el7                            YES
nodejs      openshift     12-ubi8,latest                                                   YES
dotnet      openshift     2.1-el7,2.1-ubi8,3.1-el7,3.1-ubi8,latest                         NO
golang      openshift     1.13.4-ubi7,1.13.4-ubi8,latest                                   NO
httpd       openshift     2.4-el7,2.4-el8,latest                                           NO
java        openshift     [openjdk-11](https://issues.redhat.com/browse/openjdk-11)-ubi8,[openjdk-8](https://issues.redhat.com/browse/openjdk-8)-ubi8                                   NO
nginx       openshift     1.14-el8,1.16-el7,1.16-el8,1.18-ubi8,latest                      NO
nodejs      openshift     10-ubi7,10-ubi8,12-ubi7                                          NO
perl        openshift     5.26-el7,5.26-ubi8,5.30-el7,latest                               NO
php         openshift     7.2-ubi8,7.3-ubi7,7.3-ubi8,latest                                NO
python      openshift     2.7-ubi7,2.7-ubi8,3.6-ubi8,3.8-ubi7,3.8-ubi8,latest              NO
ruby        openshift     2.5-ubi7,2.5-ubi8,2.6-ubi7,2.6-ubi8,2.7-ubi7,2.7-ubi8,latest     NO
wildfly     openshift     latest                                                           NO

[zhengxiaomei123]

After debugging the code, I got the info :The list of available Devfiles is sourced from the official odo registry as well as any other registies added via odo registry add.

[zhengxiaomei123]

So the reason is the default Devfiles registry is not ready for IBM Z&P. For client using odo on OCP/Z, This is the wrong message to list what devfiles are supported. We need make the default Devfile registry ready for Z&P.

[kadel]

The current Devfile registry that odo is using doesn't have a way to filter Devfiles based on the architecture. The current registry is just a temporary solution put in place till there is an official Devfile registry implementation.
I'm afraid that is blocked till we have the official Devfile registry with multi-arch support.

To be completely honest, we don't even have the bandwidth (and HW) to build container images and maintain and test Devfiles for alternative architectures :-(

[zhengxiaomei123]

We did a couple of experiments for making Devfiles integration and e2e tests ready for Power and IBM Z. There are some images ready from CodeReadyWorkspace and published on redhat.registry.io The details are :

java-maven:
registry.redhat.io/codeready-workspaces/plugin-java11-openj9-rhel8
java-openliberty:
registry.redhat.io/codeready-workspaces/plugin-java8-openj9-rhel8
java-quarkus:
registry.redhat.io/codeready-workspaces/plugin-java11-openj9-rhel8
java-springboot:
registry.redhat.io/codeready-workspaces/plugin-java8-openj9-rhel8
python:
registry.redhat.io/codeready-workspaces/plugin-java8-openj9-rhel8
nodejs:
registry.redhat.io/codeready-workspaces/plugin-java8-openj9-rhel8

I am sure this can be used for tests but not sure IF this can also be used for client issue.
Also another way is to contact multi-arch team (Jeremy Poulin ) to help build those images.

[sarveshtamba]

Supporting single vs multi arch images in a stack - https://github.com/devfile/api/issues/244

[zhengxiaomei123]

For integrating devfiles related integration and e2e tests on nightly CI for IBM Z&Power, we need to change the devfiles used in tests. One way is to use other secure devfiles registry on GitHub repo . But there are still some subset of tests are hardcoded with devfiles for x86 . Those devfiles are put on tests/examples/source/devfiles directory. The solution is #4290

[zhengxiaomei123]

There are some java devfiles in default registry not use multi-arch images. Open the issue here to get multi-support: https://github.com/eclipse/che/issues/18610

[zhengxiaomei123]

There would be a new Devfile Registry (https://github.com/devfile/registry) based on OCI images to use it as DefaultDevfileRegistry instead of the current registry hosted on Github (odo-devfiles/registry).

[zhengxiaomei123]

In order to support multi-arch , a couple of things need to happen for the new Devfile Registry(in order):

1. https://github.com/devfile/api/issues/239 devfile spec/library
2. https://github.com/devfile/api/issues/244 in devfile spec/library
3. Tools like odo may need changes to implement that: https://github.com/devfile/api/issues/244
4. The stack in the registry needs to be updated to support it

This is due to finish in June of 2021.

[zhengxiaomei123]

The temporary solution for Z&P: Setting up a secure registry with multi-arch images in devfiles. Those images are published publicly on regsitry.redhat.io .To access this registry , the redhat login is required. For OCP user, it is ok. For people who use odo even with plain Kubernetes, it is not working.

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[zhengxiaomei123]

keep this issue open to wait for the support of devfiles registry for multi-arch.