Che: Adding Power Support to che CI environment

[lysannef]

Feature Request

As you have a running CI environment at https://ci.codenvycorp.com/job/che-master-ci/ ,
we would like to know if you would be interested in adding power support .
We can provide power support by means of instances at OSU/OSL. The request form to be filled is at http://osuosl.org/services/powerdev/request_hosting/

Please note the IBM Advocate is Gerrit Huizenga ([email protected])
Kindly let us know if any additional information is required.

The Tensorflow community has official builds running by use of the OSU environment. https://github.com/tensorflow/tensorflow#community-supported-builds

OS and version:
arch: ppc64le
OS: ubuntu16.04

As a continuation to issue #2510 @TylerJewell @benoitf

[ghatwala]

@skabashnyuk - Could you please share your comments/thoughts on above CI proposition ?

[skabashnyuk]

CC @slemeur @l0rd

[l0rd]

I am not familiar with OSU/OSL. @skabashnyuk do you have more context?

[ghatwala]

OSU/OSL stands for Oregon state university/Open source labs providing power development hosting environments services - http://osuosl.org/services/powerdev/

The Open Source Lab partners with IBM to host POWER based servers in order to provide an open platform for innovation to the open source community.

There is a form link which needs to be filled to get power (ppc64le environments - http://osuosl.org/services/powerdev/request_powerci/ ) and attach them to your che CI server as slaves for regression.

For eg , an open source community "Tensorflow" is using the infra offered by OSU/OSL for their power based job deployments as part of community supported builds - https://github.com/tensorflow/tensorflow#community-supported-builds

Please let me know if u need any more info on this - @l0rd , would be glad to assist . Our intent is to add "power/ppc64le" support to existing che CI build environments - https://ci.codenvycorp.com/computer/.

[ghatwala]

@skabashnyuk @l0rd - any thoughts/comments on the above ... may be this proposition could be passed to CI deployment team for their comments ?

[slemeur]

Che is packaged as a set of containers which are deployed on K8S/OpenShift. Since K8S can be installed on ppc64le environment, running Che there should be possible.

• Is that something that you've already tried @lysannef and @ghatwala ?

Getting Che running on different target environments is surely interesting but who will be responsible of that CI environment and fixing the issues which might happen on ppc64?

[l0rd]

I suspect that building ppc container images may not be straightforward. We may need modify the existing Dockerfiles or create new ones for every Che component. The same for stacks.

[ghatwala]

@slemeur - answering your queries .

Is that something that you've already tried @lysannef and @ghatwala ?

We did attempt che deployment on Kubernetes/ppc64le , using the helm chart given in this repo by changing the equivalent docker image mentioned in "values.yml" to utilize locally created image ,however this failed to deploy . Please note as of now we are focussing towards che deployment via "docker" containers approach only. We are in midst of resolving che dashboard functionality error on ppc64le via docker deployment via this ticket - https://github.com/eclipse/che/issues/13197 . If possible , you could add your few cents here :-) ?

Getting Che running on different target environments is surely interesting but who will be responsible of that CI environment and fixing the issues which might happen on ppc64?

We will be taking up the ownership and help maintain the power specific jenkins job and errors. Only thing as seen from here - "Our intent is to add "power/ppc64le" support to existing che CI build environments - https://ci.codenvycorp.com/computer/." To achieve this , anyone from your Che CI deployment team needs to fill up this very simple form ( http://osuosl.org/services/powerdev/request_hosting/ ) and add IBM Advocate as Gerrit Huizenga ([email protected]). ( FYI - @gerrith3 ).
Once this ppc64le vm is created . this vm needs to be added as slave node to your jenkins server.
Later once the ppc64le specific job is created ,we could be provided job level configuration access to resolve/fix the ppc64le specific che build issues - we are ready to help on this.
Kindly let us know if any additional information is required. ?

@l0rd -answering your queries

I suspect that building ppc container images may not be straightforward. We may need modify the existing Dockerfiles or create new ones for every Che component. The same for stacks.

yes , agreed- @lysannef has already created those ppc64le specific docker images by making changes in existing dockerfiles here for all required che components/stack . She has the changes ready and once we are able to fix/resolve last outstanding functionality issue ( https://github.com/eclipse/che/issues/13197 ) of che dashboard we will be raising a PR which can then add support to ppc64le images creation via this repo directly ( FYI - We have tested that the dockerfile changes done doesn't alter the docker images creation on x86_64 ) .
Kindly let us know if any additional information is required. ?

[l0rd]

Ok the work you are doing is really cool. I didn't get at the beginning how far you already were in the process. As I commented in the other issue I believe that your work (porting Che to another arch) will be interesting for many of our users (and us as well) so I would love to document how you have proceeded.

As for the version of Che you are using I warmly recommend to use Che 7. The beta is currently available are we are going to GA it in a month. Che 6 won't be supported and won't evolve anymore.

Che 7 runs on Kubernetes only (we heavily use the k8s API and maintaining the compatibility with the Docker API as well was too onerous for us).

[ghatwala]

@l0rd -

Che 7 runs on Kubernetes only (we heavily use the k8s API and maintaining the compatibility with the Docker API as well was too onerous for us).

As far as i understood -deployment on kubernetes method can be done via helm chart and it stills needs an equivalent docker image for power - https://github.com/eclipse/che/blob/master/deploy/kubernetes/helm/che/values.yaml#L19 . This image specified is an "intel" arch specific image and wont run on "ppc64le" arch.

We have already created this image on ppc64le by tweaking dockerfiles/agents/etc as specified in other issue.

However seems like functionality wise on the che dashboard , we aren't getting "workspace creation" working correctly on ppc64le . So my queries ....

• Wouldn't we still need to create equivalent docker images for ppc64le - for deploying che on power via deployment methods - kubernetes/docker/openshift. If yes , which are the " exact docker images ( rebuilt for ppc64le) and other runtime dependencies/agents" required for che deployment on k8s for ppc64le

• Our issue seems like we are missing something within this docker image which is preventing workspace creation on che-dashboard for "ppc64le" ? Any pointers on that ? Unless we are able to get "workspace creation" functionality working correctly on ppc64le we are still far from "porting che to ppc64le arch" ? :-)

Please let us know your thoughts on above ?

[l0rd]

The list of images you need when starting a workspace depends on the stack (Java, Python, Go etc...) . You can read more details here.

Anyway you need at least the following images:

• eclipse/che-server:nightly
• centos:centos7
• eclipse/che-init-plugin-broker:v0.17
• eclipse/che-unified-plugin-broker:v0.17
• eclipse/che-machine-exec
• eclipse/che-theia:next

[slemeur]

@ghatwala Thanks for the explanation and thanks also for contributing this to the Eclipse Che community, this is very cool and exciting!

[l0rd]

As I mentioned on the other issue I have started this document that should help you

[ghatwala]

@slemeur - Will you be the one filling the OSU form ( http://osuosl.org/services/powerdev/request_hosting/ ) as mentioned in below procedure ..

We will be taking up the ownership and help maintain the power specific jenkins job and errors. Only thing as seen from here - "Our intent is to add "power/ppc64le" support to existing che CI build environments - https://ci.codenvycorp.com/computer/." To achieve this , anyone from your Che CI deployment team needs to fill up this very simple form ( http://osuosl.org/services/powerdev/request_hosting/ ) and add IBM Advocate as Gerrit Huizenga ([email protected]). ( FYI - @gerrith3 ).

Once this ppc64le vm is created . this vm needs to be added as slave node to your jenkins server.
Later once the ppc64le specific job is created ,we could be provided job level configuration access to resolve/fix the ppc64le specific che build issues - we are ready to help on this.

Kindly let us know if any additional information is required. ?

[ghatwala]

@slemeur , @l0rd - Any comments on the CI proposal ( adding power vm's to jenkins server) above ?
Or may be you could redirect to your CI team for comments ? :-)

[lysannef]

@l0rd @slemeur Has the community had any thoughts about pursuing the addition of power support to the CI environment?

[ghatwala]

@slemeur @l0rd - Any thoughts on adding - "Power" support for your official CI - https://ci.codenvycorp.com/computer/ ?

[che-bot]

Issues go stale after 180 days of inactivity. lifecycle/stale issues rot after an additional 7 days of inactivity and eventually close.

Mark the issue as fresh with /remove-lifecycle stale in a new comment.

If this issue is safe to close now please do so.

Moderators: Add lifecycle/frozen label to avoid stale mode.

[gerrith3]

@ghatwala @lysannef any update on this? Did we ever get a CI environment set up?

[lysannef]

@gerrith3 No we do not have a CI environment set up. There has been no response from the community on this issue and no response to our continuous reminders/questions as well.

@seth-priya FYI.