Odo: Making devfile package independent of odo with the possible goal of extracting it out as a separate project

Created on 20 Jul 2020 · 16Comments · Source: openshift/odo

Description

Soon other projects like che would start implementing devfile related features. As a community we should reduce the duplication efforts and allow other teams to consume devfile implementation ( adapter and parser ).

User Stories ( pending approval )

Make devfile/parser independent of odo code base
Make devfile/adapters and its coupled packages (clients) independent of odo specific codebase
Extract all the packages which could be used by other project into a separate repo ( pending discussion as this might be conterproductive as well for us )
- setup processes to allow faster/cross repo collaberation if we go with separate project

kinepic lifecyclrotten

Source

girishramnani

👍1

Most helpful comment

Hello all, commenting on here with what we discussed from the weekly IBM/RH Gitops sync this morning so that it will have more visibility.

General consensus about library is that it's a good idea, looking forward to developing
Known consumers: console backend, odo, Che
Non-golang consumers will not be able to use this library.
this library will probably live in github devfile organization
How much responsibility will this library have or do we want to leave it to the tool consuming the library?
- We should be able to say just services or just routes or ingress. For example "give me buildconfigs and instead of routes give me ingresses"
intended output of the library is just the structs of apt. type, consuming tool will be the one converting them to kube objects
We will probably have a high level of asks originally and then working towards more granular asks. Gradually we can add more interactions and functions. We can have a lot of endpoints to the library to have more flexibility.
Use https://gitter.im/devfile/community for discussions on this instead of slack channels

cc @Shraddhak22 @ranakan19 @jaideepr97 @wtam2018 @sbose78 @l0rd @kadel @girishramnani

reginapizza on 20 Jul 2020

👍4 🚀1

All 16 comments

@kadel

girishramnani on 20 Jul 2020

Just so its clear to everyone - this is post odo v2 GA release, possibly odo v2.1 release as this might affect pipelines work.

girishramnani on 20 Jul 2020

Hello all, commenting on here with what we discussed from the weekly IBM/RH Gitops sync this morning so that it will have more visibility.

General consensus about library is that it's a good idea, looking forward to developing
Known consumers: console backend, odo, Che
Non-golang consumers will not be able to use this library.
this library will probably live in github devfile organization
How much responsibility will this library have or do we want to leave it to the tool consuming the library?
- We should be able to say just services or just routes or ingress. For example "give me buildconfigs and instead of routes give me ingresses"
intended output of the library is just the structs of apt. type, consuming tool will be the one converting them to kube objects
We will probably have a high level of asks originally and then working towards more granular asks. Gradually we can add more interactions and functions. We can have a lot of endpoints to the library to have more flexibility.
Use https://gitter.im/devfile/community for discussions on this instead of slack channels

cc @Shraddhak22 @ranakan19 @jaideepr97 @wtam2018 @sbose78 @l0rd @kadel @girishramnani

reginapizza on 20 Jul 2020

👍4 🚀1

Thanks for sharing this. I have a couple of comments:

I think that some of the controller code is currently being developed here (currently used by the devconsole and che)
Why not an operator rather than a library? It would make the contract more straightforward

cc @sleshchenko @davidfestal

l0rd on 21 Jul 2020

Why not an operator rather than a library? It would make the contract more straightforward

The operator may or may not be present in the cluster, dev console & odo would be unable to rely on it.
An operator paradigm typically deploys a "managed" workload, in this case, odo and dev console aren't looking for managed workloads - users expect to have control over them.

sbose78 on 21 Jul 2020

👍1

I think that some of the controller code is currently being developed here (currently used by the devconsole and che)

Great point, Mario. We should organize and work on this with alignment to ensure:

we design and agree on the API interfaces
we avoid duplication of work :)

Let's use the devfile meeting tomorrow to get an initial working/collaboration style setup?

sbose78 on 21 Jul 2020

👍2

Why not an operator rather than a library? It would make the contract more straightforward

The operator may or may not be present in the cluster, dev console & odo would be unable to rely on it.

An operator paradigm typically deploys a "managed" workload, in this case, odo and dev console aren't looking for managed workloads - users expect to have control over them.

+1 to what everything that @sbose78 mentioned.
One of the main selling points for odo is that you just download binary, and you are ready to start using it, no setup needed.

The library should be designed in a way that everything operating with Devfiles including DevWorkspace operator can use it.

kadel on 21 Jul 2020

The operator may or may not be present in the cluster, dev console & odo would be unable to rely on it.

I understand your point. But dev console already depends on that operator to run the web terminal (devfile v2 based). Che will do the same. odo will consume the same dependency in a different way. Shouldn't we be consistent instead? How do we make sure that odo / che / dev console use the same version of the controller if we let run multiple instances on the same cluster (one embedded in odo and one embedded in an operator)?

An operator paradigm typically deploys a "managed" workload, in this case, odo and dev console aren't looking for managed workloads - users expect to have control over them.

We want users to have control through a devfile, users should not directly control how a devfile is transformed/reconciled into a pod.

One of the main selling points for odo is that you just download binary, and you are ready to start using it, no setup needed.

Couldn't odo deploy the operator if it's not there?

l0rd on 21 Jul 2020

One of the main selling points for odo is that you just download binary, and you are ready to start using it, no setup needed.

Couldn't odo deploy the operator if it's not there?

No, the biggest reason is that odo users won't have privileges to deploy operators.

Also, making operator hard requirements for odo would be a big shift in odo philosophy and architecture it would mean a full rewrite.
@gorkem What is your take on this?

kadel on 22 Jul 2020

No, the biggest reason is that odo users won't have privileges to deploy operators.

You are right. I don't think it's a good idea. But ideally, if the devworkspace operator is running on the cluster, I think odo should be able to leverage it.

l0rd on 22 Jul 2020

Theoretically speaking, the operator could/should use the library this issue is discussing… :)

metacosm on 22 Jul 2020

During our devfile F2F last week, there was a proposal to separate out the devfile package from odo into a separate go library. After a short PoC of separating devfile, we did a demo in today's call of our weekly devfile 2.0 continuation discussion, below are the updates:

About devfile-parser : this library is a golang module that projects could use to generate kubernetes objects equivalent to the guidance in a devfile.
Repo has been moved to devfile organization, which was concluded as the best place for hosting this library - https://github.com/devfile/devfile-parser
Presently the library only separates the parsing and validation code. Work is in progress on moving adapter code to complete separation. Adapter code is responsible for creating kubernetes objects from devfile.
For point 3, there are impending PRs (https://github.com/openshift/odo/pull/3596 and others coming up from @girishramnani team) - regarding refactoring of devfile commands would impact the library.
Temporary slack channel has been setup to facilitate conversations between teams which will be consuming this library. Feel free to join in #tmp-devfile-library.

Next steps are:

Define goals, refactor existing code. Continue discussion on this thread
Setup project governance in https://github.com/devfile/devfile-parser
Agree on Common library APIs with Console, Che and odo stakeholders.
Enable odo and che workspace controller to consume this library.

ranakan19 on 22 Jul 2020

👍3

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

openshift-bot on 29 Oct 2020

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

openshift-bot on 28 Nov 2020

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

openshift-bot on 29 Dec 2020

@openshift-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.