Origin: Proposal - splitting the console code into a new repo

[jwforres]

Requirement: split the openshift console code out of the origin repo before we actually split the runtime (console will still be embedded in the master). We may still split the runtime later, but we need to split the repo sooner.

• Move everything under origin/assets into a new repo, tentatively named "origin-console" (suggestions welcome)
• In origin-console repo the concat/minified/compiled/etc js and css files (the "dist" files) will be delivered instead of a bindata.go
• bindata.go in the origin repo will be an empty stub so people that dont want/need the console are still able to build origin
• a hack script will be added that
  
  
  
  • fetches origin-console/master
  
  
  • runs go-bindata to generate a real bindata.go
  
  
• above script can be used by
  
  
  
  • non ui team devs that just want to run the latest console without pulling down the whole asset dependency chain
  
  
  • devenv_ami (see below)
  
  
• devenv_ami will run the new hack script so that the docker images it produces for origin contain the built bindata.go, the resulting ami would also be running the binary with the bindata.go so QE can test with it. this also means the virtual box image would contain the correct built binary unless a dev syncs their local repo over top of it
• bindata.go in origin will be added to .gitignore once the stub is checked in, this way anyone that runs the script to build bindata for themselves won't get a git diff on the file
• fork_ami job will need another repo listed so it can use a different fork/branch of origin-console
• ui spec tests and integration tests, excluding ones that require the e2e tests to have run, will be run in a merge queue for the origin-console repo. tests requiring e2e will be disabled for now. the UI merge queue will test against the latest origin image on the docker hub
• updates to vagrant-openshift to support all of this
  
  
  
  • separate task to run just console tests
  
  
  • add new repo to the list of synced repos
  
  
• devenv_ami / fork_ami jobs will run the ui integration tests, we could possibly run the e2e ui tests here as well

[jwforres]

CC'ing folks who might care...
UI team @spadgett @jhadvig @fabianofranz @benjaminapetersen @rhamilto @sg00dwin

for the other teams CC anyone you think will want to know @pweil- @mfojtik @jcantrill @bparees @liggitt
This mainly affects the UI team, otherwise this really only affects folks that like to use the console during their day to day dev and part of this proposal is to make that as painless as possible.

[pweil-]

@erinboyd

[jwforres]

@stefwalter @gashcrumb

[jwforres]

This doesn't even have much impact on the UI team since you'll still be using the local grunt server to do development. The main changes for people contributing to the console are
1) different repo
2) delivering all files under dist instead of delivering bindata.go
3) faster UI merge queue!!

[benjaminapetersen]

Nice, all kinds of fun here :smile:

[fabianofranz]

"origin-console" a.k.a. "oc"? :)

[spadgett]

Move everything under origin/assets into a new repo, tentatively named "origin-console" (suggestions welcome)

openshift/web-console?

[spadgett]

Or openshift/origin-web-console?

[jwforres]

openshift/origin-ui was another suggestion i've heard

[jwforres]

looks like we have two votes for origin-web-console so unless anyone strongly disagrees we'll go with that

[benjaminapetersen]

Hmm. I think I like web-console since we use that terminology all the time. ui makes me think it would be a set of tools like angular-ui, jquery-ui, yui, etc. Do we want 'origin' in the name given it is used for origin, enterprise & online?

My leaning is openshift/web-console

[jhadvig]

Having origin in name does make sense, since the console will follow the same pattern as origin server has, meaning that Enterprise and Online versions are just cuts of the Origin.
Therefor I would go with openshift/origin-web-console.

[fabianofranz]

I'm good with either openshift/origin-web-console or openshift/origin-web.

[jwforres]

I'm going with openshift/origin-web-console

Next point of discussion, the asset hack scripts that will live in the new repo, if we want to rename any of those now is the time. Instead of hack/build-assests.sh do we want it to just be hack/build.sh

Or, since the bindata bits wont live in this repo, build-assets will just be a wrapper for running grunt build. And serve-local-assets is just a wrapper for grunt serve so I'd be open to dropping the scripts altogether where possible and just using grunt.

[rhamilto]

Or, since the bindata bits wont live in this repo, build-assets will just be a wrapper for running grunt build. And serve-local-assets is just a wrapper for grunt serve so I'd be open to dropping the scripts altogether where possible and just using grunt.

+1 I think it'll be more obvious to anyone using the repo. If I see a Grunt file, the first thing I do is look at the available tasks.

[jwforres]

Proposing:
hack/install-assets.sh --> hack/install-deps.sh
hack/clean-assets.sh --> hack/clean-deps.sh
hack/serve-local-asset.sh --> grunt serve
hack/build-assets.sh --> grunt build

the last one is hack/test-assets.sh, I'm still thinking about this one since it both runs the tests (grunt test) and does a diff to make sure the delivered compiled assets match up to the raw source

[jwforres]

I'm thinking we should just have a separate hack/verify-compiled-dist.sh to be consistent with the "verify" scripts in origin, i don't love the name though, thoughts?

[benjaminapetersen]

Had shared a few thoughts here a while back:

https://github.com/openshift/origin/issues/6721

Agree with Rob, Im def a fan of using raw grunt though, if doable.

[spadgett]

I'm thinking we should just have a separate hack/verify-compiled-dist.sh to be consistent with the "verify" scripts in origin, i don't love the name though, thoughts?

+1 to making them separate commands. hack/very-dist.sh ?

[benjaminapetersen]

I'm inclined to short names:

hack/clean.sh
hack/install.sh
hack/build.sh
hack/test-unit.sh
hack/test-e2e.sh
hack/test-all.sh
hack/verify.sh

So long as there isn't a danger in overlapping meaning. Its consistent with grunt clean, grunt install, grunt build, grunt test, etc. so seems pretty intuitive.

[jwforres]

FYI for the folks here that arent on the UI team, the repo has been split off to here https://github.com/openshift/origin-web-console and its merge and test tag enabled.

The second phase of the work to bring the console changes back into the builds of origin is next and will happen over the next couple days. Will update when that is complete.

[jwforres]

Question for everyone, the new repo is currently enabled for Travis, however it takes so long to pull down the npm/bower deps from scratch that running any of our tests in travis would actually take longer then just running a test on jenkins where the deps have been backed up and restored.

Is there anything you can think of that we would want to do in travis that wouldn't require the npm/bower deps?

[benjaminapetersen]

hmm. thinking, but currently nothing comes to mind.

[liggitt]

Would marking the npm and/or bower component dirs as cacheable for Travis speed things up?

https://docs.travis-ci.com/user/caching/#Arbitrary-directories

[jwforres]

Worth trying the cache, but it's over 500MB of data I think that we back up
and in Jenkins that's on disk and not going across the network like the
Travis cache does.
On May 12, 2016 11:00 PM, "Jordan Liggitt" [email protected] wrote:

Would marking the npm and/or bower component dirs as cacheable for Travis
speed things up?

https://docs.travis-ci.com/user/caching/#Arbitrary-directories

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/openshift/origin/issues/8809#issuecomment-218940833

[spadgett]

Wondering if we can install a subset of npm dependencies, enough to run the unit tests or even just jshint.