Openwhisk: Move components out into individual repos

[csantanapr]

The idea is to move components into separate repositories without damaging the contributors and operators experience.

Currently everything is one repo make some work easier:

• easy to get all files
• no version dependencies between inner components

The problem it will not scale as the contributions grow

Acceptance criteria:

• Handle dependencies between components
• Handle testing, tagging, and releasing between components
• Repo by itself should be self contained that dev, unit tests can be done in an easy way
• Continues Integration and Continues Deployment properly maintain
• No Humans to release and tag, everything should be automated

[csantanapr]

@mrutkows I created an Epic instead of an issue since this will take some time to do, but we can make progress in some components short term and others long term

[rabbah]

For contrast https://github.com/docker/docker 1370 contributors, one repo, includes docs and cli.

[jberstler]

When splitting existing repos into smaller, specialized ones, I found the following command to be really helpful:

git filter-branch --prune-empty --tree-filter 'rm -rf <folders that don't belong in new repo>' HEAD

If you start with an existing repo, and run this command and replace <folders that don't belong in new repo> with the list of folders which do not belong in the new specialized repo, you will be left with all the original commits/history for just the folders you want in the new repo. This pruned repo can then be pushed to your new URL.

For reference: https://www.git-scm.com/book/en/v2/Git-Tools-Rewriting-History#The-Nuclear-Option:-filter-branch

[csantanapr]

@rabbah what's your opinion based on your docker reference?
That we should not split components long term and have everything in one repo.

I'm not trying to disagree or agree, just want to hear your opinion and proposal.

There are big successful open source projects that keep things in one repo and there are others that split them into different repos.

This issue is to discuss pro and cons, short term and long term. What to split and what not to split. Or not to split anything l, etc...

That's why I made it an Epic 😉

[csantanapr]

Short term if we split the docs I think I want to leave a submodule in its place short term to make it easier to find the specific docs that match the code.

and also for someone lurking into github can click on the docs folder and go the matching version of the docs

Feedback is welcome

[mrutkows]

IMO, the catalog ecosystem is a separate user/dev. community and needs to grow independently of the base/core code (even using docker as the example, docker/distribution... ).

[rabbah]

Taking new stock of some candidate bits that we can move out of the main repo:

1. mobile starter app
2. ansible files for distributed deployments
3. reconsider the cli (requires that we boost our API based testing)
4. python sdk (formerly the cli)
5. some/all of the language runtimes (#1316)
6. docker sdk
7. docs (as noted already)

[rabbah]

A number of these components have moved to new repos.
@csantanapr can you provide your summary chart here and then we can close this issue?

[csantanapr]

Yes

[csantanapr]

[dgrove-oss]

@csantanapr can we put the image of the components/repo mapping somewhere in the project web pages too? I think this is really useful information.

[csantanapr]

Thanks, you mean the main web page openwhisk.org ?

[dgrove-oss]

maybe in the main README.md (https://github.com/apache/incubator-openwhisk/tree/master/docs#readme) in a new repo overview subsection?