A common problem that is only going to get worse is that we want to make a change in some code that is shared across all pipelines.
To facilitate this, we need a tool which uses the git history to find changes that can also be applied in other pipelines.
Ideally, this will be used by an automated GitHub robot of some kind which will automatically make the changes an open PRs in branches on all pipelines (as done in conda-forge and several other communities).
People have discussed this back at the main cookiecutter repo:
https://github.com/audreyr/cookiecutter/issues/784
There is an implementation available here:
https://github.com/senseyeio/cupper
I will test it out and see if it works for our purposes.
so we basically create a branch TEMPLATE for every pipeline, that is created from the beginning, when the pipeline was created using the template. This will conserve a snapshot of the origin template it was created from in its separate branch, but connected to the pipeline Git history.
Once new template releases are present, checkout the TEMPLATE branch, apply commits from the new template version, commit and make an automated pull pull request. This will only merge changes on the template, not on the pipeline.
For the pipelines that have not been build from the cookiecutter template, we need to introduce a new root commit, which introduces the creation from the template. From this commit, the TEMPLATE branch has to be created.
I think this strategy is good but I think we need to slightly modify it for the latter case, when the pipeline has not been built from the TEMPLATE branch. Instead of doing a git rebase as suggested in the stackoverflow link I think we should do a regular merge between the TEMPLATE branch and master.
Implemented
Most helpful comment
People have discussed this back at the main cookiecutter repo:
https://github.com/audreyr/cookiecutter/issues/784
There is an implementation available here:
https://github.com/senseyeio/cupper
I will test it out and see if it works for our purposes.