Rspec-core: RFC: Combine multiple RSpec repositories into a single one.

Created on 24 Jan 2018  路  10Comments  路  Source: rspec/rspec-core

Summary

Combine rspec-core, rspec-mocks, rspec-expectations, rspec-support, rspec-dev (which would be mostly deprecated with this proposal), the rspec meta-gem, and possible rspec-rails into a single repository.

Motivation

Making cross-cutting changes to RSpec involves a complicated dance of PRs. Selected examples:

  • Upgrading rubocop involved multiple PRs, that all needed to be somewhat kept in sync as a result of changes/feedback, and are still pending a final rspec-dev sync. As well as been a pain at the time, this has caused confusion for a subsequent contribution.
  • Adding to rspec-support to fix a bug in (say) rspec-mocks requires a PR to support, a PR to mocks pointing at that branch to demonstrate that the support change works, merging the support PR, then redoing the mocks PR to point back at master. (e.g. https://github.com/rspec/rspec-mocks/pull/726)
  • Any of the rspec-dev tasks to "sync" configuration to all repos requires a PR for each repo.
  • complex and partially overlapping CI builds

It's also not always clear to issue reporters which project is appropriate for an issue. We shouldn't expect newcomers to know which sub-project an issue could be related to!

Approach

Issues

Given a motivation is to simplify issue reporting, we should migrate them all to the main rspec project (which current has issues disabled) using something like github-issues-import. We can use tags (mocks, etc...) to track what came from where, but for new issues they would be optional.

Inflight PRs

We should explicitly close or merge every open PR. This will include closing many stale PRs, that will need to be re-opened against the new monorepo. We can mitigate this with a notice period (i.e. "one month from now we're merging repos").

CI

I see two ways to set up CI:

  1. Run all the specs for each project on every CI. Conceptually the simplest, potential downside is runtime. Given a lot of build time is build setup (installing ruby, gems, etc...) I suspect this will work out fine.
  2. Add conditional build logic to identify which projects are likely affected based on file paths changed, and only run those. Speeds things up, but more complicated and not technically as thorough (though probably good enough).

This is where rspec-rails would be an interesting addition, because it has a test matrix with rails versions that isn't applicable to the other projects and would probably force us to option 2. I'm leaning towards keeping it out, at least to begin with. We can always incorporate it later!

For now we should benchmark option 1.

rspec-dev

We need to audit rspec-dev to see what things it does are still appropriate and how to best incorporate them into the monorepo. Given it's primarily used to "manage" multiple repos, my hope is it can be mostly deprecated. This will be investigated in the prototype suggested below.

Drawbacks and mitigations

  • Git history is harder to work with. We should keep around the old git projects for spelunking purposes.
  • Intermingled git logs. I'm not worried about this. Possibly consider it an upside, being able to see what was happening across the entire project.
  • Easy to accidentally add unwanted dependencies. Since the code is "there" this potentially makes it easier to accidentally introduce coupling between the various components. I figure we have enough history and discipline keeping them separate that this shouldn't be an issue in practice.
  • wrap up or manually transfer all hanging pull requests in Expectations/Mocks/Support/Core
  • align git version tags - they overlap, and we need to keep them in sync
  • ~CI limit of simultaneous jobs constraints our a bit tighter~

Next steps

  • [x] Feedback from other members of core team.
  • [ ] Create a new project to test this idea (rspec-monorepo), including setting up CI and incorporating rspec-dev.
picture xaviershay
👍5 ❤2 🎉1

Most helpful comment

I very much support this idea. The amount of overhead we deal with due to having separate repos is demoralizing. Elixir (the main language I dealt with the pass couple years) has first-class support for monorepos with many sub-applications via its umbrella feature and I'm a big fan of the approach when you're tooling supports it.

There's also another potential benefit we get out of a monorepo: it can greatly lessen the effort involved in extracting a feature out of the core gems and into an extension gem. For example, if we ever want to extract any_instance into a separate gem so it doesn't ship with rspec-mocks (something I'd advocate we consider for RSpec 4), with our current approach, extraction would be difficult. We'd have to refactor rspec-mocks to promote some new things to first-class extension APIs so the any_instance code could use it. That would be a lot of effort. With a monorepo, we could potentially move the any_instance code into a sub-directory of the monorepo, w/o needing to define new extension APIs. Instead, we can just include the any_instance spec suite in our travis build, so we can keep ourselves from breaking the any_instance logic.

possibly rspec-rails into a single repository.

I'm in favor of not merging rspec-rails. It's pretty different, and not part of core RSpec. The CI build is way different with all the rails versions it supports. And we've talked about potentially de-coupling it's versioning from the versioning of the rest of RSpec.

Given a motivation is to simplify issue reporting, we should migrate them all to the main rspec project (which current has issues disabled) using something like github-issues-import. We can use tags (mocks, etc...) to track what came from where, but for new issues they would be optional.

Great idea. A few additional thoughts:

  • It would be good to cleanup our issues before doing this so we don't import issues that should really be closed.
  • The merging of issues could potentially happen before (or after) the code is merged. The fewer changes in flight at once, the better.
  • When importing an issue into the monorepo, can we include a link back to the original issue?
  • It would be great if we could have the import script close the original issue, with a link to the issue in the monorepo.
  • I think there's value in tagging issues with which gem the issue applies to on an ongoing basis, not just for imported issues.
  • We probably want to prevent the creation of new issues once we've imported, but not turn off the issues feature of the old repos entirely, so folks can still browse/search/etc old issues. Not sure what github supports here.

This will include closing many stale PRs, that will need to be re-opened against the new monorepo.

Honestly, I think we can just close stale PRs, with a note saying the person can re-open a new PR on the monorepo if they want to see it get merged. I don't think we need to go through the effort of manually importing PRs (and automating it sounds hard). Cleaning up open PRs would be really helpful as a preliminary step.

Run all the specs for each project on every CI. Conceptually the simplest, potential downside is runtime. Given a lot of build time is build setup (installing ruby, gems, etc...) I suspect this will work out fine.

I would favor this approach over the more complicated analysis approach you mentioned. I think we can optimize this using a well-designed build matrix. The cukes are generally the slow part of our build. I'd favor doing something like:

  • One matrix entry for all the spec suites (and other quick little checks like yard coverage, etc)
  • One matrix entry for each cuke suite (core, expectations, and mocks)

So that would be 4x matrix entires per supported Ruby version...which sounds bad, but isn't actually any worse than our current situation, where we've got 4 builds (core, expectations, mocks, support) per supported version. And it should run simpler and faster.

We can consider not running the cukes on all supported ruby versions if there's a concern about the number of matrix entries.

This is where rspec-rails would be an interesting addition, because it has a test matrix with rails versions that isn't applicable to the other projects and would probably force us to option 2. I'm leaning towards keeping it out, at least to begin with. We can always incorporate it later!

I don't think I can even wrap my head around how we'd incorporate rspec-rails in a monorepo build. I very much advocate for keeping it separate. It's a special beast.

We need to audit rspec-dev to see what things it does are still appropriate and how to best incorporate them into the monorepo. Given it's primarily used to "manage" multiple repos, my hope is it can be mostly deprecated.

There's a couple different things rspec-dev provides:

  • Canonical copies of some files that we then push out to all the other repos. Once we have a mono repo, we can just have one copy of each file, with no tooling to sync it up needed.
  • A bunch of rake tasks for supporting RSpec development. Some of these are just there to deal with the pain of not having a monorepo (e.g. tasks to change branches in all repos, etc), and these can probably be dropped. Other tasks support doing things like bumping versions on all gems (which we'll still need), cutting releases, generating stats for blog posts, etc, and we'll still want these tasks--they'll just go in the root Rakefile of the monorepo.

I don't think we need to stringently audit rspec-dev and try to bring everything over--instead, we can just bring things over as we have the need for a task.

Create a new project to test this idea (rspec-monorepo), including setting up CI and incorporating rspec-dev.

This is a great idea. Would rspec-monorepo be a prototype, and then we'd re-do the merge on rspec/rspec for real? A couple thoughts/ideas:

  • rspec-monorepo should probably start off as a fork of rspec/rspec so you can test out merging into an existing repo with history.
  • Whoever works on this should keep copious notes on how they did it so we can easily reproduce on the rspec repo. Automating the process with some scripts is probably good, although as a one-time process we shouldn't try to go overboard.

One thing you didn't mention: how are we going to actually merge the source history of the repos? Is there a good tool for that? Obviously, all the SHAs are going to change. If possible, it would be really cool if whatever tool we used to import commits amended the message of each commit with a note like [Imported from rspec/rspec-core@abc1239237e8234bedf] so there's a clear link from each commit to the original. And we could potentially create a link the other way, too: since github lets you leave comments directly on commits, our import tool could leave an automated comment on each commit linking to its new location in the monorepo. Although, given the number of commits, I can imagine that would be super slow....so probably not worth it.

While I'm 100% on board with this idea, I can't say I'm excited (or particularly available) to do much of the work here. Is there someone who is willing to lead this effort? (Looking at you, @xaviershay :)...)

picture myronmarston on 25 Jan 2018
❤1 🎉1 👍1

All 10 comments

@rspec/rspec feedback please :)

xaviershay picture xaviershay on 24 Jan 2018

Background reading on monorepos:

We don't have enough code (nor should we ever) to start hitting some of the bigger downsides of a monorepo.

And also my own relevant background: I was responsible for the CI teams at Square for many years, dealing with a large Java monorepo and many disparate ruby repositories.

xaviershay picture xaviershay on 24 Jan 2018

I support this idea.

soulcutter picture soulcutter on 25 Jan 2018

I very much support this idea. The amount of overhead we deal with due to having separate repos is demoralizing. Elixir (the main language I dealt with the pass couple years) has first-class support for monorepos with many sub-applications via its umbrella feature and I'm a big fan of the approach when you're tooling supports it.

There's also another potential benefit we get out of a monorepo: it can greatly lessen the effort involved in extracting a feature out of the core gems and into an extension gem. For example, if we ever want to extract any_instance into a separate gem so it doesn't ship with rspec-mocks (something I'd advocate we consider for RSpec 4), with our current approach, extraction would be difficult. We'd have to refactor rspec-mocks to promote some new things to first-class extension APIs so the any_instance code could use it. That would be a lot of effort. With a monorepo, we could potentially move the any_instance code into a sub-directory of the monorepo, w/o needing to define new extension APIs. Instead, we can just include the any_instance spec suite in our travis build, so we can keep ourselves from breaking the any_instance logic.

possibly rspec-rails into a single repository.

I'm in favor of not merging rspec-rails. It's pretty different, and not part of core RSpec. The CI build is way different with all the rails versions it supports. And we've talked about potentially de-coupling it's versioning from the versioning of the rest of RSpec.

Given a motivation is to simplify issue reporting, we should migrate them all to the main rspec project (which current has issues disabled) using something like github-issues-import. We can use tags (mocks, etc...) to track what came from where, but for new issues they would be optional.

Great idea. A few additional thoughts:

  • It would be good to cleanup our issues before doing this so we don't import issues that should really be closed.
  • The merging of issues could potentially happen before (or after) the code is merged. The fewer changes in flight at once, the better.
  • When importing an issue into the monorepo, can we include a link back to the original issue?
  • It would be great if we could have the import script close the original issue, with a link to the issue in the monorepo.
  • I think there's value in tagging issues with which gem the issue applies to on an ongoing basis, not just for imported issues.
  • We probably want to prevent the creation of new issues once we've imported, but not turn off the issues feature of the old repos entirely, so folks can still browse/search/etc old issues. Not sure what github supports here.

This will include closing many stale PRs, that will need to be re-opened against the new monorepo.

Honestly, I think we can just close stale PRs, with a note saying the person can re-open a new PR on the monorepo if they want to see it get merged. I don't think we need to go through the effort of manually importing PRs (and automating it sounds hard). Cleaning up open PRs would be really helpful as a preliminary step.

Run all the specs for each project on every CI. Conceptually the simplest, potential downside is runtime. Given a lot of build time is build setup (installing ruby, gems, etc...) I suspect this will work out fine.

I would favor this approach over the more complicated analysis approach you mentioned. I think we can optimize this using a well-designed build matrix. The cukes are generally the slow part of our build. I'd favor doing something like:

  • One matrix entry for all the spec suites (and other quick little checks like yard coverage, etc)
  • One matrix entry for each cuke suite (core, expectations, and mocks)

So that would be 4x matrix entires per supported Ruby version...which sounds bad, but isn't actually any worse than our current situation, where we've got 4 builds (core, expectations, mocks, support) per supported version. And it should run simpler and faster.

We can consider not running the cukes on all supported ruby versions if there's a concern about the number of matrix entries.

This is where rspec-rails would be an interesting addition, because it has a test matrix with rails versions that isn't applicable to the other projects and would probably force us to option 2. I'm leaning towards keeping it out, at least to begin with. We can always incorporate it later!

I don't think I can even wrap my head around how we'd incorporate rspec-rails in a monorepo build. I very much advocate for keeping it separate. It's a special beast.

We need to audit rspec-dev to see what things it does are still appropriate and how to best incorporate them into the monorepo. Given it's primarily used to "manage" multiple repos, my hope is it can be mostly deprecated.

There's a couple different things rspec-dev provides:

  • Canonical copies of some files that we then push out to all the other repos. Once we have a mono repo, we can just have one copy of each file, with no tooling to sync it up needed.
  • A bunch of rake tasks for supporting RSpec development. Some of these are just there to deal with the pain of not having a monorepo (e.g. tasks to change branches in all repos, etc), and these can probably be dropped. Other tasks support doing things like bumping versions on all gems (which we'll still need), cutting releases, generating stats for blog posts, etc, and we'll still want these tasks--they'll just go in the root Rakefile of the monorepo.

I don't think we need to stringently audit rspec-dev and try to bring everything over--instead, we can just bring things over as we have the need for a task.

Create a new project to test this idea (rspec-monorepo), including setting up CI and incorporating rspec-dev.

This is a great idea. Would rspec-monorepo be a prototype, and then we'd re-do the merge on rspec/rspec for real? A couple thoughts/ideas:

  • rspec-monorepo should probably start off as a fork of rspec/rspec so you can test out merging into an existing repo with history.
  • Whoever works on this should keep copious notes on how they did it so we can easily reproduce on the rspec repo. Automating the process with some scripts is probably good, although as a one-time process we shouldn't try to go overboard.

One thing you didn't mention: how are we going to actually merge the source history of the repos? Is there a good tool for that? Obviously, all the SHAs are going to change. If possible, it would be really cool if whatever tool we used to import commits amended the message of each commit with a note like [Imported from rspec/rspec-core@abc1239237e8234bedf] so there's a clear link from each commit to the original. And we could potentially create a link the other way, too: since github lets you leave comments directly on commits, our import tool could leave an automated comment on each commit linking to its new location in the monorepo. Although, given the number of commits, I can imagine that would be super slow....so probably not worth it.

While I'm 100% on board with this idea, I can't say I'm excited (or particularly available) to do much of the work here. Is there someone who is willing to lead this effort? (Looking at you, @xaviershay :)...)

myronmarston picture myronmarston on 25 Jan 2018
❤1 🎉1 👍1

I'm on board with the mono repo idea, would certainly simplify keeping everything in sync!

On the CI side I'd recommend we not split out the various builds into matrices, as it will (ironically) slow the build down. Travis seems to limit the number of in progress builds at a time (one of the reasons why rspec-rails builds are so slow), which is fair given its a free service, so having one build per Ruby version will probably work out faster in practise.

JonRowe picture JonRowe on 26 Jan 2018
👍1

馃憤 for the monorepo idea.

Some ideas for the import of commits:

  • Keep original author info, so that they will appear in the contribution graph on GitHub
  • Keep original commit dates as imported commit's author dates
  • Add commits in chronological order for easier history investigation

    • Commits "Add foo" in rspec-support and "Use foo from rspec-support" in rspec-core should be close in the merged history

yujinakayama picture yujinakayama on 26 Jan 2018
👍1

Some ideas for the import of commits:

Oh interesting, I was just planning to start a new repo and copy the code over, with initial commit being "see the old repos". I can play around with various ways of importing history ... guess it should be pretty straight forward since there's no way they could conflict with one another.

xaviershay picture xaviershay on 26 Jan 2018

Oh interesting, I was just planning to start a new repo and copy the code over, with initial commit being "see the old repos". I can play around with various ways of importing history ... guess it should be pretty straight forward since there's no way they could conflict with one another.

I think for the prototype repo, copying in all the code is OK so you can work on getting CI setup, etc., but for the final merge I think the extra effort to preserve the source history as much as possible is valuable. I like @yujinakayama's ideas.

myronmarston picture myronmarston on 26 Jan 2018
👍1

Sorry I have made basically no progress on this. I'm going to de-commit from it rather than let it go on in limbo.

I made this prototype repo to start playing around with ideas if anyone wants to pick it up: https://github.com/rspec/rspec-monorepo-prototype

xaviershay picture xaviershay on 10 Apr 2018

We could merge the other repos while keeping all the history.
I had some success with the steps described here (https://thoughts.t37.net/merging-2-different-git-repositories-without-losing-your-history-de7a06bba804):
cd /tmp git clone [email protected]:rspec/rspec-core.git git clone [email protected]:rspec/rspec-expectations.git cd rspec-core git remote add rspec-expectations ../rspec-expectations git fetch rspec-expectations git co -b merge-rspec-expectations git merge -S --allow-unrelated-histories rspec-expectations/master
https://github.com/rspec/rspec-core/pull/2665

p8 picture p8 on 2 Oct 2019
Was this page helpful?
0 / 5 - 0 ratings

Related issues

fabioperrella picture fabioperrella  路  3Comments
ankit8898 picture ankit8898  路  3Comments
deepj picture deepj  路  3Comments
marcparadise picture marcparadise  路  6Comments
benmmurphy picture benmmurphy  路  6Comments