Next.js: [monorepo tooling] Convert from taskr to Bazel

[Timer]

We would love to make our monorepo development experience way faster and more ergonomic. It'd be awesome to drop taskr to something like Bazel, or another deterministic build system.

tl;dr replace yarn dev in <rootDir>/package.json w/ Bazel

This task isn't for the faint of heart! It'll require a ton of effort.

[alexeagle]

✋
could we also make next.js usage better under Bazel in exchange? :)

[Timer]

@alexeagle I'd love to learn more about what you have in mind!

[alexeagle]

Sorry that was a rushed comment yesterday. Hi! I don't mean to make this like a quid-pro-quo; obviously we all want to make our software better within our time constraints.

From https://github.com/vercel/next.js/issues/9589#issuecomment-570427743 I see that so far, issues affecting Bazel aren't in scope. As an OSS maintainer I get the need to constrain your scope and maybe there are enterprise users who would use their paid support plan to fund the work. But if that doesn't happen, I hope we can still merge some fixes of this kind. We've done a pretty good job (IMO) on rules_nodejs to make nearly all programs behave well under Bazel so I expect there are very few of these fixes needed. (Maybe we can entirely workaround on the Bazel side but you end up with some smelly user config files like https://github.com/bazelbuild/rules_nodejs/blob/3af24495881c02ad38798dda92d9e93e9a1aa0c0/examples/create-react-app/BUILD.bazel#L16-L58 for CRA)

As for the OP here,

• do you think only the folders with taskfile (only packages/next) is in scope to start with?
• how do you feel about no longer keeping output files like https://github.com/vercel/next.js/blob/canary/packages/next/amp.js in version control?
• I think we'll have to do some back-and-forth between the contributor on this issue and maintainers who understand the current devx well

[Timer]

@alexeagle We'd be more than happy to explore fixing the build directory not being writable error for Bazel if we had a access to someone from the Bazel team to collaborate with on it! The enterprise comment was more so referring to if we had to do 100% of the discovery and resolution ourselves, without someone who can point us in the right direction or explain the technical needs (we're not familiar with Bazel).

---

Regarding OP, packages/next would be an awesome first win. I'd also be really curious in figuring out the story for the next-polyfill-nomodule or create-next-app that skips re-running the build command if nothing has changed.

Each package has its own set of build approaches:

• packages/next: A mix of ncc to emit single-file JS bundles and 1-to-1 TS/JS mappings through TS or Babel respectively.
• packages/create-next-app: Uses ncc to emit a single-file JavaScript bundle (webpack wrapper).
• packages/react-dev-overlay: tsc -d -p tsconfig.json
• packages/react-refresh-utils: tsc -d -p tsconfig.json

(ordered in assumed difficulty level, hardest to easiest)

---

how do you feel about no longer keeping output files like canary/packages/next/amp.js in version control?

SGTM as long as they're generated.

[alexeagle]

Cool, I'm playing with it a bit now, but I think we'll need to improve rules_nodejs support for yarn workspaces

[alexeagle]

What do you think about moving all the devDependencies to the root package.json? It will be easier to make this work today. In my understanding of yarn workspaces, this doesn't break anything, lets you maintain just one list of these libs and their versions, and of course the individual packages don't need to note their devDeps since it's not relevant in the distribution. Do you know of an advantage in declaring devDeps in the subdir package.json's?

Is the canary branch the right code to branch from? Should we have a development branch for this or just keep code in PRs and merge partially-working bits (not interfering with mainline dev until we are ready, of course)

We should pick a first milestone. I guess it should be to build an identical release artifact as https://unpkg.com/[email protected]/ ? If so my plan would be to first add an npm script that builds both and compares, then we reduce that diff down to zero.

[alexeagle]

Dropping some partway-working bits here https://github.com/vercel/next.js/compare/v9.4.4...alexeagle:bazel#diff-98d8674aa753464c5bc0a56a4b1edce9

[Timer]

1. Hmm, we'd probably prefer to not move all the dev dependencies unless this significantly complicates things. These packages are actually bundled & shipped as part of Next.js, so they're more-so "actual dependencies" (but not really) rather than only used for dev. I don't feel strongly on this, though.
2. canary is the correct branch, yes.
3. Close parity with the 9.4.4 binary would be awesome!

---

In terms of incrementalism, I think it could be a quick win to use Bazel for next-polyfill-nomodule / react-dev-overlay / react-refresh-utils before we go all the way for the next package. However, if you haven't seen anything that majorly complicates the next package, we can proceed with that.

I'd be happy to merge these PRs upstream as you're iterating so you're not chasing a moving target (assuming they somewhat work for a subset of the compiled resources).

[alexeagle]

just a ping to keep this alive - it's still on my good-intentions backlog for sometime when extra cycles or a volunteer mentee appear

[Timer]

Sounds great, thanks!

[asvny]

Hey @alexeagle I like to volunteer but definitely need a mentee if so happy to help ... btw I am a newbie with Bazel but I like to learn it.

[alexeagle]

@asvny thanks for volunteering - I think this will be a hard first Bazel project so I'm not sure where you would start. Also I've had even less time lately than before. Come chat with us in Bazel slack in #javascript and maybe we can figure it out.

[asvny]

how about I start with an easy one like packages/react-refresh-utils or packages/react-dev-overlay ?

[lukasholzer]

@Timer I would be happy if I can help you with the bazelification 🤓 I want to convert our internal react app to Next.js with MDX and we do everything with bazel. So if you need some help maybe you can outline your roadmap or explain a little bit more in detail. As @alexeagle mentioned maybe in the bazel slack: https://slack.bazel.build