Xterm.js: Should we use a monorepo structure for addons?

[Tyriar]

I'm beginning to think it will be much easier to manage the addons if we just include them as part of the xterm repo (in <root>/addons/<addon>?) but still upload them separately to npm. That way we could test them all as part of our CI to ensure breakages don't happen across versions, like what might have happened to ligatures recently. The main downsides I see are that issues will all remain in the 1 repo (maybe that's good?) and they won't be as good examples of what an external addon should be.

[Tyriar]

Another perk of this, we can merge in the webgl renderer into the repo once it's converted to an addon, without worrying as much about it breaking because of private API (we just won't ship it yet).

[mofux]

I have been using monorepos before and always found them very useful. We could even use yarn's workspaces feature to test unpublished addons during development. Speaking of workspaces, I was thinking of a structure like this:

- packages
  - xterm-core
  - xterm-addon-search
  - xterm-addon-webgl

We might think about taking the @xterm namespace on npm and publish our own packages underneath: @xterm/core, @xterm/addon-webgl, @xterm/addon-search...

With regard to issues for addons: Maybe we can create a tag for every addon, so we can pin issues to them?

[Tyriar]

I put together a reasonable system for getting yarn to work in https://github.com/xtermjs/xterm.js/issues/2164 using --modules-folder in yarnrc, looks like these workspaces will let me move the ws module back into the attach addon 👍

[Tyriar]

- packages
  - xterm-core
  - xterm-addon-search
  - xterm-addon-webgl

I looked into using workspaces but it looks like we would need to shuffle everything around because the root package.json needs to be marked as private. I think keeping the current layout where core is the primary repo is the way to go. The layout in https://github.com/xtermjs/xterm.js/pull/2164 which seems to work really well is:

• src/ (core)
• addons/xterm-addon-web-links
• addons/xterm-addon-search
• ...

Note that the naming of the folders in addons is very important as they have to be identical to what we call the addons. I think this could also be addons/@xterm/addon-search if we wanted to go with that naming.

[mofux]

I'm not sure if #2164 is the right way to do it. With the workspaces feature, the root project (root package.json) has to be private for a good reason. The root project only serves as an empty mono-repo container that is not supposed to get published. It also shouldn't have code by itself (except maybe some helper scripts that help us publish or test the packages). The directory names in which the packages reside shouldn't really matter. Say we have a structure like this:

- packages
    - core
    - addon-web-links
    - addon-search

our root package.json could look like this:

{
  "name": "xtermjs",
  "private": true,
  "workspaces": [
    "packages/*"
  ]
}

Now here is the nice thing: from within the addon-web-links package we could simply include @xtermjs/core as a dependency to its package.json, and yarn will automatically link the core package to its node_modules (given that the core package defined the name @xtermjs/core in its package.json).

IMO it's better to have the core package on the same level as the addons (and not as a parent), because it cleanly separates all packages from each other.

[Tyriar]

One major downside of that I see, apart from breaking git history with all of src (I don't think git log follows file moves without some extra magic), is that the folder structure is more of an obstacle; if I want to work on the xterm module part I would need to navigate the folders:

• packages
  
  
  
  • xterm (core would be confusing as we have a core module inside this)
  
  
  • src
    
    
    
    
    
    • common
    
    
    
    • somefile.ts
    
    
    
  
  

That's a lot of levels to get to the source code that is worked on 99% of the time.

It would be good from the perspective of the demo treating it more like a real node module, but I don't think that outweighs the downsides.

[mofux]

Thanks for sharing your concerns, I think I got so used to drill down two folders deeper in my other project that I didn't really think about that issue anymore 😅But yeah, I can understand that this is major inconvenience 99% of the time. You've mentioned above that you've used the --modules-folder in yarnrc, but I can't find that yarnrc in PR #2164. From what I've read, the --modules-folder changes the location where node_modules are installed, but that would also mean that all our devDependencies of core would end up in the addons folder, no? Is there a reason to use this at all?

[Tyriar]

I ended up removing --modules-folder as it didn't work the way I expect it to, when running yarn in the addon ../../node_modules would get cleared out and replaced with just ws and @types/ws, I was expected it to be additive but it makes sense now that I think about it. Instead of trying to duplicate the dependencies right now everything just uses the root node_modules and addon dependencies become devDependencies of the root package.json (just ws and @types/ws for attach testing).

Each addon only has a single npm script:

"prepublishOnly": "../../../node_modules/.bin/tsc -p src"

So npm publish will always do a build beforehand, another complication with sharing the modules or duplicating them in the addons is that the typescript versions could differ and cause confusion. They all share the same primary dependencies anyway so sharing makes sense (puppeteer, mocha, typescript, etc.).

[mofux]

I see, that's fine with me 👍 Let's go with #2164 then - and thanks for all the effort you are putting into the whole restructuring / refactoring ❤️