Parcel: One-off/local plugins

Created on 12 Mar 2020  Ā·  12Comments  Ā·  Source: parcel-bundler/parcel

šŸ™‹ feature request

While I haven’t written any meaningful plugins yet (mostly due to #4320), I feel like I will make use of them once I understand them. @jakearchibald and I have grown very fond of writing project-specific, one-off plugins when using Rollup (e.g. in PROXX, and a talk about one-off plugins), and I think approach would be similarly great for Parcel!

However, you _need_ to currently install plugins as proper npm dependencies. Which means they either have to be published, or you put them in a sub-folder with their own package.json and then do something like npm i -S ./plugins/parcel-transformer-my-transformer.

šŸ’ Possible Solution

My idea is to distinguish local/one-off plugins from proper npm plugins in .parcelrc by a leading ./:

{
  "extends": "@parcel/config-default",
  "transformers": {
    "*.png": ["./plugins/my-transformer.js"]
  }
}
Feature ✨ Parcel 2

Most helpful comment

The primary driver for me was to encourage community contributions. There are thousands of Babel plugins that companies have written internally and back when I would go around talking to them I found that they had built a lot of the same exact plugins.

I think that just the act of putting something in a ā€œpackageā€ does a lot to encourage people to consider open sourcing their code. I think everyone wants to, but aren’t sure where to even begin when it’s just a file in their repo.

I was thinking that doing this would help ā€œbootstrapā€ the Parcel community, and give it a stronger sense of collaboration. The requirement was always sorta intended to disappear as a limitation at some point.

Maybe there’s a better middleground though, maybe it’s okay that it’s a local path as long as that path is still structured like a package and that package is marked as private.

And maybe I’m totally wrong about the effect this will have on the community. It’s really just a hypothesis based on my experience with Babel plugins and multi-package repos. But it’s totally possible that it would just intimidate people out of creating plugins altogether. It’s hard to say.

On the point about caching, it’s definitely _more difficult_ than just operating against a version, but it seems like you’ve already got a lot of the infrastructure there for caching those kinds of deps (Thanks @padmaia). If you think there’s a performance problem with it though, I might encourage a warning to be output that you are using a local plugin and those can be slower.

All 12 comments

This is by design, we don't really want to allow this as it would be very hard to cache this and therefore would result in slower builds and cache invalidation that should not have happened. (there's a similar issue with locally linked packages and monorepo's which we haven't figured out yet).

I'd suggest using a monorepo (yarn workspaces or something similar) for this use-case. (this wouldn't solve the caching issue but is at least valid in the parcel config)

https://github.com/parcel-bundler/parcel/issues/3397#issuecomment-521353931

But having it in a monorepo/linked vs locally makes no difference regarding caching....

I don’t quite agree with the reasoning, but it sounds like you have put a lot of thought into this and I can respect that :) And, after all, if necessary I can work with the local-install workaround.

I'll reopen this for further discussion since we will have the same issues with caching for locally linked plugins in a monorepo anyway (which we're already working on).

I think the other reason we had for this was to encourage people to share their plugins on npm (or a private registry in their company) rather than just keeping them to themselves. However, I could see some very project specific plugins where maybe this doesn't make sense (and you could use the private flag in a monorepo package anyway).

cc. @jamiebuilds. What do you think?

The primary driver for me was to encourage community contributions. There are thousands of Babel plugins that companies have written internally and back when I would go around talking to them I found that they had built a lot of the same exact plugins.

I think that just the act of putting something in a ā€œpackageā€ does a lot to encourage people to consider open sourcing their code. I think everyone wants to, but aren’t sure where to even begin when it’s just a file in their repo.

I was thinking that doing this would help ā€œbootstrapā€ the Parcel community, and give it a stronger sense of collaboration. The requirement was always sorta intended to disappear as a limitation at some point.

Maybe there’s a better middleground though, maybe it’s okay that it’s a local path as long as that path is still structured like a package and that package is marked as private.

And maybe I’m totally wrong about the effect this will have on the community. It’s really just a hypothesis based on my experience with Babel plugins and multi-package repos. But it’s totally possible that it would just intimidate people out of creating plugins altogether. It’s hard to say.

On the point about caching, it’s definitely _more difficult_ than just operating against a version, but it seems like you’ve already got a lot of the infrastructure there for caching those kinds of deps (Thanks @padmaia). If you think there’s a performance problem with it though, I might encourage a warning to be output that you are using a local plugin and those can be slower.

This is a tough one. I've definitely got a lot of 'local' plugins that I use in various Rollup projects. Currently I copy and paste them, because they work for that project, but they aren't good enough to be widely usable. My intention is to publish them once they are, but maybe I'll get lazy. Parcel's strategy prevents me being lazy.

On the other hand, it makes rapidly developing a plugin pretty painful, especially at the start of a project where there may be a lot of plugins, that are later rolled into one, or one plugin that's split into many etc etc, so there's a barrier there that might make me think, "nah I'll just use Rollup instead".

As a workaround for this issue (https://github.com/parcel-bundler/parcel/issues/4879), I had to create a custom resolver that fixes the ordering of mainFields. Annoying that I need to jump through these hoops for such a trivial change.

@speigg Ideally Parcel would get most of those things "right" for everyone and you'd never need to configure them. But when Parcel gets something "wrong", we then need to know:

  • For how many people is this "wrong"?
  • Is there a "right" solution that will work for everyone/more people?
  • Can the community adapt to our "wrong" solution and make something good out of it?
  • If we made a specific change, what would be the actual implications of that?

If we made this sort of thing a configuration option, most people will just find some Stack Overflow question and make the same change in every config without ever raising it with us.

When someone does start a conversation about it, it will exist solely in a GitHub issue with 300 comments almost all of which will lack nuance. People will have their opinion based on their need at a point in time and will defend it without actually testing it in the community and answering the above questions.

These "hoops" you have to jump through force two things:

  1. For the people unwilling to jump through those hoops, they'll probably find other solutions that will over time "normalize" the ecosystem (more on this in a second)
  2. For the people willing to jump through those hoops, they're starting an important exploration to see if the change they want to make is something that Parcel should adopt.

For 1, I want Parcel to get things like module resolution right 100% of the time. But that's impossible today because every ecosystem does things in different ways, and few people are working towards normalizing any of that. The biggest damage that Webpack/Babel made to the ecosystem was allowing people to easily configure things that should never have been configuration points. The result is that lots of tools have half-baked solutions that only work for them, which creates pressure on other tools to support their half-baked solutions, which causes the ecosystem to degrade more and more over time. Forcing people to jump through hoops inverts that relationship and forces the community to examine these integration points.

For 2, if someone does jump through all these hoops, that means they are invested in the problem. That they think this is the "right" way to solve the problem. And because we push them towards publishing their solution as a package, they will likely have other users that they will talk to in issues, they'll iterate on the problem, they'll identify the problems with their solution, and then at the end of the day we have download stats/github stars/etc to show us how broad of a problem this is. All of that together will allow Parcel to make a really informed opinion.

@jakearchibald

On the other hand, it makes rapidly developing a plugin pretty painful, especially at the start of a project where there may be a lot of plugins, that are later rolled into one, or one plugin that's split into many etc etc, so there's a barrier there that might make me think, "nah I'll just use Rollup instead".

I think this is a good point, Parcel should be addressing how difficult it is to develop a plugin. There are ways of solving this I think:

  • Ship a scaffolding mechanism in Parcel to easily create a plugin package
  • Allow local/linked plugins to be run when NODE_ENV=development or something along those lines

    • Alternative: Don't allow local/linked plugins when CI=true or something

    • Alternative: --dangerously-enable-local-plugin=./...

    • Alternative: Do something to people's apps that they would never want to ship to production

@jamiebuilds thanks for explaining the rationale, it does make a lot of sense. For what it’s worth, I didn’t have to publish my custom resolver, all I had to do was add it to my dependencies like this:

{
  "dependencies": {
    "parcel-resolver-esm": "file:packages/parcel-resolver-esm"
  }
}

Hey folks,

Nothing smart to add to this discussion but I recently tried to create some plugins and I used @speigg technique with npm file:. I would rather be able to use a relative path inside .parcelrc.

I think I get the "Parcel should just work for everyone" but I share many of @jakearchibald arguments:

I've definitely got a lot of 'local' plugins that I use in various Rollup projects.

On the other hand, it makes rapidly developing a plugin pretty painful...

Just to add another use-case to this, the reason I want to reference a local plugin is so that I can test the plugin does what is expected

I don't want to unit test the individual functions I pass to new Transformer({ ... }) because I don't want to test internals of Parcel

Was this page helpful?
0 / 5 - 0 ratings

Related issues

oliger picture oliger  Ā·  3Comments

466023746 picture 466023746  Ā·  3Comments

humphd picture humphd  Ā·  3Comments

davidnagli picture davidnagli  Ā·  3Comments

algebraic-brain picture algebraic-brain  Ā·  3Comments