Snowpack: New Config Format

[FredKSchott]

Spun off from this conversation: https://github.com/pikapkg/snowpack/pull/135#issuecomment-573448468

Our configuration right now is spread across a mix of CLI flags and package.json configuration. Now that the project has grown, at this point i think we're well overdue for a top-level snowpack.config.json configuration file.

I'd also like to use this as an opportunity to clean up our different options all into one place. It would be great if we could mimic TypeScript's behavior here, where every config file value can also be passed in as a CLI flag, so that effectively you have 1 config interface but you can pass those configuration values via both CLI flag and read from config.

The only thing that may make sense to keep in package.json would be "webDependencies", since that's so closely tied to "dependencies"/"devDependencies". But, maybe that's just can support in both places, similar to how "@babel/preset" will grab your supported browsers from either your babelrc configuration or the top-level "browserlist" property.

Would love help on this if anyone's interested. The first step will be to cleanup our options into a single table like this: https://www.typescriptlang.org/docs/handbook/compiler-options.html We can do that in this issue (or a Gist) before ever committing to code. Then the other tricky part will be getting the configuration reading logic hammered out (which files take precedence, do we read up the file tree to support monorepos, etc). There may be libraries/packages that can help us out here.

[drwpow]

Something like this?

| Option | Type | Default | Example | Description |
|:--------------|:-----------:|:----------------|:----------------------------|:------------------------------|
| --dest | string | web_modules | --dest dist/web_modules | Specify destination directory. |
| --clean | boolean | false | --clean | Clear out the destination directory before install. |
| --optimize | boolean | false | --optimize | Transpile, minify, and optimize installed dependencies for production. |
| --babel | boolean | false | Transpile installed dependencies. Also enabled with --optimize. |
| --include | string | — | --include 'src/**/*.js' | Auto-detect imports from file(s). Supports glob. |
| --exclude | string | '**/__tests__/*' '**/*.@(spec\|test).@(js\|mjs)' | --exclude '*.test.js' | Exclude files from --include. Follows glob’s ignore pattern. |
| --strict | boolean | false | --strict | Only install pure ESM dependency trees. Fail if a CJS module is encountered. |
| --no-source-map | boolean | false | --no-source-map | Skip emitting source map files (.js.map) into dest. |
| --remote-package | string[] | — | --remote-package react,16.12.0 react-dom,16.12.0 | name,version pair(s) of packages that should be left unbundled and referenced remotely. Example: foo,v4 will rewrite all imports of foo to {remoteUrl}/foo/v4 (see --remote-url).
| --remote-url | string | https://cdn.pika.dev | --remote-url https://unpkg.com | Configures the domain where remote imports point to. |
| --nomodule | string | — | --nomodule src/index.js | Your app’s entry file for generating a <script nomodule> bundle|
| --nomodule-output | string | app.nomodule.js | --nomodule-output dist/app.nomodule.js | Filename for nomodule output |

It would be great if we could mimic TypeScript's behavior here, where every config file value can also be passed in as a CLI flag, so that effectively you have 1 config interface but you can pass those configuration values via both CLI flag and read from config.

+1. I personally will use CLI flags in npm scripts up until ~3 or 4 then prefer to move to a config. But for those libraries that mostly work out-of-box, it’s nice to not have all that setup. I like providing both options for people.

Or even mixing-and-matching. Sometimes my TypeScript configs can get long, so it’s nice to be able to run tsc --noEmit so that it respects the entire config file, but I can simply type-check without output.

Then the other tricky part will be getting the configuration reading logic hammered out (which files take precedence, do we read up the file tree to support monorepos, etc). There may be libraries/packages that can help us out here.

I’m interested in the config in general, but not familiar with this part myself. I’ll start looking at how other libraries handle it (Jest, Prettier, Babel, ESLint, etc.)

[drwpow]

Also what are your thoughts on exposing Rollup config overrides as part of this (e.g. a user could add a Rollup plugin for snowpack to use on top of the base config)? It’d definitely be advanced, but would help give people more utility for their app at no cost.

[Monchi]

+1 to support both CLI option and config file. Basically configurations should be constructed with easy-to-read configuration file such as snowpack.config.json, then CLI options overrides that basic configuration.

Also how about something like snowpack.config.js? This would be useful in the situation below.

const isProd = process.env.NODE_ENV === 'production';

module.exports = {
  dest: 'web_modules`,
  optimize: isProd,
}

Prettier uses cosmiconfig to handle multiple configuration formats. The configuration doc would be helpful.

[FredKSchott]

@Monchi cosmicconfig looks good, and I'd heard it mentioned for this kind of thing before as well. One thing that it looks like it doesn't tackle is merging with CLI args, so we'd have to do that ourselves (probably by just keeping our current yargs setup).

@dangodev I'd like to see a good use-case for supporting a custom rollup config (that can't be solved by improving webpack) before we add it, but assuming there is one out there and we see it, then yes I'm all for custom rollup config. namedExports & dedupe are already kind of going down this route, I just really want to make sure that if we do add it we make sure that learning-Rollup-implementation-details-via-config is a last resort.

That table looks great! If you even want to go ahead and add that to the docs site separately, I think there's value in getting that up onto the docs site today :D

@dangodev I know you expressed interest in this specific project, do you want to own this? It's not super urgent so no rush, and i think we can do it without a breaking change / forcing a v2.0 bump.

[drwpow]

Yeah! I’ll probably take a crack at a rough prototype just for discussion later this week or early next at the latest

On Jan 20, 2020, at 23:18, Fred K. Schott [e-mail] wrote:


@Monchi cosmicconfig looks good, and I'd heard it mentioned for this kind of thing before as well. One thing that it looks like it doesn't tackle is merging with CLI args, so we'd have to do that ourselves (probably by just keeping our current yargs setup).

@dangodev I'd like to see a good use-case for supporting a custom rollup config (that can't be solved by improving webpack) before we add it, but assuming there is one out there and we see it, then yes I'm all for custom rollup config. namedExports & dedupe are already kind of going down this route, I just really want to make sure that if we do add it we make sure that learning-Rollup-implementation-details-via-config is a last resort.

That table looks great! If you even want to go ahead and add that to the docs site separately, I think there's value in getting that up onto the docs site today :D

@dangodev I know you expressed interest in this specific project, do you want to own this? It's not super urgent so no rush, and i think we can do it without a breaking change / forcing a v2.0 bump.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

[drwpow]

Just poking around at how some other projects (that I associate with good configuration) deal with configs: they usually have some src/config/ folder with custom logic that deals with resolution and priority.

• Babel
• Prettier
• Jest
• ESLint

So from a small sample, it seems like the following is necessary:

1. Establish a priority order (e.g. for Prettier, package.json takes priority over .prettierrc, .prettierrc takes priority over .prettierrc.json, …)
2. Start with a default config
3. Once a config is found, validate the config, and throw an error if the config doesn’t match the schema (the more specific the error, the better)
4. If the config is valid, apply it over the default config
5. When running the CLI, prioritize flags > config file
6. Have tests that make sure that all the above 👆 doesn’t break

I’ll look into cosmicconfig and some other solutions, but it looks like the popular projects still have a good deal of custom logic (Prettier is by far the simplest, and I’ll probably try to mirror that as much as possible). But regardless, I think the above is a pretty good criteria.

Bonus (maybe out of scope, but to consider)

It’d be awesome to look into a VS Code plugin (a la Prettier) that helps users with configs. I’ve run across that a lot lately—I’ve been diving into Google Cloud and their VS Code extension is a huge help in providing good feedback in YAML files.

Also marketing and all that stuff.

[Monchi]

@dangodev
Looks good!

I’ve been diving into Google Cloud and their VS Code extension is a huge help in providing good feedback in YAML files.

Absolutely. I can work on this in parallel with you.

[FredKSchott]

1 thing that I want us to keep in mind is that those are all big projects, and we're still relatively small. They can afford to take on the maintainance costs of a big project like custom config loading (across monorepos, nesting, merging, multiple config languages, etc). We can't.

But I definitely see the benefits to rolling our own: we can keep it simple, not try to support every single config language imaginable = cleaner docs, more deterministic behavior = easier to debug, bring in as few dependencies as possible, etc. So if we do roll our own I'd want to lean into those.

So I'd personally prefer to start with cosmicconfig (limiting languages/nesting/merging/behavior as much as possible for v1) and then only replace that with our own if we weren't happy with it. But, if rolling our own is a project people are interested in implementing, I support the work as long as we are targeting the benefits/goals outlined above.

[FredKSchott]

Completed in #184 , will go out in next version