Webpack-cli: [ROADMAP] Webpack-CLI v4 Roadmap

[evenstensberg]

Summary

This is the roadmap to keep progress of webpack-cli v4 which is on the next branch of webpack.

TODO

Stable

• [ ] Choose a lead developer ALWAYS TO READ

In order to avoid conflicts and misunderstandings what happened before, together we select a leading developer. This person makes the final decision, when a dispute arises. If a situation arises when the whole team believes that the lead developer is wrong, then one of the developers from @webpack/core-team is involved in the dispute, his/her/it decision is last and not subject to dispute.

• [x] CLI arguments from core https://github.com/webpack/webpack/pull/10515 + tests
• [x] Allow multiple types for arguments, i.e.
  webpack --stats (means stats: true),
  webpack --stats=verbose (means: stats: verbose),
• [x] Allow negative boolean args, i.e. webpack --no-stats (means: stats: false)
• [x] Allow multiple same arguments, i.e. webpack --entry=./src/one.js --entry=./src/two.js
• [x] To solve the points above, we may need to seach good CLI helper, like yargs-parser and etc. Update - we decided to go with commander.js
• [x] Verify webpack-dev-server + webpack-cli:
  
  
  
  • [x] Tests for webpack serve
  
  
  • [x] Relax dependencies, i.e. webpack-cli should works with webpack-dev-server@3 and webpack-dev-server@next
  
  
• [x] Compatibility with webpack@4 + tests (always)
• [x] Write Documentation, need clarify What we should write
• [x] Refactor https://github.com/webpack/webpack-cli/blob/next/packages/webpack-cli/lib/utils/cli-flags.js . We need to determine what arguments we are leaving, what we remove in favor CLI arguments from core, for example --sourcemap is not good candidate (https://github.com/webpack/webpack-cli/blob/next/packages/webpack-cli/lib/utils/cli-flags.js#L236), because --devtool used in webpack and most developers are used, but verbose is used in most of CLI tools and will be great to keep it for better DX (it is alias for --stats=verbose), not sure, it can be misleading, need dicussion
• [x] 0CJS: webpack-cli should be shipped with info and init commands out of box, need discussion about serve command
• [x] 0CJS: webpack-cli should works without the --config option + tests
• [x] Update all dependencies to latest and setup dependabot
• [x] Part of the code in JS, part in TypeScript, let's choose one
• [x] Default stats value should be taken from user configurations, if user doesn't provide stats options (or aliases like verbose/quiet)
• [x] Resolve stale PRs (close/rewrite/add to project webpack-cli@5 https://github.com/webpack/webpack-cli/projects/5)
• [x] Resolve stale Issues (close/rewrite/add to project webpack-cli@5 https://github.com/webpack/webpack-cli/projects/5)
• [x] Resolve all TODO in tests and fix smotetest for windows

Post-beta (non-blocking)

• [x] Try to avoid using unnecessary dependencies, we should try to keep CLI small and fast
• [x] Clean up code
• [x] More Smoketests and improving watch smoketest
• [ ] webpack interactive
• [ ] Allow other packages register own command and flags for CLI
• [ ] Union generate-loader + generate-plugin + webpack-defaults, we need discussion how to better to do it
• [x] Simplify and reduce unnecessary architecture stratification, for example https://github.com/webpack/webpack-cli/blob/next/packages/logger/src/index.ts, we do not need packages consisting of three lines of code, Need to look at all the other packages.
• [x] Increase coverage
• [x] Setup webpack bot for github actions

Feel free to come up with feedback

[renatoagds]

@evenstensberg cli-gui will be released with next or standalone?

[ematipico]

The UI package will be released as a standalone package, probably we will release a minor version just to support the cli command

[daniele-orlando]

Nice roadmap of solid improvements.

[evenstensberg]

Roadmap blocking a beta as per now:

• Progress flag needs a rebrush
• Adding interpret
• Supporting mjs and pre-processing files
• Adding verbose flags
• Dev-server version bump

[evenstensberg]

Removed webpack-config-plugins as it's not fit for webpack v5 yet. Also upgraded our peerDep + dep to webpack v5. After fixing failing tests and those regressions we are go.

[TheLarkInn]

Let's add webpack/webpack-cli#1126 to blocking list.

[godcrampy]

I'd like to write some smoke-tests! Can I get some pointers as to what kind of smoke tests are to be written besides the ones which are already present. What all stuff are these tests supposed to display?

[evenstensberg]

Fixing the smoketests for watch mode would be really nice, right now those are kinda flaky, so we've disabled the CI to run them.

Could you check that out? The tests are supposed to show that the CLI is rigid and doesn't break under harsh conditions 👍

[endiliey]

Hi there, this is sort of unrelated but can I suggest to use "Squash and Merge pull request" instead of "Merge pull request" moving forward. Currently, its very hard to track changes between version of webpack-cli because a lot of commits in a PR is all retained. Here is an example:

https://github.com/webpack/webpack-cli/compare/v3.3.5...next

Due to using "merge pull request", there are lot of "extra" commits being sent here. Even something like "fix: revert :("

If squash merge was used, it will be something like below, which is more cleaner

• chore(init): rephrase comment #1061
• chore(compiler): refactor emoji rendering logic in compiler #1060
• chore: refactor #1058
• chore(GroupHelper): minor tweak #1059

An example is https://github.com/webpack/webpack-dev-server/compare/v3.8.2...v3.9.0
Less number of commits, and related PR is linked

[evenstensberg]

I think one of the reasons why we changed from squash to merge was because people wanted the entire commit log, but I'm happy to do squashing, it's cleaner, if there's no objections

[godcrampy]

I tried running the smoketests as is without any changes. Currently they return Test for child compilation hang, exiting. Is this _delay_ a problem with core webpack or is the current smoketest causing it to behave this way.

Also, for the temporary files that are created during the smoketest, will it help if instead of appending comments we add some javascript snippets to check the watch mode performance.

[evenstensberg]

Do you have a stack trace @godcrampy ?

[godcrampy]

When I run $ npm run smoketest I get the following:

============================ RUNNING FOLDER: watch ============================
============================ ITERATION: 1/100  =====================================
Running: ./watch/watch.array.smoketest.js
Running: ./watch/watch.single.smoketest.js

============================ ARRAY/CHILD COMPILATION ============================

[
  {
    output: {
      filename: './prod.js',
      path: '/home/godcrampy/Code/Sandbox/node-sandbox/webpack-next/repos/webpack-cli/smoketests/watch/binary2'
    },
    target: 'node',
    entry: '/home/godcrampy/Code/Sandbox/node-sandbox/webpack-next/repos/webpack-cli/smoketests/watch/index.js',
    mode: 'production',
    devtool: 'eval-cheap-module-source-map',
    plugins: [ [DefinePlugin] ]
  },
  {
    output: {
      filename: './dev.js',
      path: '/home/godcrampy/Code/Sandbox/node-sandbox/webpack-next/repos/webpack-cli/smoketests/watch/binary'
    },
    target: 'node',
    entry: '/home/godcrampy/Code/Sandbox/node-sandbox/webpack-next/repos/webpack-cli/smoketests/watch/index.js',
    mode: 'development',
    plugins: [ [DefinePlugin] ]
  }
]

assert.js:92
  throw new AssertionError(obj);
  ^

AssertionError [ERR_ASSERTION]: Test for child compilation hang, exiting
    at Timeout._onTimeout (/home/godcrampy/Code/Sandbox/node-sandbox/webpack-next/repos/webpack-cli/smoketests/watch/watch.array.smoketest.js:63:24)
    at listOnTimeout (internal/timers.js:531:17)
    at processTimers (internal/timers.js:475:7) {
  generatedMessage: false,
  code: 'ERR_ASSERTION',
  actual: true,
  expected: false,
  operator: 'strictEqual'
}

============================ SINGLE COMPILATION ============================

{
  entry: '/home/godcrampy/Code/Sandbox/node-sandbox/webpack-next/repos/webpack-cli/smoketests/watch/index.js',
  target: 'web',
  output: {
    filename: 'bundle.js',
    path: '/home/godcrampy/Code/Sandbox/node-sandbox/webpack-next/repos/webpack-cli/smoketests/watch/bin'
  }
}

assert.js:92
  throw new AssertionError(obj);
  ^

AssertionError [ERR_ASSERTION]: Test for child compilation hang, exiting
    at Timeout._onTimeout (/home/godcrampy/Code/Sandbox/node-sandbox/webpack-next/repos/webpack-cli/smoketests/watch/watch.single.smoketest.js:58:24)
    at listOnTimeout (internal/timers.js:531:17)
    at processTimers (internal/timers.js:475:7) {
  generatedMessage: false,
  code: 'ERR_ASSERTION',
  actual: true,
  expected: false,
  operator: 'strictEqual'
}

The exact iteration it fails varies between 1 to 3 for my machine. The current error is shown when the webpack compiler exceeds _5_ seconds for compiling the given changes on watch mode.I think that the arbitrary five second wait time to decide whether the process is stuck is a bad metric.

These exceptions come from here:
https://github.com/webpack/webpack-cli/blob/next/smoketests/watch/watch.single.smoketest.js#L56
https://github.com/webpack/webpack-cli/blob/next/smoketests/watch/watch.array.smoketest.js#L60

[ematipico]

At the moment the output of the new CLI is not correct. It doesn't provide all the information.
It should provide the exact same information that the function stats.toString(options.stats) emits.

Also, the result of current output is misleading and incorrect. I will take a shoot at that if anyone won't.

[rishabh3112]

@ematipico
What all is needed to be done for standard output?, like in what format should I give output.

[ematipico]

I will tackle the --verbose flag

[LunaticProgrammer]

Hi @evenstensberg I would like to work on the documentation of the Webpack-CLI v4.

[parth-gr]

Hi, I would like to work on the documentation of the new Webpack-CLI.

[rishabh3112]

@LunaticProgrammer @parth-gr Thanks for showing interest. You may start by playing around webpack-cli and studying changelog. Ask anything if you need more help.

[Parikshit-Hooda]

I am in too. Would love to take up this project if selected for GSoC.

[Parikshit-Hooda]

@rishabh3112 Is there a slack or a communication line/group that I can enter?

[rishabh3112]

This issue is the best place for discussing about webpack-cli v4 documentation.

[Parikshit-Hooda]

Update: I have gone through a youtube playlist on webpack. Would go through some more videos and blogs today and hopefully do some meaningful activity on this repo. Thanks.

[LunaticProgrammer]

Hii @rishabh3112 can you please link the changelog here

[rishabh3112]

Here is the changelog https://github.com/webpack/webpack-cli/blob/next/CHANGELOG.md.

[alexander-akait]

@webpack/cli-team Please :+1: on this comment who are actively working on the package and who wants to.

[alexander-akait]

@webpack/cli-team Great!

Let's start discussion about:

Part of the code in JS, part in TypeScript, let's choose one

I think that we once made a choice in favor of TypeScript, so let's stick to it, it is not high priority because we more priority issues, but we should create new code using TypeScript and after stable release migrate rest of a code to TypeScript, anyway if somebody want to do it right now, we can coordinate it and do it right now

Put :heart: on this issue if you agree or write comment here with with arguments why you do not agree

[snitin315]

I will migrate the code for webpack-cli to TypeScript. :smile:

[rishabh3112]

Comparison between different arg parsers
| Name | --arg=value | --no-arg | defaults | Install size | Repo | npm |
|--------------|-------------|----------|----------|---------|---------------------------------------|--------------------------------------------|
| yargs-parser | Yes | Yes | Yes | 81.6 kB | https://github.com/yargs/yargs-parser | https://www.npmjs.com/package/yargs-parser |
| minimist | Yes | No | Yes | 32.4 kB | https://github.com/substack/minimist | https://www.npmjs.com/package/minimist |
| commander | Yes | Yes | Yes | 102 kB | https://github.com/tj/commander.js | https://www.npmjs.com/package/commander |
| | | | | | | |

[rishabh3112]

@webpack/cli-team fill up above table / suggest other options you would like to be considered. Thanks!

[alexander-akait]

Vote for commander.js

[alexander-akait]

@rishabh3112 Looks all votes for commander.js, you can start a work

[snitin315]

Regarding --stats, should we apply a general solution like --multiType-flag which will allow -

• webpack --multiType-flag => multiType-flag: true
• webpack --multiType-flag value => miltiTypeFlag: 'value'

Or we will be supporting multi types for stats only?

[alexander-akait]

Why we need support multiple stats?

[snitin315]

not multiple stats but multiple types for stats.

I'm thinking to add a property multiType : true for stats inside cli-flags.js ?
Any suggestions ?

[alexander-akait]

@snitin315 I can't understand, please clarify

[snitin315]

No worries I will create a Draft PR for same.

[snitin315]

0CJS: webpack-cli should be shipped with info and init commands out of box, need discussion about serve command.

I think now cli supports info and init out of the box.
https://github.com/webpack/webpack-cli/blob/f1876c125be90e0b0ae96e6e9dbaf91ed447c779/packages/webpack-cli/package.json#L27-L28

/cc @rishabh3112 am I right?

[rishabh3112]

Yes, but serve is still left for integration as decided.

[alexander-akait]

@rishabh3112 Let's send a commit with integration, we will fix an problems with options late

[rishabh3112]

@evilebottnawi Sure, I will send PR then.

[billyjanitsch]

Hi, I just wanted to cross-link this comment https://github.com/webpack/webpack-cli/issues/1222#issuecomment-697154223 because this might be the more appropriate issue:

Please, please reconsider having webpack-cli depend directly on @webpack-cli/init (#1609). The latter is an enormous module with so many dependencies (particularly jscodeshift, which is huge and very outdated). This doesn't affect bundle size, of course, but it does affect install time and npm's ability to dedupe modules. Plus, the init functionality is only useful when initializing a project, and most projects probably don't use it to initialize themselves anyway.

This issue came up back when webpack v4 was in beta. There's nothing wrong with @webpack-cli/init existing, but webpack-cli is currently _required_ to invoke webpack via CLI and it would be really nice to have some way of doing this without bringing in half of the npm registry. All that most people need is to be able to add a simple build script to package.json. webpack-cli v3 did the right thing by including only this minimal core.

In an empty project, npm install webpack-cli@next (4.0.0-rc.0) currently installs 82mb of dependencies (!) whereas npm install webpack-cli@3 only installs 6mb.

[alexander-akait]

@billyjanitsch yes, we will do it

[lopopolo]

@evilebottnawi is there a ticket to track the removal of @webpack-cli/init as a required dependency of webpack-cli?

[alexander-akait]

@lopopolo We search way to not remove it, but reduce size

[alexander-akait]

webpack interactive

We have separate issue and PR for this

Allow other packages register own command and flags for CLI

Already supports, but not documented, we need do some improvements (I will do it in near future)

Union generate-loader + generate-plugin + webpack-defaults, we need discussion how to better to do it

We have separate issue for this

Todo for tests also have own issues

So let's close