Copy-webpack-plugin: New files are not being copied in watch-mode

[ekonoval]

I skimmed over the related bugs (#252 )
and they seem to be fixed, so I thought maybe it's a new bug in the newest 5-th version as minor version is low.

• Operating System: windows 10
• Node Version: 10.15.3
• NPM Version: 6.4.1
• webpack Version: 4.31.0
• copy-webpack-plugin Version: 5.0.3

Expected Behavior

On watch start everything works fine
http://prntscr.com/nt54s1

then I copy new image file into /assets/sub/3.jpg

I expect the new file to be copied to /dist/assets/sub/3.jpg
cause watcher is running.

Actual Behavior

New file doesn't get copied into /dist/assets/sub/3.jpg

http://prntscr.com/nt560z

until I rename some existing file (1.jpg or 2.jpg)
then 3.jpg gets properly copied suddenly.
Or I should re-run watcher command or build again

Code

I have a simplified config like this:

const path = require('path');
const CleanWebpackPlugin = require('clean-webpack-plugin');
const CopyWebpackPlugin = require('copy-webpack-plugin');


module.exports = {
    mode: 'development',

    devtool: 'inline-source-map',

    entry: {
        'main': './assets/fake.js',
    },
    output: {
        filename: '[name].bundle.js',
    },

    plugins: [
        new CleanWebpackPlugin({verbose: true}),
        new CopyWebpackPlugin([
                {
                    from: './assets/**',
                    ignore: ['*/_src/**/*'],
                    to: '[path][name].[ext]',
                }
            ],
            {copyUnmodified: true}
            //{logLevel: 'debug', copyUnmodified: true}
        )
    ],
};

How Do We Reproduce?

I've created a repo where you can reproduce this behavior
https://github.com/ekonoval/webpack-copy

[alexander-akait]

Thanks for issue, i investigate this in near future, maybe problem on webpack side

[0nky0]

it is not only for new files, it happens even on existing files.

when i run webpack for the first time, webpack output is fine. but. on recompilation, the webpack output is missing all the files given to copy-webpack-plugin to handle.

is it related to webpack itself?

here are some more details about versioning:

$ node -v
v11.15.0
$ npm -v
6.9.0
$ cat package.json | grep copy-webpack-plugin
    "copy-webpack-plugin": "^5.0.3",
$ cat package.json | grep '"webpack'
    "webpack": "^4.34.0",
    "webpack-cli": "^3.3.4",
    "webpack-dev-server": "^3.7.1"

[0nky0]

@evilebottnawi: although you are extra busy. can you please only state whether there is an issue with the module itself or is it webpack?

[alexander-akait]

@y0y0z please create reproducible test repo, i think your problem not related to above

[0nky0]

@evilebottnawi: thank you for the quick reply. i can try and create a reproducible test but it will take me looots of time to strip my code in order to create one. anyhow, i am including more details and i hope it will shed some light

i configured the copy plugin to manage the following files:

• favicon.ico
• mainfest.json
• robots.txt

this is a snippet of the output for the first time webpack-devserver is launched. see how the files above are being managed:

⚠ ｢wdm｣: Hash: 3c7ff4e45d92d4c2b4e9
Version: webpack 4.34.0
Time: 11043ms
Built at: 06/13/2019 12:21:21 PM
                                Asset       Size  Chunks             Chunk Names
                               app.js    436 KiB    main  [emitted]  main
                          favicon.ico    163 KiB          [emitted]  
                           index.html  500 bytes          [emitted]  
                        manifest.json  350 bytes          [emitted]  
                           robots.txt   26 bytes          [emitted]  

but, when changed detected and re-compilation occur. the output above is different. the files above are not being manged:

ℹ ｢wdm｣: Compiling...
⚠ ｢wdm｣: Hash: 5d004963b5ca7f9a1877
Version: webpack 4.34.0
Time: 558ms
Built at: 06/13/2019 1:17:18 PM
                                Asset       Size  Chunks             Chunk Names
                               app.js    436 KiB    main  [emitted]  main
                           index.html  500 bytes          [emitted]  

[alexander-akait]

What is version webpack-dev-server? Looks like bug in webpack-dev-server@2 (now version is 3). Anyway will be great to look on reproducible test repo, looks like something wrong in configuration (but can be bug).

[0nky0]

@evilebottnawi: as you can see in my previous comment, i am using [email protected].

i am sure that a shared repository will be much easier to look for the problem. but can you still please share what could be the issue with the configuration if it works except for re-compilation?

[alexander-akait]

@y0y0z

i am sure that a shared repository will be much easier to look for the problem

Yes, feel free to ping me when you done this

Can you still please share what could be the issue with the configuration if it works except for re-compilation

Without configuration i can't say, it's just my guess

[0nky0]

@evilebottnawi: since i cannot share all the code online even after i stripped it, i have created a private repository an assigned you as a contributor. would appreciate your assistance :)

[alexander-akait]

@y0y0z thanks, can you provide link on repo (no invites on email :confused: )

[0nky0]

@evilebottnawi: i have just modified my previous comment to include a link.

[alexander-akait]

Thanks again, in my todo list (try to find time on this week to investigate this problem, there are a lot of other issues)

[frehner]

I think I just ran into the same issue as outlined by @y0y0z above:

• initial compilation copies files
• subsequent compilations do not include those files, _unless_ the files themselves were changed

of note, I'm using clean-webpack-plugin to remove all items in the directory on every compilation (due to restrictions of the environment I'm working in), so in effect every time I save any file, the files watched by copy-webpack-plugin are removed

If you would like a repo to look at, I can provide that as well.

[mathiscode]

+1 - I had to alter my workflow to not use copy-webpack-plugin because of this 🙁

[veeramarni]

@frehner I guess your solution looks like a workaround. Can you share your changes?

[frehner]

My current workaround was actually to downgrade clean-webpack-plugin back to v1. For some reason that makes it all work again, so I’m not sure who’s broken right now haha. But those two plugins don’t appear to work well together at the moment.

[0nky0]

@frehner: i am suffering exactly from the same behavior and i am also using the clean-webpack-plugin version 3.0.0.
```bash
$ cat package-lock.json | jq '.dependencies["clean-webpack-plugin"].version'
"3.0.0"
````
which version did you use?

[alexander-akait]

Somebody can create minimum reproducible test repo with clean-webpack-plugin?

[0nky0]

@evilebottnawi: mine includes it and i already set you as a contributor on the repository i shared with you

[0nky0]

If you would like a repo to look at, I can provide that as well.

@frehner: please do so and make sure that clean-webpack-plugin is a part of it :)

[frehner]

If you would like a repo to look at, I can provide that as well.

@frehner: please do so and make sure that clean-webpack-plugin is a part of it :)

Here's the repo and how to get started: https://github.com/CanopyTax/single-spa-inspector#how-to-contribute

You should only need to follow steps 1-6. Once you've run npm start, there will be a folder called build that will be "cleaned" and then assets put back into it on every file save. There's no need for you to actually open Firefox or anything :)

I just downgraded clean-webpack-plugin to v1, but you can upgrade it again to see it not working.

[0nky0]

@evilebottnawi: could you please share whether there was any progress on the matter?

[alexander-akait]

@y0y0z it is open source, i can't spend all of my time on this, you can start testing and search problem, it is in my todo

[0nky0]

@evilebottnawi: i know it is, was only asking to make sure that both os us are not spending precious time on the same thing. anyhow, i love you :)

[alexander-akait]

@y0y0z feel free to investigate, i am write here when i starting search problem

[kdepp]

I found the root cause after reading thru change log and code of clean-webpack-plugin

Since [email protected], you need to add an option to prevent unchanged files from removal, while copy-webpack-plugin only emit files when they are changed.

new CleanWebpackPlugin({
    cleanStaleWebpackAssets: false,
});

https://github.com/johnagan/clean-webpack-plugin/issues/125

So their "breaking" changes accidenlty affect this lib. 😭

[0nky0]

@kdepp:
thank you. now with the cleanStaleWebpackAssets option, the webpack output directory holds the files that were not changed (though the verbose output of copy-webpack-plugin seems not to show the difference in execution when toggeling copy-webpack-plugin).

@frehner:
please verify that it solves your issue.
if it does, let's page @evilebottnawi in order to not invest time on the issue and close it.

[frehner]

https://github.com/CanopyTax/single-spa-inspector/pull/21

Looks like that worked. Thanks for figuring it out!

[prograhammer]

Looks like assets won't get removed with this temporary work-around. If you delete a file, you'll see it doesn't get removed in the target/dist directory.

[dkthehuman]

@evilebottnawi If it'd still help, I created a minimal test repo to demonstrate the bug along with the repro steps in the README: https://github.com/dkthehuman/webpack-copy-bug

[alexander-akait]

@dkthehuman https://github.com/webpack-contrib/copy-webpack-plugin/issues/385#issuecomment-508914721

[alexander-akait]

In next major release we will always emit assets, so no need this option for CleanWebpackPlugin

[alexander-akait]

Close in favor https://github.com/webpack-contrib/copy-webpack-plugin/issues/261, will be fixed in next release, workaround:

 new CleanWebpackPlugin({
      cleanStaleWebpackAssets: false,
    }),

[hdodov]

I think the problem isn't in clean-webpack-plugin. I'm having the same issue and although I'm using clean-webpack-plugin too, I tried both setting cleanStaleWebpackAssets: false and disabling it completely, but that didn't resolve the problem.

Here's my config:

new CopyWebpackPlugin([
    {
        from: 'src/icons',
        to: 'icons'
    }
]),

Basically, I want dist/icons to be a mirror of src/icons.

And by the way, clean-webpack-plugin has a setting to ignore only some files:

new CleanWebpackPlugin({
    cleanAfterEveryBuildPatterns: [
        '!icons/**/*'
    ]
}),

So instead of setting cleanStaleWebpackAssets to false, you can simply tell the clean-webpack-plugin to ignore whatever is copied by copy-webpack-plugin.

[Bjego]

@evilebottnawi which version do you mean with the next major version? Is it 6.0.0, or is it already released?

[ricokareem]

thank you to everyone on this thread. it seems as though the combination of cleanStaleWebpackAssets: false, and copyUnmodified: true, works for my project.

        new CleanWebpackPlugin({
            cleanStaleWebpackAssets: false,
        }),
        new CopyPlugin([
            {
                from: '**/*.tpl',
                to: 'templates',
                context: '_assets/templates',
                copyUnmodified: true,
            },
        ]),

note: you can also add force: true to CopyPlugin just in case, but for me it wasn't necessary.