Webpack-dev-server: Docker support

[geddski]

currently when running webpack dev server inside of a docker container, the changes to the source files are not picked up by webpack, so no compilation happens.

Using boot2docker vm on OSX.

[crestenstclair]

+1 having the exact same issue, would appreciate an update

Is essentially this issue:

https://github.com/remy/nodemon/issues/419

[Rodeoclash]

+1

[bradbarrow]

+1

[yvann]

+1 the same for me, I can't use it.

[evandegr]

+1

[myzone]

+1

[muraken720]

+1

[joernroeder]

+1

[johnknaack]

+1

[saulshanabrook]

Use this project: https://github.com/brikis98/docker-osx-dev

It picks up file changes events

[tomchentw]

docker-osx-dev works great for me. Also its woking well with piping.

[natemara]

I am having the same problem, but I feel it's for a different reason. I'm using the --host 0.0.0.0 option to allow the server to listen on all interfaces, but when used in conjunction with the --hot flag, this causes problems. The hot reloader uses the provided host to create an absolute url instead of using a relative url, so all of my reloading ajax calls are going to http://0.0.0.0:8080/socket.io/ instead of just socket.io/, which would work no matter what the host/port was. I'm also proxying the server behind nginx that happens to be running on the same port, but if I wasn't, then this issue would be even worse.

[xaviervia]

+1

[vectart]

+1

[xaviervia]

@natemara about your issue, I worked around that by setting the config:

module.exports = {
  entry: {
    app: [
      "webpack-dev-server/client?:8080/",
      'webpack/hot/dev-server',
      "./src/app.coffee"
    ]
  }
}

...buuut, big disclaimer, it just fixed the socket.io address, not the livereload, which is still not working (webpack doesn't rebuild the files)

[fordlee404]

+1

[plouc]

+1

[svenhornberg]

+1

[jweidler]

+1

[ghost]

The problem has been sovled by Mark Wolfe, I have tested it in my MacBookPro.

[fordlee404]

I fixed the problem with webpack config:

watchOptions: {
    poll: true
}

@fzx thanks!

[vyorkin]

@sokra I believe it's not related to webpack-dev-server. It's just vboxfs that doesn't support fsevents or inotify.

[vyorkin]

@geddski p.s.: here is a good post about dev env with docker on OS X. My solution is to just set poll: true or if polling is too slow for you use smth like dinghy.

[langri-sha]

+1

[jimthedev]

Seems like this should probably be closed since the problem is with Virtual Box's (likely reasonable) decision to not pass through inotify events. The current relatively common fix is to use https://github.com/brikis98/docker-osx-dev

This is true for anyone using node, python, or any other language in docker who wants watchers. It really has little to do with Webpack. Perhaps it just needs to be documented, but there's little to do from a code standpoint.

[langri-sha]

@jimthedev, apparently it's not even related to just vboxsf. It is a known issue for all NFS mounts.

Here is an excerpt from the PM2 documentation:

When working with NFS devices you'll need to set usePolling: true as stated in this chokidar issue.

It seems that the best course of action is to make this information more prominent by putting it in the documentation where suitable, as you suggest.

[jimthedev]

@langri-sha, the tricky part of documenting/educating on this is that polling has implications for large project structures. It isn't always the best option. In some scenarios it might be ideal, in others, a better option might be to use docker-osx-dev to marshall the diff'ing through alternative means while leaving your webpack config untouched.

[marlonbernardes]

Thanks @fordlee404, setting watchOptions also worked for me!

watchOptions: {
    poll: true
}

[nrempel]

I believe this is an issue with docker (caused by virtual box and other virtualization tools not supporting inotify events), not webpack-dev-server.

There is a proposed solution here: https://github.com/docker/docker/issues/18246

[jimthedev]

You are correct. This is an issue with Virtual Box.

[vinogradov]

watchOptions: {
    poll: 1000 // <-- it's worth setting a timeout to prevent high CPU load
},

[yangmillstheory]

Changes are picked up by the container in Docker Mac Beta.

[bartvanremortele]

For anyone running into this issue that is using a Jetbrains IDE like Webstorm, Intellij, ... Try:

Please try turning 'Safe write' option ( Settings | Appearance & Behavior | System Settings | Use "safe write" (save changes to temporary file first)) off

Took me about 4 hours to figure this out.

[mrnonz]

watchOptions: {
    poll: true
  }

This worked for me !!

[Rastusik]

just to recap: the solution from @bartvanremortele works better than the one from @vinogradov and @mrnonz , because there is no cpu overhead at all.

Please try turning 'Safe write' option ( Settings | Appearance & Behavior | System Settings | Use "safe > write" (save changes to temporary file first)) off

[max-mykhailenko]

@yangmillstheory Doesn't work with docker beta 20. Can you show your webpack config. I use OSX, ST3, webpack-dev-middleware, webpack-hot-middleware and express as server.

[yangmillstheory]

@max-mykhailenko I'm on Version 1.11.1-beta12 (build: 7528).

Can you try https://github.com/yangmillstheory/react-docker-app?

[max-mykhailenko]

@yangmillstheory thanks, I'll check it and write results

[aethant]

I'm running into this issue with Docker beta 20, and what I'm noticing is that it "works" inconsistently. Same code, same configs to Docker and Webpack, and it'll work once, maybe a few times, then stop. Then won't work a number of times. Not sure what's causing that.

Noticing that specific IDEs/editors are mentioned here, too, I'm wondering if vim is part of the issue.
With the Docker+Webpack setup running, I opened Atom and attempted editing the file. No dice. Restarted Docker+Webpack, edited same file, seems to respond (worked for ~5min consistently).

Why is this (file systems aren't a topic I'm well-versed on)? Is there a way to adjust vim's settings or am I stuck with Atom for the time being?

[Rodeoclash]

@aethant Have you tried setting :set backupcopy=yes in Vim?

[max-mykhailenko]

@yangmillstheory @aethant Clean react-docker-app works! I upgrade it from node 4.4 to 6.3 and it still works. Today I'll try dev server in my config instead express. Maybe problem is there.

[aethant]

@Rodeoclash that seemed to work, but then i noticed still providing 3-4 saves before Webpack stops responding. Atom is still responding consistently.

Something I tried: started up Docker + Webpack. Opened file with vim, saved til it stopped updating. Open Atom, saves have no effect. Restart Docker + Webpack, save with Atom, ~8x. Updates consistently. Open same file in vim, save has no effect, and now saving in Atom is ignored as well. I wonder if the vim .swp file somehow is gumming things up?

[NickStefan]

Any other findings? @aethant

Also having this issue. Hot loading was working well but in the past few days the hot loading on docker has been breaking about 4 times a day. Docker-compose down && docker-compose build && docker-compose up doesnt fix it. We've been having to kill the mac meta deamon and restart it. Vim vs atom vs sublime doesnt seem to affect it (different people at our office use one or the other)

[NickStefan]

I wonder if the mac beta GUI vs the docker quickstart teminal makes any difference

[d3viant0ne]

@NickStefan - What version of Docker are you running. I haven't had any issues with Docker / Webpack since moving to the beta on mac.

@geddski As far as boot2docker support goes, I don't really see the point in supporting something that was handed it's hat quite a while ago and has now been officially deprecated in favor of docker-machine. The vboxfs has countless failings in regards to shared fs performance and reliability.
//cc @sokra ^^

@yangmillstheory - That repo also doesn't work on rc4-beta.20 for me. I'll dig into it and see if I can figure out where it's falling down.

As an aside, I'm writing docs related to Docker & Webpack2 for webpack.io and I would appreciate any insights / issues & pitfalls in webpack/webpack.io#44 from all you docker users so we can make that portion of the docs as robust as humanly possible. I believe docker for web development has significant benefits & should have first class support in Webpack.

[d3viant0ne]

@NickStefan - There is no Docker for Mac GUI as of yet. It just opens the existing Kitematic

Also, Is whatever code base you are having issue with public? I can take a look at it and see if I can work out the issue.

[NickStefan]

@d3viant0ne Not public, unfortunately, and apologies for calling it a full fledged GUI. What I meant by GUI, was the whale that gets added to the upper right (haha) versus just running the script bash --login '/Applications/Docker/Docker Quickstart Terminal.app/Contents/Resources/Scripts/start.sh'. Sounds like that inconsequential though.

• docker compose:

version: '2'
services:
   /* lots of other services */
    studio:
        build:
            context: ./node
            dockerfile: Dockerfile-studio
        volumes:
            - ./node:/usr/src
        ports:
            - "8080:8080"

• a simple docker file that exposes the dev server port
• the webpack set up from survivejs: https://github.com/survivejs/react-boilerplate/blob/master/webpack.config.js

One of the reasons I'm thinking this is webpack related is that we also have a django docker image that hot loads without any problems.

[d3viant0ne]

@NickStefan As an exercise for writing the Docker / Webpack docs I have been dockerizing popular boilerplates people are basing their projects off of.

I'll dockerize https://github.com/survivejs/react-boilerplate and hopefully it either shines a light on an issue in webpack-X or helps you work through your issue.

It's also worth nothing that it's just as possible we find an issue with the docker beta as it is we find one in webpack. There has been a significant amount of code churn for a release candidate, a theme for lots of projects these days unfortunately.

[aethant]

@NickStefan for me, no. Vim, even after applying :set backupcopy=yes, still makes Webpack inside of a container sputter out. Atom has been working without issue all the same, however.

[d3viant0ne]

@aethant - I would assume this has to do with the buffer mechanism behind vi/vim/nvim

[aethant]

@d3viant0ne Most likely you are correct. That's what I assumed, as well. But it seems like a considerable issue, given the number of vi* users. Other issues had presented the :set backupcopy=yes solution as a workaround, but to no avail. Kinda bummed. Vim mode in Atom just isn't the same. :/

[d3viant0ne]

@aethant You have a public project where you run into the described behavior? As a Linux / VI / Docker guy I would be curious to tinker and see if I can find a viable resolution to the VI / VIM issue.

[aethant]

Unfortunately @d3viant0ne, it's a work project. Here's a gist with my webpack config and how I'm starting webpack-dev-server inside of express. Hopefully it'll be of some use to you in investigating. If you need some of the Dockerfile/docker-compose.yml bits as well, let me know.

[d3viant0ne]

@aethant It's actually the docker portion I am most curious about. Trying to figure out the common denominator between my docker Webpack setup that I don't have problems with and the setups of people that do. Outside of those still on Virtualbox of course.

[aethant]

@d3viant0ne sorry, Friday fuzz brain on my part. I included the previously mentioned gist to contain my dev Dockerfile and docker-compose.yml file.

[Cmdv]

@aethant been looking all over for a solution like this, do you have a link to a repo was interested in what npm run dev did as I'm trying to sort out a very slow docker/webpack rebuild process I've inherited (40 secs!!). The only downfall is that this is also a universal app!! thanks

[aethant]

@Cmdv npm run dev runs my dev script inside package.json, which runs a gulp routine, including gulp-nodemon. I can't say I have ~40s build times (like <2s-ish), but I don't know the size of your project, either.

[Cmdv]

@aethant cool thank you, could you pop up the gulp file too? just wanting to get an idea of someone's gulp file. This project is using grunt Watch and webpack.optimize.CommonsChunkPlugin so I think this is doing a shed load of IO which is slowing stuff loads!!! (sorry not really issue related)

[d3viant0ne]

@Cmdv - I'm saying this as a gulp guy

When working within docker / shared file systems you are already going to pay a performance price. When you add gulp, which has to read more into memory from that file system for every action, that price goes up significantly.

I've cut my development cycle down considerably by limiting my gulp to utils outside of webpack and used straight npm scripts or @kentcdodds p-s when I wanted/needed to get the bloat out of my package.json or needed a bit more horsepower than an npm script could provide.

It may be worth adding a duplicate script for execution within a docker container to test out. Myself and the rest of my dev team found it to be much more performant at dev time.

[Cmdv]

@d3viant0ne thanks for the advice, that is normally how I go about projects and just have npm run my scripts. Think in this situation it's a combination that the project is using grunt watch and I'm not able to get webpack-dev-server to play nice with docker and this universal app due to the serverside rendering and also webpack is splitting into chunks.

Adding minChunks: inside the webpack.optimize.CommonsChunkPlugin({.......)}
and cacheDirectory to the loaders helped things too.

{
    test: /\.js?$/, loader: 'babel', exclude: /node_modules/,
    query: {
    cacheDirectory: true, 
    presets: ['babel-preset-stage-3', 'babel-preset-react', 'babel-preset-es2015']
}

Because right now the app's html is serverside driven I'm not able to set an index.html I assume it would require some middleware in the express/server part of the app.

Kind of enjoy inheriting crazy apps and thinking it might just be quicker to rebuild it all 😂

[langri-sha]

We now have access to CHOKIDAR_USEPOLLING.

Run your containers as such:

docker run $opts -e \"CHOKIDAR_USEPOLLING=1\" $image $command

[d3viant0ne]

@Cmdv - Given the rise of Universal in Angular2 and it already being a thing in React , I'll make a note to figure this out & get it into the HowTo.

Thus far professionally, the answer to issues with getting the dev server & docker to play well together has always been nginx.

What's the rest of your stack composition look like in your docker-compose.yml for dev?

[Cmdv]

@d3viant0ne Ah cool, personally I've kept away from the whole universal apps as I still think it's still too adhock but that's just me! 😋

But notes in the how to would be amazing been going round in circles for a good week now!!
JavaScript, it's always the setup that takes the longest haha

proxy:
  image: jwilder/nginx-proxy
  environment:
    - SERVICE_IGNORE=true
  ports:
    - "8080:80"
    - "44300:443"
  volumes:
    - "/var/run/docker.sock:/tmp/docker.sock:ro"

creativepreview:
  image: docker2.******/creativepreview/app:latest
  links:
    - capiweb:capi.local
    - redis:redis.local
    - mongo:mongo.local
    - filesizeweb:filesize.local
  expose:
    - "8005"
  ports:
    - "8005:8005"
  environment:
    - ENVIRONMENT=dockerdev
    - VIRTUAL_HOST=creativepreview.local
    - VIRTUAL_PORT=8005
    - AWS_ACCESS_KEY_ID
    - AWS_REGION=eu-west-1
    - AWS_SECRET_ACCESS_KEY
  volumes:
    - /var/run/docker.sock:/tmp/docker.sock
    - ../../../:/srv/****
  command: bash -c 'grunt dev'

Then it runs the other containers.

[d3viant0ne]

Tagging the above as a reminder: webpack/webpack.io#44

[SpaceK33z]

Closing this because it is a solved problem.

TL;DR - Use --watch-poll or watchOptions: { poll: true } if watching doesn't work correctly. In the upcoming docs overhaul, there will be a section dedicated to Docker.

[max-mykhailenko]

With new docker all works fine, My problem was in using volumes_from instead of volumes in docker-compose.yaml for node container

[enjikaka]

I'm still having issues with webpack-dev-server in docker with docker-compose and nginx-proxy.

version: '2'

services:
  nginx-proxy:
    image: jwilder/nginx-proxy
    container_name: nginx-proxy
    ports:
      - "80:80"
    volumes:
      - /var/run/docker.sock:/tmp/docker.sock:ro
  web-dev:
    build: .
    working_dir: /app
    volumes:
      - "./:/app"
      - /app/node_modules
    expose:
      - 9090
    environment:
      - NODE_ENV=development
      - VIRTUAL_HOST=app.dev
    command: npm run web:dev
    depends_on:
      - nginx-proxy

npm run web:dev === webpack-dev-server --config webpack/webpack.config.web.dev.js --hot --inline --watch-poll

All I get is this from nginx-proxy when trying to visit app.dev in the browser;

2016/09/14 11:04:04 [error] 33#33: 
*9 connect() failed (111: Connection refused) while connecting to upstream, 
client: 172.18.0.1, 
server: app.dev, request: "GET / HTTP/1.1", 
upstream: "http://172.18.0.4:9090/", 
host: "app.dev"

webpack-dev-server doesn't seem to allow connection or something? An express server instead of the webpack-dev-server works fine.

[SpaceK33z]

@enjikaka: you need to add --host 0.0.0.0 to your webpack-dev-server command. By default it only allows connections from localhost.

[enjikaka]

@SpaceK33z Thanks, that made it work!

[nickdima]

Quick question for people using webpack in docker for development. Isn't it slow for non trivial apps with lots of files? After all it's running inside a VM.

[langri-sha]

Yes, compared to running in your host environment, the typical setup will be slower, especially if you're using a hosted hypervisor. It's mostly impacting me on the first run and file transfers, but with proper caching these issues usually dissolve with the ease of running everything at once (development server, test runners, etc).

[enjikaka]

@nickdima We use webpack inside Docker for deving at TIDAL. (Pretty huge app) I've only tried in on my Windows machine so fat though. Works great there. :)

[bigmassa]

Not sure if this is related but I have been having a similar issue with running webpack and docker.
Scenario is on osx running webpack --watch from host to build my angular project which is linked in via a volume regularly crashes the container when it detects a file change. Similarly on ubuntu the container does not crash but the webpack --watch fails to recognise there is a change (which led me here).

I have found that increasing the number of file watchers in both os's have totally fixed my issues. For ubuntu by increasing the amount of inotify watchers https://github.com/guard/listen/wiki/Increasing-the-amount-of-inotify-watchers and for osx ive been running sudo webpack --watch.

Again this may not help but may help but thought it was a good idea to post anyway.

[evfreethy]

Anybody using RubyMine or a JetBrains IDE - please see @bartvanremortele's comment to fix this issue

[jackbraj]

I also have the same problem using laravel-mix which runs webpack.
I am using it inside docker container and my editor is phpstorm on Mac Sierra.

[Angelinsky7]

For those interested (on Window), i'm currently working on a solution inspired by another....
You can try it at https://github.com/Angelinsky7/Docker-Volume-Watcher/releases and let me know.
it uses a filewrapper and sends notification to the docker volume container

[patroza]

@Angelinsky7 you're a genius! thanks a lot!

[Angelinsky7]

@patroza :-)

[cristianfraser]

@SpaceK33z Was a Docker section added with the docs overhaul that I'm missing or not?

[magicsgxie]

In the remote server(centos7), I use command (npm run dev) to start webpack-dev-server, when I close the session, the "webpack-dev-server" is closing. How can i solve this probleam?
the dev's config like this
"dev": "webpack-dev-server --inline --progress --config build/webpack.dev.conf.js",