Serverless-webpack: error with scandir node_modules during packaging

[jmparsons]

Here's an example repo:

https://github.com/jmparsons/webpack-test

I'm using serverless 0.19. The latest webpack and serverless webpack.

I'm getting Error: ENOENT: no such file or directory, scandir 'node_modules'

After running serverless deploy in the src/tester folder.

cd src/tester
serverless deploy

Full output:

{ Error: ENOENT: no such file or directory, scandir 'node_modules'
    at Object.fs.readdirSync (fs.js:910:18)
    at readDir (/Users/_/Desktop/webpack-test/node_modules/webpack-node-externals/index.js:12:19)
    at nodeExternals (/Users/_/Desktop/webpack-test/node_modules/webpack-node-externals/index.js:95:65)
    at Object.<anonymous> (/Users/_/Desktop/webpack-test/webpack.config.js:8:15)
    at Module._compile (module.js:569:30)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:503:32)
    at tryModuleLoad (module.js:466:12)
    at Function.Module._load (module.js:458:3)
    at Module.require (module.js:513:17)
    at require (internal/module.js:11:18)
    at ServerlessWebpack.validate (/Users/_/Desktop/webpack-test/node_modules/serverless-webpack/lib/validate.js:94:28)
    at ServerlessWebpack.tryCatcher (/Users/_/Desktop/webpack-test/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/Users/_/Desktop/webpack-test/node_modules/bluebird/js/release/promise.js:512:31)
    at Promise._settlePromise (/Users/_/Desktop/webpack-test/node_modules/bluebird/js/release/promise.js:569:18)
    at Promise._settlePromiseCtx (/Users/_/Desktop/webpack-test/node_modules/bluebird/js/release/promise.js:606:10)
    at Async._drainQueue (/Users/_/Desktop/webpack-test/node_modules/bluebird/js/release/async.js:138:12)
    at Async._drainQueues (/Users/_/Desktop/webpack-test/node_modules/bluebird/js/release/async.js:143:10)
    at Immediate.Async.drainQueues (/Users/_/Desktop/webpack-test/node_modules/bluebird/js/release/async.js:17:14)
    at runCallback (timers.js:781:20)
    at tryOnImmediate (timers.js:743:5)
    at processImmediate [as _immediateCallback] (timers.js:714:5)
  errno: -2,
  code: 'ENOENT',
  syscall: 'scandir',
  path: 'node_modules' }
Serverless: Bundling with Webpack...
Time: 428ms
   Asset     Size  Chunks             Chunk Names
index.js  4.76 kB       0  [emitted]  index
   [0] ./index.js 524 bytes {0} [built]
   [1] ./Tester.js 731 bytes {0} [built]
   [2] /Users/_/Desktop/webpack-test/node_modules/babel-runtime/core-js/json/stringify.js 95 bytes {0} [built]
   [3] /Users/_/Desktop/webpack-test/node_modules/core-js/library/fn/json/stringify.js 242 bytes {0} [built]
   [4] /Users/_/Desktop/webpack-test/node_modules/core-js/library/modules/_core.js 117 bytes {0} [built]
   [5] /Users/_/Desktop/webpack-test/node_modules/babel-runtime/helpers/classCallCheck.js 208 bytes {0} [built]
Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Creating Stack...
Serverless: Checking Stack create progress...
.....
Serverless: Stack create finished...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Uploading service .zip file to S3 (1.6 KB)...
Serverless: Validating template...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
....................................
Serverless: Stack update finished...
Service Information
service: webpack-test-

[HyperBrain]

Hi @jmparsons 😉 ,

can you move the serverless.yml to the root of your service and change it this way:

# serverless.yml
service: webpack-test-tester

frameworkVersion: ">=1.1.0 <2.0.0"

custom:
  webpackIncludeModules: true
  webpack: webpack.config.js

plugins:
  - serverless-webpack

package:
  exclude:
    - config.json

provider:
  name: aws
  runtime: nodejs6.10
  stage: dev
  profile: airdev01
  region: us-east-1
  memorySize: 128

functions:

  post:
    handler: src/tester/index.post
    events:
      - http:
          path: tester/post
          method: post
          cors: true

Specifying node_modules/** should not be necessary as the plugin only bundles node modules that are actually used by your code.

In general, the service definition (yml) should be located at the root and reference the handlers within the subdirs. The exclude also might be configured within your webpack config. But just give it a try first with the above "rooted" serverless.yml.

I think the root cause is, that node-externals does not find an adjacent node_modules folder which is also located in the root after you do a npm install.

[jmparsons]

@HyperBrain Thanks for getting back to me.

I'm coming from a major project that has exceeded the endpoints, so I'm trying to carry my structure over and break it into smaller ones.

https://serverless.com/framework/docs/providers/aws/guide/services#organization

I have no problems with it working in root on any of my serverless projects, but with nested serverless.yml files I'm experiencing this, but I need it to separate services.

[HyperBrain]

Ok I see.

I just saw that node externals has a modulesDir option (see https://www.npmjs.com/package/webpack-node-externals). Instead of moving the serverless.yml, you could try to use nodeExternals({ modulesDir: '../../node_modules' }). Serverless (and the plugin) always uses its service dir (/src/tester in your case) as base reference, so it might be sufficient to tell nodeExternals where to find the modules in your webpack.conf.js.

[jmparsons]

@HyperBrain Good stuff that was the ticket! Thank you very much!

[HyperBrain]

You're welcome ;-)