Elasticsearch-js: Errors when using with `pkg`

[mojzu]

🐛 Bug Report

I upgraded one of our projects to use "@elastic/elasticsearch" version 7.2.0. The library works great in development but fails in our production builds. This appears to be because we use pkg to bundle production builds for deployment. Since the API is lazy loaded, and "require" calls are made within a function, pkg doesn't see them and excludes the files from the build.

To Reproduce

Try to use @elastic/elasticsearch library methods in a pkg binary. Results in errors such as:

'ElasticsearchError "{"code":"ElasticsearchError.Up"}": Error: Cannot find module \'./api/cluster.health.js\'\n1) If you want to compile the package/file into executable, please pay attention to compilation warnings and specify a literal in \'require\' call. 2) If you don\'t want to compile the package/file into executable and want to \'require\' it from filesystem (likely plugin), specify an absolute path in \'require\' call using process.cwd() or process.execPath.'

Expected behavior

I've been able to work around this by replacing lazyLoad with calls to require, but I'm guessing that isn't the desired solution (since our application only runs on a server the startup performance isn't a big concern). Would be possible to maintain the performance benefits by splitting the require and build steps?

Your Environment

• node version: 12
• @elastic/elasticsearch version: 7.2.0
• os: Linux

[delvedor]

Hello! It seems that lazy loading is supported by pkg via https://github.com/zeit/pkg#config.
I tried it in my local machine and it's not working, probably is related to https://github.com/zeit/pkg/issues/404.

Not sure what else I can do here.

[mojzu]

Hi, thanks for the response. Pkg only handles require calls with string literals, the linked config documentation mentions in the second paragraph that this case isn't supported and these files must be manually included.

Except there is no way using pkg to manually include them where a dependency in node_modules expects to find them, hence the problem. We run into this issue with some number of our dependencies and it's almost always related to some kind of runtime lazy loading/file inclusion.

To be fair this isn't an issue with this library, I thought it might count as a bug because the deprecated elasticsearch library did work in pkg. And we've generally found it easier to resolve these issues from the library side as pkg gets updates sporadically.

I've written a manual patch that overwrites the lazy loading with direct require calls which seems to be working for our case, so if you don't see a way to make this work feel free to close.

Thanks for your time,
Sam

[delvedor]

Hello! I was able to make this work, this is how my package.json looks like:

{
  "name": "pkgtest",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "bin": {
    "run": "index.js"
  },
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "Tomas Della Vedova",
  "license": "ISC",
  "dependencies": {
    "@elastic/elasticsearch": "^7.2.0"
  },
  "pkg": {
    "assets": [
      "node_modules/@elastic/elasticsearch/api/**/*"
    ]
  }
}

And this is my index.js file:

'use strict'

const { Client } = require('@elastic/elasticsearch')
const client = new Client({ node: 'http://localhost:9200' })

client.info(console.log)

Then if I run ./pkgtest I get the expected results.

[mojzu]

That fixed it! Thank you for taking the time to help.

It looks like I was being stupid and thought the pkg.assets were static files moved into a subdirectory in the pkg snapshot filesystem, but it appears they can be copied into the root of the filesystem as well.