Pkg: Recursive spawn of pkg binary fails

[EricMCornelius]

Minimal example:

$ cat index.js

const spawn = require('child_process').spawn;

if (process.argv[2] === 'run') {
  const proc = spawn('./testbin', ['test'], {stdio: 'inherit'});
}
else {
  console.log(process.argv);
}

$ pkg -o testbin -t latest-linux-x64 index.js

$ ./testbin run

module.js:542
    throw err;
    ^

Error: Cannot find module 'test'
    at Function.Module._resolveFilename (module.js:540:15)
    at Function.Module._resolveFilename (pkg/prelude/bootstrap.js:1269:46)
    at Function.Module._load (module.js:470:25)
    at Function.Module.runMain (pkg/prelude/bootstrap.js:1298:12)
    at startup (bootstrap_node.js:227:16)
    at bootstrap_node.js:649:3

Seems that this check in bootstrap.js is the culprit:

if (process.env.PKG_EXECPATH === EXECPATH) {
  process.argv.splice(1, 1);
} else {
  process.argv[1] = DEFAULT_ENTRYPOINT;
}

PKG_EXECPATH is being set as an environment variable transparently later on in the bootstrap shims:

function setOptsEnv (args) {
    var pos = args.length - 1;
    if (typeof args[pos] === 'function') pos -= 1;
    if (typeof args[pos] !== 'object' || Array.isArray(args[pos])) {
      pos += 1;
      args.splice(pos, 0, {});
    }
    var opts = args[pos];
    if (!opts.env) opts.env = require('util')._extend({}, process.env);
    if (opts.env.PKG_EXECPATH === 'PKG_INVOKE_NODEJS') return;
    opts.env.PKG_EXECPATH = EXECPATH;
  }

I'm not sure what the intention of deleting the entrypoint argument in the event of self-execution is - I'm sure there's a point that I'm missing.

Note that manually re-inserting the entrypoint as a command line argument works:

./testbin /assets/index.js run

However, the behavior seems odd. I'm working on a command line tool which orchestrates/coordinates commands (think ansible) - and self-invocation of the tool unexpectedly fails due to the above.

[jjhesk]

maybe this guy can help

[EricMCornelius]

@igorklopov - is there a reason for the process.argv.splice(1, 1); removal of a commandline argument when self-executing?

[tavisrudd]

@igorklopov I'm also running into this bug when trying to upgrade from 4.2.6 on https://github.com/unbounce/iidy which invokes itself. Would you accept a PR that allows us to stop https://github.com/zeit/pkg/blob/20e6d00c5d679e2d29e49f6dfc5cea258cf7d71e/prelude/bootstrap.js#L1336 from passing PKG_EXECPATH through to child procs? Commenting that line out solves the issue for us.

[tavisrudd]

FYI, I'm only noticing this bug on Linux. The same iidy tests that expose it run fine on macos.

[chanphillip]

I got the same issue. @tavisrudd 's solution works for my case too!

[divyenduz]

I am running into the same issue. Here is a minimal reproduction: https://github.com/divyenduz/pkg-npm-run

Any known workarounds? :)

[matthieukern]

We encounter the issue on our side as well. Our workaround was (on Windows, but I think the same would work on other OS) to not directly run the process, but to wrap it with a cmd.exe that would overwrite the PKG_EXECPATH environment variable:

exec(`cmd /V /C "set PKG_EXECPATH= && my_program.exe"`)

[Akkadius]

This took care of my issue as well

I spent a few hours on this one, was completely bizarre

    if (process.platform === 'linux') {
      startProcessString = util.format(
        'PKG_EXECPATH=; ./bin/%s server-launcher %s &',
        path.basename(process.argv[0]),
        argString
      );
    }

My actual error

stderr: internal/modules/cjs/loader.js:582
    throw err;
    ^

Error: Cannot find module '/home/eqemu/server/server-launcher'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:580:15)
    at Function.Module._resolveFilename (pkg/prelude/bootstrap.js:1287:46)
    at Function.Module._load (internal/modules/cjs/loader.js:506:25)
    at Function.Module.runMain (pkg/prelude/bootstrap.js:1316:12)
    at startup (internal/bootstrap/node.js:320:19)
    at bootstrapNodeJSCore (internal/bootstrap/node.js:659:3)

Nothing in my code does a require for server-launcher so I was sitting here trying to find some magical require statement where I was passing in something funky. This was incredibly elusive

pkg 4.4
node 10.16.0
os Ubuntu 18.04 LTS

[edwardfoyle]

I was able to combine the suggestions of @matthieukern and @Akkadius into this fairly simple workaround using execa (I'm sure it can be done just using the native child_process as well):

const winCallSelf = async args => {
  const { stdout, stderr } = await execa.command(`cmd /V /C "set PKG_EXECPATH= && ${process.argv[0]} ${args.join(' ')}"`, {
    stdio: 'inherit',
    shell: 'cmd.exe',
  });
  if (stderr) {
    throw new Error(stderr);
  }
  return stdout;
};

const nixCallSelf = async args => {
  const { stdout, stderr } = await execa.command(`PKG_EXECPATH=; ${process.argv[0]} ${args.join(' ')}`, {
    stdio: 'inherit',
    shell: 'bash',
  });
  if (stderr) {
    throw new Error(stderr);
  }
  return stdout;
};

module.exports = {
  callSelf: process.platform.startsWith('win') ? winCallSelf : nixCallSelf
}