When executing a process which spawns a child process, which in turn creates another child process, the empty environment variables are passed through as 'undefined' (the string literal, NOT an absent value). If you than for example do an npm install
command from that child process, bad things happen:
npm WARN invalid config access=undefined
npm WARN invalid config also=undefined
npm WARN invalid config https-proxy=undefined
npm WARN invalid config Must be a full url with \'http://\'
npm WARN onload-script at Function.Module._resolveFilename (module.js:485:15)
npm WARN onload-script { Error: Cannot find module \'undefined\'
To my knowledge: this only happens in ConEmu, a popular console emulator on Windows. When i downgrade to node 7.9, this problem does not occur.
I created a small github repo to reproduce the problem.
git clone [email protected]:nicojs/reproduce-conemu-child-process-environment-bug.git
npm install
.npm test
This test spawns a child process, which in turn starts a new child process which logs the process.env
to console. The tests verifies that npm_config_onload_script
(one of the env variables) is set to ''
.
When ran from within ConEmu: the test failes
1) the env should contain "npm_config_onload_script":
Error: "npm_config_onload_script": "undefined"!"
The test also prints the environment variables to screen. You can see a lot of "undefined" values (the string literal, not an absent value).
For example: "npm_config_onload_script": "undefined",
If ran from cmd or git-bash using mintty directly (or on a POSIX environment): the test passes
√ should contain "npm_config_onload_script"
If you now look at the environment variables on screen, you don't see the "undefined" values. Just empty strings.
For example: "npm_config_onload_script": "",
I reported this issue to ConEmu first, but the developer pointed out to me that this seems to be regression in node. See the discussion with Maximus5 here: https://github.com/Maximus5/ConEmu/issues/1209
Thanks for the bug report. Does it reproduce outside of npm? The behavior of process.env hasn't changed between v7.x and v8.x so I expect it's caused by something in npm.
I've thought of that as well, but I'm using npm 5.3.0 in all cases so that doesn't seem to be the issue.
Maybe something changed to 'child_process'
? I'm using require('child_process').exec
and require('child_process').execSync
.
Can you reproduce without npm being involved? If so, can you post that test case? (It's fine to do that inline here in a comment.)
Hmm good point.
Yes, I can. In one simple script using node -e
.
// start.js
const execSync = require('child_process').execSync;
process.env.foobar = '';
const stdout = execSync('node -e "const execSync = require(\'child_process\').execSync;' +
'execSync(\'node -e "console.log(process.env.foobar);"\', { stdio: [0, 1, 2]});"')
.toString().trim();
if (stdout === '') {
console.log('foobar is empty, stuff works.');
} else {
console.error(`foobar was "${stdout}", using ConEmu?`);
}
If i run that script from a ConEmu tap using cmd shell it says:
> node start.js
foobar was "undefined", using ConEmu?
From command prompt and cmd.exe:
>node start.js
foobar is empty, stuff works.
Well, that does point to ConEmu. FWIW, it works in the expected way on Linux and OS X too.
Well, that does point to ConEmu. FWIW, it works in the expected way on Linux and OS X too.
ConEmu maintainer Maximus5 says it does not change any environment variables. Also, it is translated to the actual string value 'undefined'
. This seems to be something that happens in the JavaScript world.
If you change my script to print the entire environment like this:
const execSync = require('child_process').execSync;
process.env.foobar = '';
const stdout = execSync('node -e "require(\'child_process\').execSync(\'node -e "console.log(process.env);"\', { stdio: [0, 1, 2]});"')
.toString().trim();
console.log(stdout);
If i run that script from a ConEmu tap it says:
> node start.js
...
foobar: 'undefined',
...
From cmd.exe:
>node start.js
...
foobar: undefined,
...
Can't reproduce, I'm afraid; it prints foobar: '',
for me.
cc @nodejs/platform-windows in case any of you can reproduce.
Just double checking: you have Node version 8.0.0 or 8.2.1 and running from inside ConEmu with cmd shell?
Did you let it print the entire process.env
(my second inline sample)? In that case it should not print foobar: ''
, instead it should print foobar: undefined
(but from ConEmu it prints foobar: 'undefined'
). I've just tested it on an other windows 10 machine with ConEmu and was able to reproduce it using node version 8.1.2.
Unable to reproduce with ConEmu 161206 (stable build) and node 8.2.1 on Windows 10. Both npm install
in your repository and your snippet work just fine. Which ConEmu version are you using? How is your shell inside ConEmu configured? Please post the full shell command line along with any additional options.
170402 (preview), but just now also installed 161206 and same issue.
These are my ConEmu settings, but also tried on an other laptop with default settings: https://gist.github.com/nicojs/3abd0b4b2a996c1633bd97c9fd33ec27
See this screenshot for a side by side comparison of what i'm doing. On the left you can see the content of the file "start.js" (which is the first code snippet i shared) and executed from within ConEmu. On the right site you can see the same file executed from command prompt. Also on screen is the about window of ConEmu (with the version).
EDIT:
These are my shell settings for cmd (which are default). But i can reproduce with all shells. Tested with gitbash, powershell and cmd (64bits and 32bits)
@nicojs, could you provide output of env
from both ConEmu and cmd.exe?
And node -e console.log(process.env)
?
I feel slightly naked for doing this. Everything for the cause. Please don't make fun of the size of my P...ATH
https://gist.github.com/nicojs/4de5af23fa937deb8372ced562ef0878
I did the same thing on both console applications and output can be found in the gist:
> type start.js
> node start.js
> SET
> node -e "console.log(process.env)"
I can repro:
node - v8.1.3
windows 10x64 build 15063
ConEmu 170517x64
ConEmu shell - cmd.exe /k "%ConEmuBaseDir%\CmdInitRef.cmd" -new_console:a
CmdInitRef.cmd
@rem !!! Do not change this file in-place, change its copy instead !!!
@rem !!! Otherwise you will lose your settings after next update !!!
@echo off
rem Simple "ver" prints empty line before Windows version
rem Use this construction to print just a version info
cmd /d /c ver | "%windir%\system32\find.exe" "Windows"
rem Now we form the command prompt
rem This will start prompt with `User@PC `
rem set ConEmuPrompt0=$E[m$E[32m$E]9;8;"USERNAME"$E\@$E]9;8;"COMPUTERNAME"$E\$S
set ConEmuPrompt0=
rem Followed by colored `Path`
set ConEmuPrompt1=%ConEmuPrompt0%$P
if NOT "%PROCESSOR_ARCHITECTURE%" == "AMD64" (
if "%PROCESSOR_ARCHITEW6432%" == "AMD64" if "%PROCESSOR_ARCHITECTURE%" == "x86" (
rem Use another text color if cmd was run from SysWow64
set ConEmuPrompt1=%ConEmuPrompt0%$P
)
)
rem Carriage return and `$` or `>`
rem Spare `$E[90m` was specially added because of GitShowBranch.cmd
if "%ConEmuIsAdmin%" == "ADMIN" (
set ConEmuPrompt2=$$
) else (
set ConEmuPrompt2=$G
)
rem Finally reset color and add space
set ConEmuPrompt3=$S
if /I "%~1" == "/git" goto git
if /I "%~1" == "-git" goto git
goto no_git
:git
call "%~dp0GitShowBranch.cmd" /i
goto :EOF
:no_git
rem Set new prompt
PROMPT %ConEmuPrompt1%%ConEmuPrompt2%%ConEmuPrompt3%
feel slightly naked for doing this. Everything for the cause. Please don't make fun of the size of my P...ATH
We're all friends here...
Procmon output:
child (node) 4640
High Resolution Date & Time: 02/08/2017 09:17:07.5786824
Event Class: Process
Operation: Process Start
Result: SUCCESS
Path:
TID: 10428
Duration: 0.0000000
Parent PID: 4640
Command line: node -e "const execSync = require('child_process').execSync;execSync('node -e "console.log(process.env.foobar);"', { stdio: [0, 1, 2]});"
Current directory: d:\code\4node\
Environment:
=D:=d:\code\4node
ALLUSERSPROFILE=C:\ProgramData
ANSICON=182x10000 (182x51)
ANSICON_DEF=7
APPDATA=d:\refael\AppData\Roaming
CommonProgramFiles=C:\Program Files\Common Files
CommonProgramFiles(x86)=C:\Program Files (x86)\Common Files
CommonProgramW6432=C:\Program Files\Common Files
COMPUTERNAME=REFAELUX
ComSpec=C:\WINDOWS\System32\cmd.exe
ConEmuANSI=ON
ConEmuAnsiLog=
ConEmuArgs=/dir "d:\code\node"
ConEmuArgs2=
ConEmuBackHWND=0x00060A5A
ConEmuBaseDir=C:\Tools\maint\conemu\ConEmu
ConEmuBuild=170517
ConEmuConfig=
ConEmuDir=C:\Tools\maint\conemu
ConEmuDrawHWND=0x00060A22
ConEmuDrive=C:
ConEmuHooks=Enabled
ConEmuHWND=0x000606E4
ConEmuIsAdmin=ADMIN
ConEmuPalette=<Solarized>
ConEmuPID=3252
ConEmuPrompt1=$P
ConEmuPrompt2=$$
ConEmuPrompt3=$S
ConEmuServerPID=540
ConEmuTask={Shells::cmd}
ConEmuWorkDir=d:\code\node
ConEmuWorkDrive=d:
foobar=
FSHARPINSTALLDIR=C:\Program Files (x86)\Microsoft SDKs\F#\4.1\Framework\v4.0\
GOPATH=C:\bin\dev\go\work
GOROOT=C:\bin\dev\go
HOME=d:\refael
HOMEDRIVE=d:
HOMEPATH=\refael
JAVA_HOME=C:\WINDOWS\System32\Java
JDK_HOME=C:\WINDOWS\System32\Java
LC_ALL=en_GB.UTF-8
LOCALAPPDATA=d:\refael\AppData\Local
LOGONSERVER=\\REFAELUX
MSMPI_BIN=c:\Program Files\Microsoft MPI\Bin\
NODE_PRESERVE_SYMLINKS=1
NODE_REPL_MODE=strict
NUMBER_OF_PROCESSORS=4
OS=Windows_NT
Path=C:\Tools\maint\conemu\ConEmu\Scripts;C:\Tools\maint\conemu;C:\Tools\maint\conemu\ConEmu;C:\Program Files\Docker\Docker\Resources\bin;c:\Program Files\Microsoft MPI\Bin\;C:\WINDOWS\system32\Tools;C:\WINDOWS\system32\downlevel;C:\WINDOWS\system32\Java\jre\bin;C:\WINDOWS\system32;C:\WINDOWS\system32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\Wbem;C:\WINDOWS;C:\bin\dev\node;C:\bin\dev\python27;C:\bin\dev\python27\Scripts;C:\bin\git\usr\bin;C:\bin\git\cmd;C:\Tools\maint;C:\Tools\dev;C:\code\tools;C:\code\tools\scripts;.\node_modules\.bin\;C:\Program Files\dotnet\;C:\Program Files (x86)\Windows Kits\8.1\Windows Performance Toolkit\
PATHEXT=.JS;.;.CMD;.BAT;.EXE;.MSC;.COM
PROCESSOR_ARCHITECTURE=AMD64
PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 58 Stepping 9, GenuineIntel
PROCESSOR_LEVEL=6
PROCESSOR_REVISION=3a09
ProgramData=C:\ProgramData
ProgramFiles=C:\Program Files
ProgramFiles(x86)=C:\Program Files (x86)
ProgramW6432=C:\Program Files
PROMPT=@CLINK_PROMPT$P$$$S
PSModulePath=C:\Program Files (x86)\WindowsPowerShell\Modules;C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules;C:\bin\dev\gcloud\google-cloud-sdk\platform\PowerShell;C:\Program Files (x86)\AWS Tools\PowerShell\
PUBLIC=C:\Users\Public
SEE_MASK_NOZONECHECKS=1
SystemDrive=C:
SystemRoot=C:\WINDOWS
TEMP=c:\temp\usr
TMP=c:\temp\usr
USERDOMAIN=REFAELUX
USERDOMAIN_ROAMINGPROFILE=REFAELUX
USERNAME=refael
USERPROFILE=d:\refael
windir=C:\WINDOWS
_NT_SYMBOL_PATH=srv*c:\bin\symbols*http://msdl.microsoft.com/download/symbols;srv*c:\bin\symbols*http://chromium-browser-symsrv.commondatastorage.googleapis.com
grandchild (cmd) 12272
High Resolution Date & Time: 02/08/2017 09:17:07.9592797
Event Class: Process
Operation: Process Start
Result: SUCCESS
Path:
TID: 13168
Duration: 0.0000000
Parent PID: 12272
Command line: C:\WINDOWS\System32\cmd.exe /d /s /c "node -e console.log(process.env.foobar);"
Current directory: d:\code\4node\
Environment:
ALLUSERSPROFILE=C:\ProgramData
ANSICON=182x10000 (182x51)
ANSICON_DEF=7
APPDATA=d:\refael\AppData\Roaming
CommonProgramFiles=C:\Program Files\Common Files
CommonProgramFiles(x86)=C:\Program Files (x86)\Common Files
CommonProgramW6432=C:\Program Files\Common Files
COMPUTERNAME=REFAELUX
ComSpec=C:\WINDOWS\System32\cmd.exe
ConEmuANSI=ON
ConEmuAnsiLog=undefined
ConEmuArgs=/dir "d:\code\node"
ConEmuArgs2=undefined
ConEmuBackHWND=0x00060A5A
ConEmuBaseDir=C:\Tools\maint\conemu\ConEmu
ConEmuBuild=170517
ConEmuConfig=undefined
ConEmuDir=C:\Tools\maint\conemu
ConEmuDrawHWND=0x00060A22
ConEmuDrive=C:
ConEmuHooks=Enabled
ConEmuHWND=0x000606E4
ConEmuIsAdmin=ADMIN
ConEmuPalette=<Solarized>
ConEmuPID=3252
ConEmuPrompt1=$P
ConEmuPrompt2=$$
ConEmuPrompt3=$S
ConEmuServerPID=540
ConEmuTask={Shells::cmd}
ConEmuWorkDir=d:\code\node
ConEmuWorkDrive=d:
foobar=undefined
FSHARPINSTALLDIR=C:\Program Files (x86)\Microsoft SDKs\F#\4.1\Framework\v4.0\
GOPATH=C:\bin\dev\go\work
GOROOT=C:\bin\dev\go
HOME=d:\refael
HOMEDRIVE=d:
HOMEPATH=\refael
JAVA_HOME=C:\WINDOWS\System32\Java
JDK_HOME=C:\WINDOWS\System32\Java
LC_ALL=en_GB.UTF-8
LOCALAPPDATA=d:\refael\AppData\Local
LOGONSERVER=\\REFAELUX
MSMPI_BIN=c:\Program Files\Microsoft MPI\Bin\
NODE_PRESERVE_SYMLINKS=1
NODE_REPL_MODE=strict
NUMBER_OF_PROCESSORS=4
OS=Windows_NT
Path=C:\Tools\maint\conemu\ConEmu\Scripts;C:\Tools\maint\conemu;C:\Tools\maint\conemu\ConEmu;C:\Program Files\Docker\Docker\Resources\bin;c:\Program Files\Microsoft MPI\Bin\;C:\WINDOWS\system32\Tools;C:\WINDOWS\system32\downlevel;C:\WINDOWS\system32\Java\jre\bin;C:\WINDOWS\system32;C:\WINDOWS\system32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\Wbem;C:\WINDOWS;C:\bin\dev\node;C:\bin\dev\python27;C:\bin\dev\python27\Scripts;C:\bin\git\usr\bin;C:\bin\git\cmd;C:\Tools\maint;C:\Tools\dev;C:\code\tools;C:\code\tools\scripts;.\node_modules\.bin\;C:\Program Files\dotnet\;C:\Program Files (x86)\Windows Kits\8.1\Windows Performance Toolkit\
PATHEXT=.JS;.;.CMD;.BAT;.EXE;.MSC;.COM
PROCESSOR_ARCHITECTURE=AMD64
PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 58 Stepping 9, GenuineIntel
PROCESSOR_LEVEL=6
PROCESSOR_REVISION=3a09
ProgramData=C:\ProgramData
ProgramFiles=C:\Program Files
ProgramFiles(x86)=C:\Program Files (x86)
ProgramW6432=C:\Program Files
PROMPT=@CLINK_PROMPT$P$$$S
PSModulePath=C:\Program Files (x86)\WindowsPowerShell\Modules;C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules;C:\bin\dev\gcloud\google-cloud-sdk\platform\PowerShell;C:\Program Files (x86)\AWS Tools\PowerShell\
PUBLIC=C:\Users\Public
SEE_MASK_NOZONECHECKS=1
SystemDrive=C:
SystemRoot=C:\WINDOWS
TEMP=c:\temp\usr
TMP=c:\temp\usr
USERDOMAIN=REFAELUX
USERDOMAIN_ROAMINGPROFILE=REFAELUX
USERNAME=refael
USERPROFILE=d:\refael
windir=C:\WINDOWS
_NT_SYMBOL_PATH=srv*c:\bin\symbols*http://msdl.microsoft.com/download/symbols;srv*c:\bin\symbols*http://chromium-browser-symsrv.commondatastorage.googleapis.com
@refack if it's not too much to ask, can you also confirm that you cannot reproduce it in Node v7.9? Just to make sure that it is indeed related to node 8 and up.
@nicojs was just about to... but bad news:
d:\code\4node$ node790 -v
v7.9.0
d:\code\4node$ node790 conemu-bug.js
foobar was "undefined", using ConEmu?
d:\code\4node$ node6 conemu-bug.js
foobar was "undefined", using ConEmu?
d:\code\4node$ node6 -v
v6.11.1
d:\code\4node$ node4 conemu-bug.js
foobar was "undefined", using ConEmu?
d:\code\4node$ node4 -v
v4.8.2
Seems like it's a ConEmu thing, but I'm dubugging
Okay, I found the problem, this is caused by ConEmuHk.dll which is injected into all processes by default. (I had this feature disabled in my ConEmu setup so I was unable to reproduce the issue.)
By default, ConEmu injects its own DLL into all processes, including node:
Wow! Disabled it and stuff works again like a charm!
Anyone running into this issue: setting can be found here:
So i guess an issue at ConEmu side after all (or in the injected dll). Tricky stuff. Thanks to all for your time! It took me several days to understand what was going wrong, but solving the issue only took a couple of hours. 💐
I'm reopening since there is a node bug here:
__An env variable with an undefined
value gets converted into the string 'undefined'
__
@refack see https://github.com/nodejs/node/issues/4920
This is documented behavior though.
@refack see #4920
Ack, but that's just part of the story, here node is spawned in an environment with an undefined
value...
So how do we proceed? Apparently, node converts null to undefined, which it should not do? And only when the ConEmuHK.dll is injected into child processes an empty value is transformed to null
? Is that the correct conclusion?
Node treats an empty string as an empty string. If it doesn't, that's a bug but I doubt that's the case here, or we would have received a small avalanche of bug reports by now.
I think a more likely explanation is that the parasitic code that ConEmu injects does something to GetEnvironmentVariableW() that breaks node. In fact, it seems like an inescapable conclusion because you mentioned that disabling injection of that DLL fixes the issue.
Thanks @bnoordhuis
@refack @Maximus5 do you agree with this assessment?
I'm focusing to reproducing the bug I see on the node side (handling of an environment variable with a Null
value).
If I can reproduce without ConEmuHK.dll
then I'll open a PR.
P.S. I agree with @bnoordhuis that the "empty string part" of this issue is ConEmu's responsibility.
Try to open non-existent file before environment access. Actually any function (which do SetLastError) called before environment access may confuse node I think.
Environment variable contains empty string, but Node checks for non-zero GetLastError() instead of described ERROR_ENVVAR_NOT_FOUND.
Node.js checks this: result > 0 || GetLastError() == ERROR_SUCCESS
. I'm not a Windows programmer, I've no idea if that is correct, but that logic hasn't changed in five or six years.
Looks like it was changed in node 8.x because version 7.x doesn't suffer from the problem.
I'll dig a bit.
@refack Any news?
First @Maximus5 patched ConEmuHK.dll so it does not leak or swallow WIN32 errors anymore - https://github.com/Maximus5/ConEmu/commit/5324aa03500fdb9d198f152c93f23eed57113858
I had a bit of trouble with https://github.com/nodejs/node/pull/14641, but I think it's something I can manage once I get a free hour. That PR should close this issue.
I am still affected by this bug in some CI setup without ConEmu. For the time being I downgraded nodejs to 6.11.1. Could we put the label confirmed-bug on this issue?
@payload which version were you working with before?
8.4.0
Experiencing this too, was driving me insane preventing me from building sqlite3 with node-gyp.
Win 10 / ConEmu / Node 8.8.0
After reading this thread I just tried it in cmd.exe and it worked first time.
@v9Chris glad this issue could help you. I know the feeling :metal:
That's why software updates are recommended.
Any news whether node can be hardened against this without fixing the emulator? Sometimes these things are not under your control.
Possibly related, without ConEmu:
var exec = require('child_process').exec
process.env.test = ''
if (process.argv[2] === 'trigger') {
try { require('fs').readFileSync('foobar') } catch (e) {}
}
exec('node -p "process.env.test === \'undefined\' ? \'BUG\' : \'OK\'"', function (err, stdout) {
if (err) throw err
console.log(stdout)
})
Output:
> node test.js
OK
> node test.js trigger
BUG
But I can only trigger this on 4.8.7 x86 (also tried 9.2.0, 8.4.0, 8.0.0, 7.9.0, no problems there).
Hello,
Maybe I have found something very similar related to the access of an undefined process.env variable in Windows. The env variable that does not exist in this test is "DUMMY_VAR".
The test code is extremely simple and I cannot give an explanation of the difference results I get with node v8.9.4:
process.env.applicationpath = "";
console.log("typeof process.env.applicationpath = " + typeof(process.env.applicationpath));
if (!process.env.DUMMY_VAR) {
// if (true) {
console.log(" typeof process.env.applicationpath = " + typeof(process.env.applicationpath));
}
console.log("typeof process.env.applicationpath = " + typeof(process.env.applicationpath));
Output:
typeof process.env.applicationpath = string
**typeof process.env.applicationpath = undefined**
typeof process.env.applicationpath = string
Which I think is wrong.
If I uncomment the "if true" line and I comment the if that access the undefined environment var the it works fine. The output now is:
typeof process.env.applicationpath = string
**typeof process.env.applicationpath = string**
typeof process.env.applicationpath = string
I have tested the same code with node v5.5 and it gives the right expected result. In some way it seems that accessing the undefined environment variable is affecting the results in the other?
In some way it seems that accessing the undefined environment variable is affecting the results in the other?
I suspect this is also why fs.readFileSync
triggers the bug in my example. It's not the reading of an (unexisting) file that triggers it per se but that readFileSync
does something with/to the environment.
BTW @payload I was unable to trigger it on v6.
In some way it seems that accessing the undefined environment variable is affecting the results in the other?
@addaleax recently fixed https://github.com/nodejs/node/pull/18463 which sounds like the example given by @ajflash.
Yes, it’s exactly that bug.
/cc @nodejs/lts @gibfahn Just so you are aware, this seems to be a bug that quite a few people are running into and a (pretty much zero-risk) fix has been landed on master two weeks ago. It might be good to have this in 8.10.0, even if it doesn’t technically fulfill the lived-in-Current-for-2-weeks rule.
Sounds like a good candidate for expediting, will comment in https://github.com/nodejs/node/pull/18463.
Can this be closed now?
@bzoz I think it might be easier to wait until this has been fixed in the v8.x branch, at least so we don’t forget about it
I've landed the fix in v8.x-staging, so I'll close.
Most helpful comment
Wow! Disabled it and stuff works again like a charm!
Anyone running into this issue: setting can be found here:
So i guess an issue at ConEmu side after all (or in the injected dll). Tricky stuff. Thanks to all for your time! It took me several days to understand what was going wrong, but solving the issue only took a couple of hours. 💐