Better-sqlite3: Installation issues on Windows

[Rundik]

> [email protected] install C:\home\coding(projects)\mini_projects\books_db\node_modules\integer
> node-gyp rebuild


C:\home\coding(projects)\mini_projects\books_db\node_modules\integer>if not defined npm_config_node_gyp (node "C:\Program Files\nodejs\node_modules\npm\node_modules\npm-lifecycle\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild )  else (node "C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js" rebuild )
Traceback (most recent call last):
  File "C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\gyp\gyp_main.py", line 16, in <module>
    sys.exit(gyp.script_main())
  File "C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\gyp\pylib\gyp\__init__.py", line 545, in script_main
    return main(sys.argv[1:])
  File "C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\gyp\pylib\gyp\__init__.py", line 538, in main
    return gyp_main(args)
  File "C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\gyp\pylib\gyp\__init__.py", line 514, in gyp_main
    options.duplicate_basename_check)
  File "C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\gyp\pylib\gyp\__init__.py", line 98, in Load
    generator.CalculateVariables(default_variables, params)
  File "C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\gyp\pylib\gyp\generator\msvs.py", line 1916, in CalculateVariables
    generator_flags.get('msvs_version', 'auto'))
  File "C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\gyp\pylib\gyp\MSVSVersion.py", line 434, in SelectVisualStudioVersion
    versions = _DetectVisualStudioVersions(version_map[version], 'e' in version)
KeyError: '2017'
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\lib\configure.js:345:16)
gyp ERR! stack     at ChildProcess.emit (events.js:182:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:240:12)
gyp ERR! System Windows_NT 10.0.14393
gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild"
gyp ERR! cwd C:\home\coding(projects)\mini_projects\books_db\node_modules\integer
gyp ERR! node -v v10.10.0
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok
npm WARN [email protected] No description
npm WARN [email protected] No repository field.

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: `node-gyp rebuild`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\Rundik\AppData\Roaming\npm-cache\_logs\2018-09-21T10_07_07_803Z-debug.log

[JoshuaWise]

duplicate of https://github.com/JoshuaWise/better-sqlite3/issues/164

[eslachance]

I'd like to point out that the integer build is not the better-sqlite3 build, and in a LOT of cases I've seen with my own users (I made a module called enmap which relies on better-sqlite3 for persistence), it's at the integer build that they fail - they have no issue installing sqlite, sqlite3, or other node-gyp module. But integer just fails miserably.

Is there any specific, feature-breaking reason why integer is needed with better-sqlite3 but not with other sqlite modules?

[eslachance]

for example:
https://media.discordapp.net/attachments/364868291149168640/493724394758078467/unknown.png
https://media.discordapp.net/attachments/473491355520204800/493706304922058752/unknown.png

C:\Users\Administrator\Documents\Moonglow\node_modules\integer>if not defined npm_config_node_gyp (node "C:\Program Files\nodejs\node_modules\npm\node_modules\npm-lifecycle\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild )  else (node "C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js" rebuild )
Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch.
C:\Users\Administrator\Documents\Moonglow\node_modules\integer\build\integer.vcxproj(20,3): error 

MSB4019: The imported project "C:\Microsoft.Cpp.Default.props" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.
gyp ERR! build error
gyp ERR! stack Error: `C:\Program Files (x86)\MSBuild\14.0\bin\msbuild.exe` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onExit (C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\lib\build.js:262:23)
gyp ERR! stack     at emitTwo (events.js:126:13)
gyp ERR! stack     at ChildProcess.emit (events.js:214:7)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)
gyp ERR! System Windows_NT 6.3.9600
gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild"
gyp ERR! cwd C:\Users\Administrator\Documents\Moonglow\node_modules\integer
gyp ERR! node -v v8.12.0
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok

[JoshuaWise]

@eslachance The integer module enables 64-bit-integer support which is something that is lacking from all other sqlite libraries for Node.js. The integer library is much simpler than better-sqlite3, and its build process is also simpler. My guess is that if integer fails to build, better-sqlite3 would have failed to build right afterwards.

[eslachance]

@JoshuaWise I think my major pain point is that I've never had any issue building it myself, and have never been unable to reproduce the issue on my own, in several PCs and VMs with various versions of Windows. So it's not an easy thing to troubleshoot, and must have a very specific cause that I'd love to pinpoint.

[JoshuaWise]

@eslachance Yeah it's these hard-to-replicate issues that are the worst. I appreciate your interest in helping solve some of these issues.

[JoshuaWise]

Many installation-related bugs were fixed in version 5.0.0. It could be worth trying the new version.

[eslachance]

I'll definitely take a look! I'm using better-sqlite-pool so I'll have to fork it for testing, but I'm happy to hear this might be progressing! I'll let you know. Thanks :D

[eslachance]

Unfortunately, there's still an issue with the integer package:
https://i.typicalbot.com/57b817

[JoshuaWise]

@eslachance I found this thread which looks relevant:
https://github.com/npm/npm/issues/10421#issuecomment-378489898

[eslachance]

I'd say that's awesome but that's already what I'm telling people to do. At this point, it sounds like I should start releasing a batch file that does this because the command's starting to look like
npm i -g --prodution --vs2015 --add-python-to-path windows-build-tools node-gyp

Quite a mouthful.

[eslachance]

As a sidenote, --vs2015 is to counter the following error:

C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\Common7\IDE\VC\VCTargets\Microsoft.Cpp.Platform.targets(67,5): error MSB8020: The build tools for v140 (Platform Toolset = 'v140') cannot be found. To build using the v140 build tools, please install v140 build tools.  Alternatively, you may upgrade to the current Visual Studio
tools by selecting the Project menu or right-click the solution, and then selecting "Retarget solution". [C:\Users\Jackson\Desktop\Rino\node_modules\better-sqlite3\build\deps\action_before_build.vcxproj]

So even though https://github.com/npm/npm/issues/10421#issuecomment-378489898 indicates that the 2015 version is broken, it's the only way to get around that issue as far as I know.

[vtyr]

I'm still getting the following issue when trying to run npm i better-sqlite-pool
npm ERR! [email protected] install: node-gyp rebuild npm ERR! Exit status 1 npm ERR! npm ERR! Failed at the [email protected] install script 'node-gyp rebuild'.

I've run the following commands:
install -g node-gyp npm config set unsafe-perm=true npm install --global --production windows-build-tools --vs2015

Any advice? I'm on v4.3.0

[mceachen]

With better-sqlite version 5.0.0, and a project that has a number of other native libraries, I had to update my windows build box setup to the following:

1. Install the latest node 8 or node 10. Earlier versions of node (like 10.2.x) will not compile.
2. Right-click start, Windows PowerShell (Admin)
3. Install both vs2015 and vs2017 libraries. Each line will take ~5-10 minutes.

npm install --global --production --vs2015 --add-python-to-path windows-build-tools
npm install --global --production --add-python-to-path windows-build-tools node-gyp

1. In your project, make sure you're not fighting with old build configurations. Delete both your %USERPROFILE%/.node-gyp and your project's node_modules directories.
2. Set up %USERPROFILE%/.npmrc correctly:

msvs_version=2015
python=python2.7

(where %USERPROFILE% is your home directory).

[enbermudas]

That last solution won't work for me:

c:\Sandbox\node\pipobot\node_modules\integer>if not defined npm_config_node_gyp (node "C:\Program Files\nodejs\node_modules\npm\node_modules\npm-lifecycle\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild )  else (node "C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js" rebuild )
Los proyectos de esta solución se van a compilar de uno en uno. Para habilitar la compilación en paralelo, agregue el modificador "/m".
c:\Sandbox\node\pipobot\node_modules\integer\build\integer.vcxproj(20,3): error MSB4019: No se encuentra el proyecto importado "c:\Microsoft.Cpp.Default.props". Asegúr ese de que la ruta de acceso de la declaración <Import> es correcta y de que el archivo se encuentra en el disco.
gyp ERR! build error
gyp ERR! stack Error: `C:\Program Files (x86)\MSBuild\14.0\bin\msbuild.exe` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onExit (C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\lib\build.js:262:23)
gyp ERR! stack     at ChildProcess.emit (events.js:182:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:254:12)
gyp ERR! System Windows_NT 6.1.7601
gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild"
gyp ERR! cwd c:\Sandbox\node\pipobot\node_modules\integer
gyp ERR! node -v v11.2.0
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok
npm WARN [email protected] requires a peer of bufferutil@^3.0.3 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of erlpack@discordapp/erlpack but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of node-opus@^0.2.7 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of opusscript@^0.0.6 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of sodium@^2.0.3 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of libsodium-wrappers@^0.7.3 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of uws@^9.14.0 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of better-sqlite-pool@github:eslachance/better-sqlite-pool but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] No repository field.

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: `node-gyp rebuild`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\Enrique\AppData\Roaming\npm-cache\_logs\2018-12-05T15_23_02_776Z-debug.log

[mceachen]

@enbermudas did you skip step 3?

https://duckduckgo.com/?q=error+MSB4019 (you need to delete your node_modules directory, and it looks like there's a project file you can delete in your C:\ root directory).

[enbermudas]

@mceachen I did it with and without that node_modules folder. The error is still there.

[mceachen]

@mceachen I did it with and without that node_modules folder. The error is still there.

I'd look at the results of that DDG query. There are a bunch of solutions there.

[jshrek]

I was trying to install better-sqlite3 but kept getting errors about not finding VCBuild and MSBuild.

I first tried to install install node-gyp by itself but still had issues: npm install -g node-gyp

I then tried to install windows-build-tools which installed Visual Studio 2017 and did NOT include the v140 toolset and was then getting errors about this toolset (error MSB8020).
But Visual Studio 2015 does includes the v140 toolset and you can force windows-build-tools to install VS2015 instead of VS2017 with this:
npm install --vs2015 -g windows-build-tools

After this better-sqlite3 installed without errors.

[Huskydog9988]

I know i haven't posted in here but I still have an issue with the installation. In fact I cannot even install windows build tools. I've reinstalled node, started my pc, I don't know what could be causing this. Was done on admin windows powershell and when installing it gets stuck like this.

PS C:\WINDOWS\system32> npm install --vs2015 -g windows-build-tools

> [email protected] postinstall C:\WINDOWS\system32\'~\.npm-new-global'\node_modules\windows-build-tools
> node ./dist/index.js



Downloading python-2.7.15.amd64.msi
[>                                            ] 0.0% (0 B/s)
Downloaded python-2.7.15.amd64.msi. Saved to C:\Users\<REDACTED USER>\.windows-build-tools\python-2.7.15.amd64.msi.
Downloading BuildTools_Full.exe
[>                                            ] 0.0% (0 B/s)
Downloaded BuildTools_Full.exe. Saved to C:\Users\<REDACTED USER>\.windows-build-tools\BuildTools_Full.exe.

Starting installation...
Launched installers, now waiting for them to finish.
This will likely take some time - please be patient!

Status from the installers:
---------- Visual Studio Build Tools ----------
Still waiting for installer log file...
------------------- Python --------------------
Still waiting for installer log file...

[JoshuaWise]

@Huskydog9988 You should really ask for help within the windows-builds-tools project.

[VegeoStudios]

F:\zellybot-master\zellybot-master>npm i better-sqlite-pool

> [email protected] install F:\zellybot-master\zellybot-master\node_modules\integer
> node-gyp rebuild


F:\zellybot-master\zellybot-master\node_modules\integer>if not defined npm_config_node_gyp (node "C:\Users\highl_000\AppData\Roaming\npm\node_modules\npm\node_modules\npm-lifecycle\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild )  else (node "node C:\Users\me\AppData\Roaming\npm\node_modules\node-gyp\bin\node-gyp.js" rebuild )
internal/modules/cjs/loader.js:583
    throw err;
    ^

Error: Cannot find module 'F:\zellybot-master\zellybot-master\node_modules\integer\node C:\Users\me\AppData\Roaming\npm\node_modules\node-gyp\bin\node-gyp.js'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:581:15)
    at Function.Module._load (internal/modules/cjs/loader.js:507:25)
    at Function.Module.runMain (internal/modules/cjs/loader.js:742:12)
    at startup (internal/bootstrap/node.js:283:19)
    at bootstrapNodeJSCore (internal/bootstrap/node.js:743:3)
npm WARN [email protected] requires a peer of bufferutil@^3.0.3 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of erlpack@discordapp/erlpack but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of node-opus@^0.2.7 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of opusscript@^0.0.6 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of sodium@^2.0.3 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of libsodium-wrappers@^0.7.3 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of uws@^9.14.0 but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of better-sqlite-pool@github:eslachance/better-sqlite-pool but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] No repository field.
npm WARN [email protected] No license field.

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: `node-gyp rebuild`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\highl_000\AppData\Roaming\npm-cache\_logs\2019-02-22T01_42_43_767Z-debug.log

I have the same issues and I have tried all of these suggestions, nothing seems to be fixing it.

[mceachen]

@Huskydog9988 @VegeoStudios Did you do all 5 steps (they may not make sense, but don't skip any of them) in https://github.com/JoshuaWise/better-sqlite3/issues/170#issuecomment-441734343 ?

If you're still having issues after following those steps, try making a new project (npm --init) that only has better-sqlite3 as a dependency. It looks like you've got a number of native deps. They don't always play nicely together.

[VegeoStudios]

I figured out what the instructions meant. Looks like it's all working now!

[Huskydog9988]

@mceachen the build tools installer gets stuck waiting for installer log files. So I cannot even begin to follow your directions past opening Admin powershell. I have removed everything related to windows build tools in the “user” folder and in “path”.

Edit: So I’m beginning to suspect that this is failing due to the version on windows 10 I’m running. I’m running 1511 so I believe that might be why.

[mceachen]

@Huskydog9988 I find myself in a world of hurt in windows land if I'm not running the latest update. I'm on 10.0.17763 Build 17763.

@JoshuaWise I think you can probably close this ticket or mark it sticky?

Should the steps in https://github.com/JoshuaWise/better-sqlite3/issues/170#issuecomment-441734343 be added to the readme or https://github.com/JoshuaWise/better-sqlite3/blob/master/docs/compilation.md?

[Gikkman]

Just wanted to chime in that I had the issues described in this ticket.

I ran npm install --global --production windows-build-tools --vs2015

then ran npm i && electron-builder install-app-deps

and got it working. Using node v11.13.0, better-sqlite3 5.4.0, electron-builder: 20.40.0

[JoshuaWise]

VS2017 and VS2019 are now supported in v5.4.3.

[WilloIzCitron]

Hello i cant install this module on Repl.it