Node-sqlite3: node v10.x support

[springmeyer]

Ticket to track adding node v10 support (which came out 2 days ago: https://nodejs.org/en/blog/release/v10.0.0/.).

• [x] adding support for building against node v10 (#978)
  
  
  
  • [x] one known issue is https://github.com/mapbox/node-sqlite3/issues/889
  
  
  • [x] other task is adding v10 to the build matrix in the .travis.yml
  
  
• [x] upgrading to a version of node-pre-gyp that supports node v10 (https://github.com/mapbox/node-pre-gyp) (https://github.com/mapbox/node-sqlite3/issues/1005)
• [x] ensuring all tests pass with node v10
• [x] publishing binaries for node v10 (so users that want node v10 support don't need a compiler environment)
• [x] releasing a new version of node-sqlite3

[chbrown]

FWIW, as a naive end-user, [email protected] seems to build and work just fine on Node.js v10 (apart from the ForceSet-related deprecations during node-gyp compilation), whereas 3.1.13 was not building at all (ForceSet-related errors).

[Apollon77]

windows build for nodejs 10 is still not working

[NicoHood]

I can build v4.0.0 with node 10, however my application still fails with:

Error: Cannot find module '/usr/lib/node_modules/strichliste/node_modules/sqlite3/lib/binding/node-v59-linux-x64/node_sqlite3.node'

More error details here: https://github.com/hackerspace-bootstrap/strichliste/issues/278

I dont know why it is still looking for an older version. Is it a bug in the "strichliste" application or sqlite itself? If I symlink the new version I get symbol errors:

Error: /usr/lib/node_modules/strichliste/node_modules/sqlite3/lib/binding/node-v64-linux-x64/node_sqlite3.node: undefined symbol: _ZN2v816FunctionTemplate3NewEPNS_7IsolateEPFvRKNS_20FunctionCallbackInfoINS_5ValueEEEENS_5LocalIS4_EENSA_INS_9SignatureEEEiNS_19ConstructorBehaviorENS_14SideEffectTypeE

[ankuryu]

Whereas I can install [email protected] globally on my win10 x.64 node v10.0.0 if fails when I try to install in another package using --save

[wormen]

the build problem has been fixed with the release 10.1.0

[NicoHood]

Thanks for that note. I could get it working now with the new node update and sqlite 4.0.0!

[phiphou]

Does not work with node 10.1.0 on Windows.

[wormen]

@phiphou everything works fine, yesterday I was building a module to work with electronjs

[cannap]

not working for me on win 10 node 10.1.0

[phiphou]

@wormen Not working with @mapbox/mbtiles , opened an issue there. They currently don't use sqlite3@4x , maybe the issue.

[igor-savin-ht]

@cannap What error are you getting?

[cannap]

@igor-savin-ht

> node-pre-gyp install --fallback-to-build

node-pre-gyp ERR! Tried to download(403): https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v4.0.0/node-v64-win32-x64.tar.gz
node-pre-gyp ERR! Pre-built binaries not found for [email protected] and [email protected] (node-v64 ABI, unknown) (falling back to source compile with node-gyp)
Die Projekte in dieser Projektmappe werden nacheinander erstellt. Um eine parallele Erstellung zu ermöglichen, müssen Sie den Schalter "/m" hinzufügen.
  unpack_sqlite_dep
CUSTOMBUILD : Fatal Python error : Py_Initialize: unable to load the file system codec [C:\Users\marko\Desktop\topsecret\server-alt\node_modules\sqlite3\build\deps\action_before_bu
ild.vcxproj]
    File "C:\Python27\lib\encodings\__init__.py", line 123
      raise CodecRegistryError,\
                              ^
  SyntaxError: invalid syntax

  Current thread 0x00005458 (most recent call first):
gyp ERR! build error
gyp ERR! stack Error: `C:\Program Files (x86)\MSBuild\14.0\bin\msbuild.exe` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onExit (C:\Users\marko\AppData\Roaming\nvm\v10.1.0\node_modules\npm\node_modules\node-gyp\lib\build.js:258:23)
gyp ERR! stack     at ChildProcess.emit (events.js:182:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:235:12)
gyp ERR! System Windows_NT 10.0.16299
gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\marko\\AppData\\Roaming\\nvm\\v10.1.0\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "build" "--fallback-to-build" "--module=C:\\Users\\marko\\Desktop\\topsecret\\server-alt\\node_modules\\sqlite3\\lib\\binding\\node-v64-win32-x64\\node_sqlite3.node" "--module_name=node_sqlite3"
"--module_path=C:\\Users\\marko\\Desktop\\topsecret\\server-alt\\node_modules\\sqlite3\\lib\\binding\\node-v64-win32-x64" "--napi_version=3" "--node_abi_napi=napi"
gyp ERR! cwd C:\Users\marko\Desktop\topsecret\server-alt\node_modules\sqlite3
gyp ERR! node -v v10.1.0
gyp ERR! node-gyp -v v3.6.2
gyp ERR! not ok
node-pre-gyp ERR! build error
node-pre-gyp ERR! stack Error: Failed to execute 'C:\Program Files\nodejs\node.exe C:\Users\marko\AppData\Roaming\nvm\v10.1.0\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js
build --fallback-to-build --module=C:\Users\marko\Desktop\topsecret\server-alt\node_modules\sqlite3\lib\binding\node-v64-win32-x64\node_sqlite3.node --module_name=node_sqlite3 --module_path=C:\Users\marko\Desktop\topsecret\server-alt\node_modules\sqlite3\lib\binding\node-v64-win32-x64 --napi_version=3 --node_abi_napi=napi' (1)
node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (C:\Users\marko\Desktop\topsecret\server-alt\node_modules\sqlite3\node_modules\node-pre-gyp\lib\util\compile.js:83:29)
node-pre-gyp ERR! stack     at ChildProcess.emit (events.js:182:13)
node-pre-gyp ERR! stack     at maybeClose (internal/child_process.js:957:16)
node-pre-gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:246:5)
node-pre-gyp ERR! System Windows_NT 10.0.16299
node-pre-gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\marko\\Desktop\\topsecret\\server-alt\\node_modules\\sqlite3\\node_modules\\node-pre-gyp\\bin\\node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd C:\Users\marko\Desktop\topsecret\server-alt\node_modules\sqlite3
node-pre-gyp ERR! node -v v10.1.0
node-pre-gyp ERR! node-pre-gyp -v v0.9.0
node-pre-gyp ERR! not ok
Failed to execute 'C:\Program Files\nodejs\node.exe C:\Users\marko\AppData\Roaming\nvm\v10.1.0\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js build --fallback-to-build --module=C:\Users\marko\Desktop\topsecret\server-alt\node_modules\sqlite3\lib\binding\node-v64-win32-x64\node_sqlite3.node --module_name=node_sqlite3 --module_path=C:\Users\marko\Desktop\topsecret\server-alt\node_modules\sqlite3\lib\binding\node-v64-win32-x64 --napi_version=3 --node_abi_napi=napi' (1)

[wormen]

@cannap you have a problem not with the build

node-pre-gyp ERR! Tried to download(403): https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v4.0.0/node-v64-win32-x64.tar.gz

[Apollon77]

I can verify as soon as node 10.1 is official ... nvm cannot find it and so appveyor not too

[cannap]

@wormen yea

https://i.gyazo.com/52a84d537f77020ebb15746d9de6527e.png

Error>
<Code>AccessDenied</Code>
<Message>Access Denied</Message>
<RequestId>4290B782939415E6</RequestId>
<HostId>
2MjZwOVOr+PJ4ercqCBwLMf0YSa8iRwAQeRVhOqjvZUE3he6AgV61JJWrPZGUJEOiDXZwcm4Isw=
</HostId>
</Error>

[vico93]

I'm using Discord.RSS and still cant download the binaries (403 error) or build from source (gist)

https://github.com/synzen/Discord.RSS/issues/51

[Kiemes]

I don't understand why the S3 bucket is not public if we should download files from there?
Regardless of the version, no tar.gz can be downloaded.
https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v3.1.13/node-v64-darwin-x64.tar.gz
Or is something wrong with the URL?

[stevefan1999-personal]

@Kiemes Yes, I'm facing the same problem with X64 Windows binaries. Is the S3 bucket quota exceeded?

[aslak01]

Any progress or workarounds so far?

[asm0dey]

Can be built on node 10.4

[aslak01]

Same error for me as before with node 10.4 on MacOS.

gyp ERR! node -v v10.4.0
gyp ERR! node-gyp -v v3.6.2
gyp ERR! not ok 

node-pre-gyp ERR! node -v v10.4.0
node-pre-gyp ERR! node-pre-gyp -v v0.6.38
node-pre-gyp ERR! not ok 

[10mitri]

4.0.0 doesn't work so far on W7 / 32 / node 10.4.0

node-pre-gyp ERR! Tried to download(501): https://mapbox-node-binary.s3.amazonaw
s.com/sqlite3/v4.0.0/node-v64-win32-ia32.tar.gz
node-pre-gyp ERR! Pre-built binaries not found for [email protected] and [email protected]
 (node-v64 ABI, unknown) (falling back to source compile with node-gyp)

[SeyZ]

We've removed our sqlite3 support at Lumber (https://github.com/forestadmin/lumber) due to this issue. Waiting for the new version of node-sqlite3 to re-support it.

Please, let me know if I can be helpful in any way ;)

[vico93]

@SeyZ this is sad. Similar case with @synzen and Discord.RSS (in this case he replaced sqlite3 to sqlite).

[springmeyer]

[email protected] has now been released with node v10 support.

[epaminond]

@springmeyer , it seems you forgot to add node 10.x to "List if supported platforms" in README.
And still an error mentioned by @NicoHood above appears for me.

I'm trying to implement node v10.x support for botpress which depends on sqlite3. But even after upgrading sqlite3 to 4.0.1 an error occurs for me when trying to start an app:

 npm install sqlite3 --save
Error: Cannot find module '/home/epaminond/bot/node_modules/botpress/node_modules/sqlite3/lib/binding/node-v64-linux-x64/node_sqlite3.node'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:594:15)
    at Function.Module._load (internal/modules/cjs/loader.js:520:25)
    at Module.require (internal/modules/cjs/loader.js:650:17)
    at require (internal/modules/cjs/helpers.js:20:18)
    at Object.<anonymous> (/home/epaminond/bot/node_modules/botpress/node_modules/sqlite3/lib/sqlite3.js:4:15)
    at Module._compile (internal/modules/cjs/loader.js:702:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:713:10)
    at Module.load (internal/modules/cjs/loader.js:612:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:551:12)
    at Function.Module._load (internal/modules/cjs/loader.js:543:3)
    at Module.require (internal/modules/cjs/loader.js:650:17)
    at require (internal/modules/cjs/helpers.js:20:18)
    at Client_SQLite3._driver (/home/epaminond/bot/node_modules/botpress/node_modules/knex/lib/dialects/sqlite3/index.js:75:12)
    at Client_SQLite3.initializeDriver (/home/epaminond/bot/node_modules/botpress/node_modules/knex/lib/client.js:219:26)
    at Client_SQLite3.Client (/home/epaminond/bot/node_modules/botpress/node_modules/knex/lib/client.js:113:10)
    at new Client_SQLite3 (/home/epaminond/bot/node_modules/botpress/node_modules/knex/lib/dialects/sqlite3/index.js:58:20)

It seems the folder name isn't correct for some reason:

$ ls /home/epaminond/bot/node_modules/botpress/node_modules/sqlite3/lib/binding
node-v57-linux-x64 # Expected node-v64-linux-x64

[springmeyer]

@epaminond The fact that node-v57-linux-x64 got installed indicates that you previously ran an install with node v8, while the fact that you are seeing Cannot find module '...lib/binding/node-v64-linux-x64/node_sqlite3.node' indicates you are trying to run your application with node v10. It seems to me that you need to do a full re-install with the same node version, and therefore that that problem is on your end and not a problem in node-sqlite3. If you feel differently, please create a new issue rather than commenting on this issue. And provide the exact details to replicate the problem, not just a partial log (like above).

[epaminond]

@springmeyer , you were right, seems to work when built under the same node-version.

[laurent22]

This issue is fixed in https://github.com/laurent22/joplin/commit/3fc54d7ffdb42efdd3a825a16dca6fc37e31c555