Node-gyp: How to generate compile_commands.json file?

[JCMais]

This is more of a question than an issue, so, sorry if this is not the correct place to ask

I'm trying to configure VsCode for intelliSense while creating a Node.js native module, and for that it's interesting to have a compile_commands.json file (from official cpp extension documentation), I saw that a generator for that was added some days ago, here https://github.com/nodejs/node-gyp/commit/5c2aad83ea45308bebfe3a10520f0cf3010b5bbf

So my question is, how I can use it?

[refack]

AFAIK we don't have a nice node-gyp wrapper for that. You could work around the JS parts and call it directly.

1. Run a regular node-gyp build
2. Copy the arguments used to run GYP from the output of (1)
3. Replace the format generator (-f) with -f compile_commands_json

example interesting node-gyp output part (with -f make):

gyp info spawn /usr/bin/python
gyp info spawn args [ '/data/iojs/build/workspace/node-test-commit-linuxone/nodes/rhel72-s390x/deps/npm/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/data/iojs/build/workspace/node-test-commit-linuxone/nodes/rhel72-s390x/test/addons-napi/2_function_arguments/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/data/iojs/build/workspace/node-test-commit-linuxone/nodes/rhel72-s390x/deps/npm/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/data/iojs/build/workspace/node-test-commit-linuxone/nodes/rhel72-s390x/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/data/iojs/build/workspace/node-test-commit-linuxone/nodes/rhel72-s390x',
gyp info spawn args   '-Dnode_gyp_dir=/data/iojs/build/workspace/node-test-commit-linuxone/nodes/rhel72-s390x/deps/npm/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/data/iojs/build/workspace/node-test-commit-linuxone/nodes/rhel72-s390x/$(Configuration)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/data/iojs/build/workspace/node-test-commit-linuxone/nodes/rhel72-s390x/test/addons-napi/2_function_arguments',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.' ]

[refack]

P.S. if you feel like implementing that and submitting a PR that would be greatly appreciated.
A reasonable starting point might be where the above output comes from:
https://github.com/nodejs/node-gyp/blob/bac9e44a8c0c55ff92366de18dea4d6d5017823b/lib/configure.js#L327

[JCMais]

Btw I got this working, but I don't think it's correct in the repo.

The generator was added to tools/gyp/pylib/gyp/generator, but it should be available at gyp/pylib/gyp/generator, otherwise gyp is not going to find it.

[simark]

Not sure if there was any progress on this, but one way is to use bear. I ran bear npm install on the node-pty repo and got a seemingly correct compile_commands.json.

[marechal-p]

The command to run is something like node-gyp -- configure -f=gyp.generator.compile_commands_json, where the generator name should be a Python module name.

Problem is that I cannot find a way to access the module in tools/gyp/... as mentioned by @JCMais.

Either add tools to the sys.path, or move the file inside gyp/pylib/gpy/generator, unless there's actually a way to reference the module and we just missed it?

[laverdet]

For those who found this in Google, I had to add the .py suffix to the format argument, otherwise I received the error: "ImportError: No module named gyp.generator.compile_commands_json".

clangd 10.0.0 works fine with the compile_commands.json file generated this way but clang-tidy can't find the correct system headers on OS X so I had to inject them into the json file to get that tool working. I had the same issue with clangd on llvm 9.0.0.

Here is my wrapper script which cleans up the extra compile_flags.json that are generated for each configuration (Debug, etc) and injects system header locations. It also bounces clangd to pull in the new manifest.

#!/bin/sh
node-gyp configure --release -- -f gyp.generator.compile_commands_json.py
mv Release/compile_commands.json ./
rmdir Release
EXTRA=$(ls */compile_commands.json)
echo "$EXTRA" | xargs rm
echo "$EXTRA" | xargs -n1 dirname | xargs rmdir
ESCAPED_FLAGS=$(echo | cc -Wp,-v -x c++ - -fsyntax-only 2>&1 | \
    egrep '^ /' | sed 's/(framework directory)//' | \
    sed 's/^ /-isystem/' | perl -pe 's/\//\\\//g' | tr '\n' ' ')
perl -i -pe 's/ -c / '"$ESCAPED_FLAGS"' -c /;' compile_commands.json
killall clangd

[aminya]

Running node-gyp -- configure -f "gyp.generator.compile_commands_json" on Windows gives this error:

gyp: gyp.generator.compile_commands_json not found (cwd: <myproj>) while trying to load gyp.generator.compile_commands_json
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (~~~\node-gyp\lib\configure.js:351:16)
gyp ERR! stack     at ChildProcess.emit (events.js:315:20)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:275:12)
gyp ERR! System Windows_NT 10.0.19042
gyp ERR! command "~~~\\node.exe" "~~~\\node-gyp\\bin\\node-gyp.js" "configure" "-f" "gyp.generator.compile_commands_json"
gyp ERR! cwd <myproj>
gyp ERR! node -v v12.18.1
gyp ERR! node-gyp -v v7.1.2
gyp ERR! not ok