Vscode-cpptools: feature request: add node-gyp include paths

[bmacnaughton]

When writing node-gyp-based C/C++ extensions to nodejs, node-gyp adds the currently active node include directory to the include path. the binding.gyp file can specify additional paths. These are not configured in system environment variables (and when using nvm the location of the node include directory is dynamic which is the real issue). When using node-addon-api, the "<!@(node -p \"require('node-addon-api').include\")" is used in binding.gyp to add node-addon-api's include directory to the include path.

While node-gyp automatically adds these paths for compilation, cpp-tools does not know about them. It would be useful to have an option to automatically include the node include directory as well as those defined in binding.gyp so that vscode can find those files. Currently it squiggles them and isn't able to find declarations and definitions.

Overall it would be a nice addition to have an option of something like C_Cpp.addNodeGypIncludePaths or something similar. The nvm setting seems most important, because hardcoding a possibly non-existent node-addon-api location is not a big deal. So maybe it's more limited and something like C_Cpp.addNvmNodeIncludePath.

Thanks for your consideration.

As a workaround, I created a simple extension that implements a command to return the node include directory:

let disposable = vscode.commands.registerCommand('extension.nodeIncludeNvm', function () {
  // just return a best guess as to the include directory for this version
  // of node. presume NVM_BIN var points to bin if it exists. if not, presume
  // /usr/local/include/node. no, it's not sophisticated.
  let path;
  if (process.env.NVM_BIN) {
    path = process.env.NVM_BIN.replace(/\/bin$/, '/include');
  } else {
    path = '/usr/local/include/node';
  }
  return Promise.resolve(path);
});

It works within the "echo nodeIncludeNvm path" task:

    "tasks": [
        {
            "label": "echo nodeIncludeNvm path",
            "type": "shell",
            "command": "echo ${command:extension.nodeIncludeNvm}",
            "problemMatcher": []
        },
        {
          "label": "echo C_Cpp.default.includePath",
          "type": "shell",
          "command": "echo ${config:C_Cpp.default.includePath}",
          "problemMatcher": []
        }
    ]

but I am unable to get it to "stick" in C_Cpp.default.includePath. My project settings.json:

  "C_Cpp.default.includePath": [
    "${workspaceFolder}",
    "${command:extension.nodeIncludeNvm}",
    "${workspaceFolder}/node_modules/node-addon-api"
  ],

and executing "echo C_Cpp.default.includePath" task output is:

> Executing task: echo ${workspaceFolder},${command:extension.nodeIncludeNvm},${workspaceFolder}/node_modules/node-addon-api <

/home/bruce/github.com/bmacnaughton/nvm-node-include,,/home/bruce/github.com/bmacnaughton/nvm-node-include/node_modules/node-addon-api

If I'm just doing something wrong trying to get ${command:extension.nodeIncludeNvm} to work let me know. In any case my workaround seems to work even though the task doesn't handle the command substitution - cpptools is finding the include files.

[ruifortes]

Let me see if I understand MicroSoft strategy regarding VsCode....

Instead of helping existing open source projects (like atom) the create an alternative, them sponsor a the creation of fundamental extensions like cpptools, then develop it but just to the point where it barelly usable, them mantain it just by fixing bugs but keeping it bare minimal.
This way anyone willing to invest time creating an better alternative knows that as soon as it get's close MS will "only" then add the missing features.
I find it strange that there are not many contributers outside MS. I'm I wrong?
Maybe this is why it's so difficult to use VsCode instead of VStudio.
Lookily there's the VStudio Commutity Edition...but nevertheless it's hard to avoid it for C++ development.

Have to admire their long game strategy though

Ok, this is just a somewhat uninformed intuition from a critical jerk so.... feel free to refute me.

[sean-mcmanus]

@ruifortes Are your comments related to this feature request somehow? What features did you want? I don't have any special knowledge on Microsoft's "strategy" in regards to VS Code, Atom, and VS -- our C/C++ team is focused on making our product better and our intention is not "keeping it bare minimal". You are correct that our open source TypeScript component only gets occasional outside contributors (the language server process cpptools and cpptools-srv are closed source).

[github-actions[bot]]

This feature request is being closed due to insufficient upvotes. When enough upvotes are received, this issue will be eligible for our backlog.

[shudingbo]

need the feature

[bmacnaughton]

it's true that developing node-addons is a relatively small portion of all c/c++ work that is done, but it seems that are implementations that do not require node-gyp/node-addon-specific work, e.g., allow an environment variable that is automagically integrated into the search paths but uses late-evaluation so that it can incorporate NVM settings or similar.

closing due to "not enough upvotes" for a feature that is really critical for core c++/node development seems like the wrong criteria.

[shudingbo]

it's true that developing node-addons is a relatively small portion of all c/c++ work that is done, but it seems that are implementations that do not require node-gyp/node-addon-specific work, e.g., allow an environment variable that is automagically integrated into the search paths but uses late-evaluation so that it can incorporate NVM settings or similar.

closing due to "not enough upvotes" for a feature that is really critical for core c++/node development seems like the wrong criteria.

You are right!!

[sean-mcmanus]

@bmacnaughton We'd probably be willing to accept a PR that adds that setting.

[shudingbo]

We'd probably be willing to accept a PR that adds that setting

Thank you for the first

[bmacnaughton]

i'll take a look. thanks for the opening.

[bmacnaughton]

@shudingbo - as a workaround (not sure when i'll be able to get to this - there's a bit of a learning curve for me on vscode and vscode-cpptools) you can create a .vscode/c_cpp_properties.json file like:

{
    "configurations": [
        {
            "name": "node-addon",
            "includePath": [
                "${workspaceFolder}/**",
                "${workspaceFolder}/node_modules/nan",
                "${workspaceFolder}/node_modules/node-addon-api",
                "${env:NVM_INC}"
            ],
            "defines": [],
            "compilerPath": "/usr/bin/gcc",
            "cStandard": "c11",
            "cppStandard": "c++11",
            "intelliSenseMode": "gcc-x64"
        }
    ],
    "version": 4
}

i did change the code for nvm to create the NVM_INC env var so this was possible; the PR was accepted pretty quickly.

[shudingbo]

@shudingbo - as a workaround (not sure when i'll be able to get to this - there's a bit of a learning curve for me on vscode and vscode-cpptools) you can create a .vscode/c_cpp_properties.json file like:

{
    "configurations": [
        {
            "name": "node-addon",
            "includePath": [
                "${workspaceFolder}/**",
                "${workspaceFolder}/node_modules/nan",
                "${workspaceFolder}/node_modules/node-addon-api",
                "${env:NVM_INC}"
            ],
            "defines": [],
            "compilerPath": "/usr/bin/gcc",
            "cStandard": "c11",
            "cppStandard": "c++11",
            "intelliSenseMode": "gcc-x64"
        }
    ],
    "version": 4
}

i did change the code for nvm to create the NVM_INC env var so this was possible; the PR was accepted pretty quickly.

My workspace has More than 10 Adon. If use .vscode/c_cpp_properties.json, in one addon the cpptool always jump to another addon's define...

[sean-mcmanus]

@bmacnaughton FYI, the master branch is intended to be used with our latest binaries (cpptools/cpptools-srv), which may not have been published yet. So if you get strange/buggy results you may want to work from a branch based on the release branch instead, which matches 1.1.2 binaries. After we release 1.2.0-insiders, then that binary should be compatible with master (until we make some other breaking change). In the future, it's possible we could publish some "daily" binary that matches the master branch. We could also release our "in progress" binaries if you really want to work from the master branch.

UPDATE: The binaries necessary to work out of the "main" branch are currently available at https://github.com/microsoft/vscode-cpptools/releases/tag/1.2.0-preview .

[bmacnaughton]

this is my christmas holiday project - thanks for the pointer.

[bmacnaughton]

@shudingbo can you help me understand this:

My workspace has More than 10 Adon. If use .vscode/c_cpp_properties.json, in one addon the cpptool always jump to another addon's define...

i don't understand your workspace structure - can you provide some details on that? i'm thinking of a couple different ways to approach this but not sure what you need as a solution.

@sean-mcmanus - please let me know your thoughts about my thinking here. i'm not familiar with either vscode or the cpp extension; i do not have a good mental model of what's going on yet. and if i'm just hopelessly lost, let me know and i'll stop bugging you.

i can see two ways to approach this, one very general and one that handles basic use cases but isn't very general.

• the general one requires an async component to be added because it would have to read and evaluate binding.gyp and/or other files in order to return additional include paths.
• the basic approach can just check whether nan, node-addon-api (and possibly others) are dependencies in package.json and add hardcoded paths, based on their presence.

general approach

complicates configuration because i don't see how to do this (in a general fashion) without introducing an async component into the configuration. i don't see any other way to add arbitrary paths to the configuration - adding those in binding.gyp requires reading binding.gyp and extracting the appropriate parts. a script could clearly add any other paths it needed to from the environment, or elsewhere, as well.

but the only async function in configuration is invoked in the constructor where nothing can wait for it to complete. so this will require a more complex solution in that nothing in the normal configuration settings currently resolves asynchronously.

it seems like the work parsing the c/c++ files is in the closed-source language server so that means that calling updateServerOnFolderSettingsChange() seems to be the key to telling the language server about the include paths.

a C_Cpp.includePathScript could specify a script that would be spawned and executed. stdout would contain a delimited list (comma, colon?) of paths to be added to includePath. Any output to stderr and any errors processing the delimited list would result in nothing being added. because this is spawned it should be done async and would need to update settings on completion. given the current structure this would result in an additional call to updateServerOnFolderSettingsChange() maybe it's possible to send only a single setting to the server but i didn't see it.

there are plenty of open questions for me on this but i think i can see a path to accomplish it, though that might be my ignorance of things.

basic approach

this is straightforward and relatively simple but not in any way flexible. basically have a map of installed packages (nan and node-addon-api seem to be the only real candidates i am aware of) and if C_Cpp.addNodeLibraries is true, add them to the include path. In order to make this a synchronous activity those locations would need to be hardcoded, relative to the node_modules directory. The node include files can already be handled via ${env:NVM_INC}, so those already have a solution - no reason not to set that at either the user or system level.

it's also possible to do these async, store results in the CppProperties object, and kick off the updateServerOnFolderSettingsChange() when all are done after teaching it to merge those results into the include path as well. that would prevent hardcoding and still keep it a bit simpler than a really general facility, but not much.

miscellaneous

i'll have to play with various approaches to get things working; these are just initial thoughts and observations.

fwiw, the closed source parts made it harder to figure out what was going on where - i spent a fair amount of time looking for where '#include' files would be opened - one of multiple red-herring approaches on my part. it's a bit like trying to figure out the sound of clapping by watching one hand move.

[sean-mcmanus]

Async processing for your paths should be fine. We do the same thing for vcpkg includes -- see code related to buildVcpkgIncludePath https://github.com/microsoft/vscode-cpptools/blob/insiders/Extension/src/LanguageServer/configurations.ts#L360 (to get that vcpkg code to not be skipped you'd need to install vcpkg, see https://docs.microsoft.com/en-us/cpp/build/install-vcpkg?view=msvc-160&tabs=windows ). Note: working out of the insiders branch requires binaries from https://github.com/microsoft/vscode-cpptools/releases/tag/1.2.0-preview). Are you able to use that async code as an example? You should make sure configurationIncomplete is false until the include paths are ready -- that will delay sending the ChangeCppPropertiesNotification until those paths are added.

Yeah, all C++ parsing is done in our closed source part.

[bmacnaughton]

@sean-mcmanus - thank you. i saw the vcpkg includes but that looked to me like it was only invoked in the CppProperties constructor. ideally, adding or removing a package while developing would cause the configuration to be changed but maybe i'm missing where that would happen.

even the simple approach will change when the C_Cpp.addNodeLibraries (for lack of a better name) configuration changes. but i think you just addressed one question i had - leaving configurationIncomplete ~false~true until the include paths are ready delays sending the ChangeCppPropertiesNotification. i hadn't gotten that.

it seems that i might need to add a vcpkgPathReady equivalent too as applyDefaultIncludePathsAndFrameworks() checks that specifically (because it's async).

does that make sense?

and yes, i have no trouble using the vcpkg include code as an example. there were two parts i didn't get: 1) that configurationIncomplete would defer ChangeCppPropertiesNotification and 2) picking up on installation of nan or node-addon-api and handling the reconfiguration. you addressed 1.

[bmacnaughton]

it would get quite a bit more complex to handle the general case because the code wouldn't know what files an arbitrary script would be looking at in order to generate the include path additions. so i think going with the basic approach makes the most sense.

[sean-mcmanus]

Okay, that seems fine to me.

[shudingbo]

@shudingbo can you help me understand this:

My workspace has More than 10 Adon. If use .vscode/c_cpp_properties.json, in one addon the cpptool always jump to another addon's define...

i don't understand your workspace structure - can you provide some details on that? i'm thinking of a couple different ways to approach this but not sure what you need as a solution.

@sean-mcmanus - please let me know your thoughts about my thinking here. i'm not familiar with either vscode or the cpp extension; i do not have a good mental model of what's going on yet. and if i'm just hopelessly lost, let me know and i'll stop bugging you.

i can see two ways to approach this, one very general and one that handles basic use cases but isn't very general.

• the general one requires an async component to be added because it would have to read and evaluate binding.gyp and/or other files in order to return additional include paths.
• the basic approach can just check whether nan, node-addon-api (and possibly others) are dependencies in package.json and add hardcoded paths, based on their presence.

general approach

complicates configuration because i don't see how to do this (in a general fashion) without introducing an async component into the configuration. i don't see any other way to add arbitrary paths to the configuration - adding those in binding.gyp requires reading binding.gyp and extracting the appropriate parts. a script could clearly add any other paths it needed to from the environment, or elsewhere, as well.

but the only async function in configuration is invoked in the constructor where nothing can wait for it to complete. so this will require a more complex solution in that nothing in the normal configuration settings currently resolves asynchronously.

it seems like the work parsing the c/c++ files is in the closed-source language server so that means that calling updateServerOnFolderSettingsChange() seems to be the key to telling the language server about the include paths.

a C_Cpp.includePathScript could specify a script that would be spawned and executed. stdout would contain a delimited list (comma, colon?) of paths to be added to includePath. Any output to stderr and any errors processing the delimited list would result in nothing being added. because this is spawned it should be done async and would need to update settings on completion. given the current structure this would result in an additional call to updateServerOnFolderSettingsChange() maybe it's possible to send only a single setting to the server but i didn't see it.

there are plenty of open questions for me on this but i think i can see a path to accomplish it, though that might be my ignorance of things.

basic approach

this is straightforward and relatively simple but not in any way flexible. basically have a map of installed packages (nan and node-addon-api seem to be the only real candidates i am aware of) and if C_Cpp.addNodeLibraries is true, add them to the include path. In order to make this a synchronous activity those locations would need to be hardcoded, relative to the node_modules directory. The node include files can already be handled via ${env:NVM_INC}, so those already have a solution - no reason not to set that at either the user or system level.

it's also possible to do these async, store results in the CppProperties object, and kick off the updateServerOnFolderSettingsChange() when all are done after teaching it to merge those results into the include path as well. that would prevent hardcoding and still keep it a bit simpler than a really general facility, but not much.

miscellaneous

i'll have to play with various approaches to get things working; these are just initial thoughts and observations.

fwiw, the closed source parts made it harder to figure out what was going on where - i spent a fair amount of time looking for where '#include' files would be opened - one of multiple red-herring approaches on my part. it's a bit like trying to figure out the sound of clapping by watching one hand move.

I Create a sample in github https://github.com/shudingbo/testcpptool
Repeat mistakes jump, if click #include <fight.h> in dog.h or cat.h ,always jump to addon1's fight.h.

• addon1.gyp and addon2.gyp has configure './' in include_dirs, so use <> include fight.h, is correct
  
  
  
  • 'include_dirs': [ './','../include',"<!@(node -p \"require('node-addon-api').include\")"]
  
  

[bmacnaughton]

Repeat mistakes jump, if click #include in dog.h or cat.h ,always jump to addon1's fight.h.
i'm probably missing something pretty obvious, but how do you click #include <fight.h>? i can right-click the selected text and choose "Switch Header/Source" and it always goes to the correct source file. But i'm not sure that is what you're doing. i haven't ever clicked on the actual include text to open the file.

it seems that setting the path to node-addon-api's include directory is independent of that issue. and the approach i'm thinking of, adding node-addon-api's include directory to includePath if node-addon-api is a dependency, would solve the problem of not being able to find those files.

and a user, not workspace, setting of C_Cpp.default.includePath to "${env:NVM_INC}" would address not finding node's libraries (if you're using nvm).

let me know if i'm missing something.