Kudu: node.js deployment: Error MSB3428: Could not load the Visual C++ component "VCBuild.exe"

[bpasero]

Issue:
npm deployment produces the following error

remote: Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch.
remote: MSBUILD : error MSB3428: Could not load the Visual C++ component "VCBuild.exe". To fix this, 1) install the .NET Framework 2.0 SDK, 2) install Microsoft Visual Studio 2005 or 3) add the location of the component to the system path if it is installed elsewhere.  [D:\home\site\wwwroot\node_modules\tsd\node_modules\bufferstream\node_modules\buffertools\build\binding.sln]

To reproduce:

• enable git for a azure web site
• clone locally
• add a package.json (see contents below)
• git push to trigger deployment

package.json

{
    "name": "s",
    "description": "s",
    "version": "0.6.0",
    "author": {
        "name": "Microsoft Corporation"
    },
    "private": true,
    "engines": { "node": ">= 0.10.0 < 0.11.0" },

    "dependencies": {
        "tsd": "*"
    }
}

[bpasero]

When I do the same from my local machine, I can successfully run "npm install". I have Visual Studio installed, but I think this is just an issue with having the .NET SDK on the path?

[amitapl]

@bpasero Can you share the full output of the deployment?

[mwtian-zz]

I can reproduce the error in Azure websites with the provided package.json.

$ git push https://[email protected]:443/nodejsex.git
Password for 'https://[email protected]:443':
Counting objects: 13, done.
Delta compression using up to 12 threads.
Compressing objects: 100% (9/9), done.
Writing objects: 100% (10/10), 1.10 KiB | 0 bytes/s, done.
Total 10 (delta 1), reused 0 (delta 0)
remote: Updating branch 'master'.
remote: Updating submodules.
remote: Preparing deployment for commit id 'ed0ca6637c'.
remote: Generating deployment script.
remote: Running deployment command...
remote: Handling node.js deployment.
remote: KuduSync.NET from: 'D:\home\site\repository' to: 'D:\home\site\wwwroot'
remote: Copying file: 'package.json'
remote: Copying file: 'server.js'
remote: Node.js versions available on the platform are: 0.6.17, 0.6.20, 0.8.2, 0.8.19, 0.8.26, 0.10.
5, 0.10.18, 0.10.21, 0.10.24.
remote: Selected node.js version 0.10.24. Use package.json file to choose a different version.
remote: npm http GET https://registry.npmjs.org/tsd
remote: npm http 304 https://registry.npmjs.org/tsd
remote: npm http GET https://registry.npmjs.org/exit
remote: npm http GET https://registry.npmjs.org/open
remote: npm http GET https://registry.npmjs.org/q
remote: npm http GET https://registry.npmjs.org/q-io
remote: npm http GET https://registry.npmjs.org/mkdirp
remote: npm http GET https://registry.npmjs.org/uri-templates/0.1.2
remote: npm http GET https://registry.npmjs.org/minimist/0.0.7
remote: npm http GET https://registry.npmjs.org/semver
remote: npm http GET https://registry.npmjs.org/tv4-reporter
remote: npm http GET https://registry.npmjs.org/assertion-error/1.0.0
remote: npm http GET https://registry.npmjs.org/jsesc
remote: npm http GET https://registry.npmjs.org/tv4
remote: npm http GET https://registry.npmjs.org/js-yaml
remote: npm http GET https://registry.npmjs.org/json-pointer
remote: npm http GET https://registry.npmjs.org/miniwrite
remote: npm http GET https://registry.npmjs.org/ministyle
remote: npm http GET https://registry.npmjs.org/minitable
remote: npm http GET https://registry.npmjs.org/es6-shim/0.9.1
remote: npm http GET https://registry.npmjs.org/request
remote: npm http GET https://registry.npmjs.org/bufferstream
remote: npm http GET https://registry.npmjs.org/event-stream
remote: npm http GET https://registry.npmjs.org/universal-analytics
remote: npm http GET https://registry.npmjs.org/weak-map
remote: npm http GET https://registry.npmjs.org/uuid
remote: npm http GET https://registry.npmjs.org/detect-indent
remote: npm http 304 https://registry.npmjs.org/open
remote: npm http 304 https://registry.npmjs.org/exit
remote: npm http 304 https://registry.npmjs.org/q-io
remote: npm http 304 https://registry.npmjs.org/q
remote: npm http 304 https://registry.npmjs.org/uri-templates/0.1.2
remote: npm http 304 https://registry.npmjs.org/mkdirp
remote: npm http 304 https://registry.npmjs.org/minimist/0.0.7
remote: npm http 304 https://registry.npmjs.org/semver
remote: npm http 304 https://registry.npmjs.org/assertion-error/1.0.0
remote: npm http 304 https://registry.npmjs.org/jsesc
remote: npm http 304 https://registry.npmjs.org/tv4-reporter
remote: npm http 304 https://registry.npmjs.org/js-yaml
remote: npm http 304 https://registry.npmjs.org/tv4
remote: npm http 304 https://registry.npmjs.org/json-pointer
remote: npm http 304 https://registry.npmjs.org/miniwrite
remote: npm http 304 https://registry.npmjs.org/es6-shim/0.9.1
remote: npm http 304 https://registry.npmjs.org/request
remote: npm http 304 https://registry.npmjs.org/ministyle
remote: npm http 304 https://registry.npmjs.org/event-stream
remote: npm http 304 https://registry.npmjs.org/universal-analytics
remote: npm http 304 https://registry.npmjs.org/weak-map
remote: npm http 304 https://registry.npmjs.org/uuid
remote: npm http 304 https://registry.npmjs.org/minitable
remote: npm http 304 https://registry.npmjs.org/bufferstream
remote: npm http 304 https://registry.npmjs.org/detect-indent
remote: npm http GET https://registry.npmjs.org/configstore
remote: npm http GET https://registry.npmjs.org/chalk
remote: npm http GET https://registry.npmjs.org/jsonpointer.js/0.3.0
remote: npm http GET https://registry.npmjs.org/minichain
remote: npm http 304 https://registry.npmjs.org/chalk
remote: npm http 304 https://registry.npmjs.org/configstore
remote: npm http GET https://registry.npmjs.org/foreach
remote: npm http 304 https://registry.npmjs.org/jsonpointer.js/0.3.0
remote: npm http 304 https://registry.npmjs.org/minichain
remote: npm http 304 https://registry.npmjs.org/foreach
remote: npm http GET https://registry.npmjs.org/lodash
remote: npm http GET https://registry.npmjs.org/osenv/0.0.3
remote: npm http GET https://registry.npmjs.org/graceful-fs
remote: npm http GET https://registry.npmjs.org/has-color
remote: npm http GET https://registry.npmjs.org/strip-ansi
remote: npm http GET https://registry.npmjs.org/ansi-styles
remote: npm http 304 https://registry.npmjs.org/lodash
remote: npm http GET https://registry.npmjs.org/bufferjs
remote: npm http GET https://registry.npmjs.org/buffertools
remote: npm http 304 https://registry.npmjs.org/has-color
remote: npm http GET https://registry.npmjs.org/qs
remote: npm http GET https://registry.npmjs.org/forever-agent
remote: npm http GET https://registry.npmjs.org/node-uuid
remote: npm http GET https://registry.npmjs.org/json-stringify-safe
remote: npm http GET https://registry.npmjs.org/mime
remote: npm http GET https://registry.npmjs.org/form-data
remote: npm http GET https://registry.npmjs.org/tunnel-agent
remote: npm http GET https://registry.npmjs.org/http-signature
remote: npm http GET https://registry.npmjs.org/tough-cookie
remote: npm http GET https://registry.npmjs.org/oauth-sign
remote: npm http GET https://registry.npmjs.org/hawk
remote: npm http GET https://registry.npmjs.org/aws-sign2
remote: npm http GET https://registry.npmjs.org/through
remote: npm http GET https://registry.npmjs.org/from
remote: npm http GET https://registry.npmjs.org/duplexer
remote: npm http GET https://registry.npmjs.org/map-stream
remote: npm http GET https://registry.npmjs.org/stream-combiner
remote: npm http GET https://registry.npmjs.org/split
remote: npm http GET https://registry.npmjs.org/pause-stream/0.0.11
remote: npm http 304 https://registry.npmjs.org/ansi-styles
remote: npm http 304 https://registry.npmjs.org/strip-ansi
remote: npm http 304 https://registry.npmjs.org/bufferjs
remote: npm http 304 https://registry.npmjs.org/forever-agent
remote: npm http 304 https://registry.npmjs.org/qs
remote: npm http 304 https://registry.npmjs.org/buffertools
remote: npm http 304 https://registry.npmjs.org/json-stringify-safe
remote: npm http 304 https://registry.npmjs.org/form-data
remote: npm http 304 https://registry.npmjs.org/node-uuid
remote: npm http 304 https://registry.npmjs.org/http-signature
remote: npm http 304 https://registry.npmjs.org/tunnel-agent
remote: npm http 304 https://registry.npmjs.org/mime
remote: npm http 304 https://registry.npmjs.org/tough-cookie
remote: npm http 304 https://registry.npmjs.org/hawk
remote: npm http 304 https://registry.npmjs.org/aws-sign2
remote: npm http 304 https://registry.npmjs.org/oauth-sign
remote: npm http 304 https://registry.npmjs.org/duplexer
remote: npm http 304 https://registry.npmjs.org/through
remote: npm http 304 https://registry.npmjs.org/from
remote: npm http 304 https://registry.npmjs.org/map-stream
remote: npm http 304 https://registry.npmjs.org/stream-combiner
remote: npm http 304 https://registry.npmjs.org/split
remote: npm http 304 https://registry.npmjs.org/pause-stream/0.0.11
remote: npm http GET https://registry.npmjs.org/underscore
remote: npm http GET https://registry.npmjs.org/async
remote: npm http 304 https://registry.npmjs.org/graceful-fs
remote: npm http 304 https://registry.npmjs.org/osenv/0.0.3
remote: npm http 304 https://registry.npmjs.org/underscore
remote: npm http 304 https://registry.npmjs.org/async
remote:
remote: > [email protected] install D:\home\site\wwwroot\node_modules\tsd\node_modules\bufferstream\
node_modules\buffertools
remote: npm http GET https://registry.npmjs.org/combined-stream
remote: > node-gyp rebuild
remote: npm http 304 https://registry.npmjs.org/combined-stream
remote:
remote: npm http GET https://registry.npmjs.org/async
remote:
remote: npm http 304 https://registry.npmjs.org/async
remote: D:\home\site\wwwroot\node_modules\tsd\node_modules\bufferstream\node_modules\buffertools>nod
e "D:\Program Files (x86)\npm\1.3.21\node_modules\npm\bin\node-gyp-bin\\..\..\node_modules\node-gyp\
bin\node-gyp.js" rebuild
remote: gyp http GET http://nodejs.org/dist/v0.10.24/node-v0.10.24.tar.gz
remote: gyp http 200 http://nodejs.org/dist/v0.10.24/node-v0.10.24.tar.gz
remote: ....
remote: gyp http GET http://nodejs.org/dist/v0.10.24/node.lib
remote: gyp http GET http://nodejs.org/dist/v0.10.24/x64/node.lib
remote: gyp http 200 http://nodejs.org/dist/v0.10.24/node.lib
remote: gyp http 200 http://nodejs.org/dist/v0.10.24/x64/node.lib
remote: Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch.
remote: MSBUILD : error MSB3428: Could not load the Visual C++ component "VCBuild.exe". To fix this,
 1) install the .NET Framework 2.0 SDK, 2) install Microsoft Visual Studio 2005 or 3) add the locati
on of the component to the system path if it is installed elsewhere.  [D:\home\site\wwwroot\node_mod
ules\tsd\node_modules\bufferstream\node_modules\buffertools\build\binding.sln]
remote: gypnpm http GET https://registry.npmjs.org/assert-plus/0.1.2
remote: npm http GET https://registry.npmjs.org/asn1/0.1.11
remote: npm http GET https://registry.npmjs.org/ctype/0.5.2
remote: npm http GET https://registry.npmjs.org/argparse
remote: npm http GET https://registry.npmjs.org/esprima
remote: npm http 304 https://registry.npmjs.org/assert-plus/0.1.2
remote: npm http 304 https://registry.npmjs.org/ctype/0.5.2
remote: npm http 304 https://registry.npmjs.org/asn1/0.1.11
remote: npm http 304 https://registry.npmjs.org/argparse
remote: npm http 304 https://registry.npmjs.org/esprima
remote: npm http GET https://registry.npmjs.org/delayed-stream/0.0.5
remote: npm http 304 https://registry.npmjs.org/delayed-stream/0.0.5
remote: npm http GET https://registry.npmjs.org/punycode
remote: npm http 304 https://registry.npmjs.org/punycode
remote: npm http GET https://registry.npmjs.org/aws-sign
remote: npm http GET https://registry.npmjs.org/cookie-jar
remote: npm http 304 https://registry.npmjs.org/aws-sign
remote: npm http 304 https://registry.npmjs.org/cookie-jar
remote: npm http GET https://registry.npmjs.org/hoek
remote: npm http GET https://registry.npmjs.org/boom
remote: npm http GET https://registry.npmjs.org/sntp
remote: npm http GET https://registry.npmjs.org/cryptiles
remote: npm http 304 https://registry.npmjs.org/hoek
remote: npm http 304 https://registry.npmjs.org/boom
remote: npm http 304 https://registry.npmjs.org/sntp
remote: npm http 304 https://registry.npmjs.org/cryptiles
remote: npm http GET https://registry.npmjs.org/underscore.string
remote: npm http 304 https://registry.npmjs.org/underscore.string
remote: npm http GET https://registry.npmjs.org/qs
remote: npm http GET https://registry.npmjs.org/url2
remote: npm http GET https://registry.npmjs.org/mimeparse
remote: npm http GET https://registry.npmjs.org/mime
remote: npm http GET https://registry.npmjs.org/collections
remote: npm http 304 https://registry.npmjs.org/mime
remote: npm http 304 https://registry.npmjs.org/url2
remote: npm http 304 https://registry.npmjs.org/mimeparse
remote: npm http 304 https://registry.npmjs.org/qs
remote: npm http 304 https://registry.npmjs.org/collections
remote: npm http GET https://registry.npmjs.org/weak-map/1.0.0
remote: npm http 304 https://registry.npmjs.org/weak-map/1.0.0
remote: npm WARN optional dep failed, continuing [email protected]
remote: [email protected] node_modules\tsd
remote: +-- [email protected]
remote: +-- [email protected]
remote: +-- [email protected]
remote: +-- [email protected]
remote: +-- [email protected]
remote: +-- [email protected]
remote: +-- [email protected]
remote: +-- [email protected]
remote: +-- [email protected]
remote: +-- [email protected]
remote: +-- [email protected]
remote: +-- [email protected]
remote: +-- [email protected]
remote: +-- [email protected]
remote: +-- [email protected]
remote: +-- [email protected]
remote: +-- [email protected] ([email protected])
remote: +-- [email protected] ([email protected])
remote: +-- [email protected] ([email protected])
remote: +-- [email protected]
remote: +-- [email protected] ([email protected], [email protected], [email protected], [email protected].
11, [email protected], [email protected], [email protected])
remote: +-- [email protected] ([email protected], [email protected], [email protected])
remote: +-- [email protected] ([email protected], [email protected], [email protected], [email protected].
0, [email protected], [email protected], [email protected], [email protected], [email protected], tough-cookie
@0.12.1, [email protected], [email protected])
remote: +-- [email protected] ([email protected], [email protected])
remote: +-- [email protected] ([email protected], [email protected], [email protected], [email protected])
remote: +-- [email protected] ([email protected], [email protected], [email protected], [email protected], [email protected])
remote: +-- [email protected] ([email protected])
remote: Finished successfully.
remote: Deployment successful.
To https://[email protected]:443/nodejsex.git
   01717d1..ed0ca66  master -> master

[amitapl]

I've verified the problematic package is: bufferstream, it requires a build that currently fails.

[davidebbo]

Yes, this is a known limitation. It's documented here, in the 'Native Modules' section.

[koistya]

Azure Websites already have Python installed (optional), if they would also have installed Microsoft Visual Studio C++ 2013 for Windows Desktop (Express version works well), installing native modules would work I think.

[xdansmith]

Are there any plans to remove this limitation? MSBuild is present, so I don't see why the VC++ build tools couldn't be as well.

We're trying to move over to Azure Websites but it seems running node.js is a little trickier than we originally thought because of limitations like this.

[phairoh]

Every time that I want to love Azure I run in to issues like this and end up struggling for hours trying to figure out work arounds.

This seems like a very silly limitation as all it would take to fix it is installing the appropriate build tools. Is there any plan to fix this or are we left to figure out workarounds indefinitely or switch away from Azure to a competing cloud computing provider?

[DavidRouyer]

This is a blocker issue for a lot of people. It is almost impossible to build a decent workflow or a decent application without one or more native modules.

Nobody wants to put hundred on megabytes of node_modules dependencies inside their source control repository.
And how can we build a continuous deployment script if we can't put the "npm install" command inside our scripts ? (obviously npm install is failing due to node_gyp modules)

Please add VC++ build tools ! We don't want workarounds, we want a first class user experience when using Node.js on Azure !

[francesconero]

Yes, being able to build native modules would remove so many workarounds needed to deploy a complex node.js project!

[DavidRouyer]

Can we have an update on that ?

[davidebbo]

Sorry, there is no useful update at this time. We know it's a pain point, and we're still hoping there will be a solution in the future.

[DavidRouyer]

Is it not possible to install a VC++ compiler on the platform ?

[davidebbo]

There is more to it. It's not just a matter of installing one thing and having everything start to work. We did experiment with this at the time and there were several additional issues.

[davidebbo]

@francesconero @DavidRouyer Could you please share the name of specific native node modules that you are hitting this issue with?

Some that we already know about:

• socket.io
• bufferstream

[francesconero]

The ones I have problems with both come from the mongodb package (although one recently became an optional dependency):

• bson-ext
• kerberos

[DavidRouyer]

Actually, every module depending on node-gyp in my opinion. Your can find a list here.

Take dependencies of dependencies into account.
For example, karma and protractor need socket.io.

Database drivers (sqlite3, pg/ng-native, node-oracle, node-mssql (not sure), etc.) ...

[davidebbo]

@DavidRouyer we're looking for _specific_ ones that we can use for testing, and we'd want to prioritize the testing with modules that we know people are actually trying to use, so not the full list of everything that uses node-gyp. We'll look at the specific ones you mentioned, thanks!

[davidebbo]

Other question: if we had a solution that only worked with npm 2.11.x or later, would that be acceptable? Or are people still using the old one?

[francesconero]

I think it would be awesome! I'm sure there are still people using 2.x versions (me, for example), and the main problem I see is that peerDependencies were removed in npm 3.x.

But I mean, what better occasion to migrate the install than this? Also I think most projects would upgrade from npm 2.x to 3.x without even touching the package.json.

May I ask a question out of curiosity though? Does the npm3 requirement stem from the path length limitation?

[DavidRouyer]

I think npm 2.11.x or later is all good !
I think a lot of people have made the switch to node v4.
For people who wants stability, they can use the v4 LTS branch shipped with npm 2.14.x by default.
For people who wants to use the latest version of node, they can use the v4 branch with npm 2.14.x or npm 3. (The v5 branch is coming very soon and will be shipped with npm 3 by default).

[ahmelsayed]

The requirement comes from the supported versions of VS in node-gyp that shipped with npm in a given version. VS 2015 started being supported by node-gyp that shipped in npm 2.11.2 while VS 2013 support shipped with npm 1.3.21. Ideally we will support one version of the compiler, either 2013 or 2015, but we wanted to see if it's okay to only support newish npm releases (>= 2.11.2)

[francesconero]

@ahmelsayed makes sense, thanks! But why jump to npm 3.x, which introduces other unrelated breaking changes?

[ahmelsayed]

We just thought that 2015 was only supported in 3.x, but apparently it's supported starting from 2.11.2, David had it initially saying 3.x but has updated his comment since then to say 2.11 :)

[francesconero]

Of course! Sorry to have missed that. I wish they sent notifications for edits to comments you replied to :)

And by the way, thanks for your efforts to provide more and more features for us node.js users!

[davidebbo]

One more thing we could use from you: if you could share some simple repos that use those packages and are currently blocked by it, we would use that to validate that things work end to end. Ideally, it should be a simple repo such that:

• pushing it today results in the VCBuild.exe issue
• it should work fully if native modules can be successfully compiled

[francesconero]

@davidebbo should the project be a website? The native modules I use are used to interact with mongodb, so you would have to set up an instance before the project worked. It wouldn't be just a simple deploy unfortunately.

[davidebbo]

Another approach: email me (david.ebbo (AT) microsoft.com), and we can get you set up on a test site that has the fix, where you can try whatever you need.

[davidebbo]

The fix we made is deployed, and should allow npm native compilation to work. Some notes:

• There is a known perf issue that causes the compiler server process (mspdbsrv.exe) to hang for about a minute on first invocation.
• We've tested the following npm packages: https://github.com/KuduApps/NodeAppWithNativeNpmModules/blob/master/package.json.
• For a reason that is not fully understood yet, a subset of those packages (hiredis, bcrypt, libxmljs, fibers, node-expat, kerberos) fail to install on free/shared Web Apps, but install fine on dedicated Web Apps (i.e. basic/standard).
• Let us know if you run into an issue with a specific package.

[Benjiiim]

Hi @davidebbo,

We're having an issue with the following module which is not in the list of tested packages: lwip.
Repro and deployment logs can be found here: https://github.com/Benjiiim/azure-appservice-test

Any advice if this should work or if we need to install more dependencies through the custom script, would be greatly appreciated.

Thanks

[davidebbo]

@Benjiiim can you open a new issue? It will make it easier to track this one case instead of this long thread.