Node-gyp: How to install the source when there is no network

[lihy87]

the target machine(ubuntu 16.04) has no network, how do i install it in source?

[bnoordhuis]

Copy over the source tarball from nodejs.org and pass --tarball <location> to node-gyp or npm.

[lihy87]

Could you write the detailed steps?

[bnoordhuis]

That's really all there is to it. If anything specific is unclear, let me know and I'll try to clarify.

[gibfahn]

the target machine(ubuntu 16.04) has no network, how do i install it in source?

Are you trying to install node-gyp or node itself? Do you just want to npm install native modules?

If you've installed node and npm you already have node-gyp installed (it's a dependency of npm), you just need the headers.

Assuming you're using node v6.10.1:

Headers are stored in ~/.node-gyp/6.10.1/, you also need the installVersion file. In later versions of node-gyp these files could change, check the contents of ~/.node-gyp on a machine with internet access when you run npm i -g node-gyp && node-gyp configure and duplicate it.

curl -O https://nodejs.org/dist/v6.10.1/node-v6.10.1-headers.tar.gz # on a computer with internet
# Copy header tarball to offline machine
mkdir -p  ~/.node-gyp/6.10.1
tar -xf node-v6.10.1-headers.tar.gz --directory ~/.node-gyp/6.10.1/ --strip-components 1
echo 9 >~/.node-gyp/6.10.1/installVersion

This should be enough for the common use case, if you need something more specific please provide some actual information about what you want to do.

[bdharrington7]

Not sure if this is related or not, but I have a dependency (better-sqlite3) that has a compilation step that uses node-gyp to compile. For reasons I don't quite understand, the header files are downloaded when this compilation phase is kicked off for the first time. When this module is compiled, a folder is created at $HOME/.node-gyp that contains the header files that are downloaded, and includes a installVersion file containing the number 9 only.

When I run npm install -g node-gyp && node-gyp ./tarball.tar.gz, this folder doesn't show up at all. I've also tried npm install --tarball ./tarball.tar.gz and node-gyp --tarball ./tarball.tar.gz

node v8.1.2
npm v5.0.3

[bnoordhuis]

@bdharrington7 The tarball that node-gyp downloads contains the node.js SDK (this file), it's not related to the tarball that npm downloads from its registry when you install a package.

If you download the SDK tarball manually, invoke node-gyp like this from inside the directory that contains the add-on you want to build (with a binding.gyp in it):

$ node-gyp --tarball /path/to/node-x.y.z-headers.tar.gz rebuild

[bdharrington7]

The issue I'm trying to solve for is that the directory that contains the add-on I want to build doesn't exist at the time I want to install the headers. I'm trying to preemptively provide these headers in the expected location so that when npm install is run, it doesn't try to go out and fetch the headers.

A little more insight on the problem: I'm trying to create a development environment in Docker that works offline. As such, since the npm install layer is re-built when something in package.json changes, the installation of the node-gyp headers is no longer present in the expected location, and is subsequently re-downloaded.

[bnoordhuis]

node-gyp install will do that for you; node-gyp install --ensure to only download them when not already present.

[gibfahn]

node-gyp install will do that for you; node-gyp install --ensure to only download them when not already present.

--ensure is on by default unless you're using --tarball IIRC

_EDIT:_ lib/configure.js#L77

[abhagupta]

@bdharrington7 did you get past this problem in your docker container?

[bdharrington7]

Yeah, I download and place the header files like this, but you’ll have to figure out where to get the binary, I’m getting them from an internal repo.

ENV NODE_VERSION v8.9.4
ENV NODE_FILENAME node-${NODE_VERSION}-linux-x64.tar.gz
ENV NODE_URL https://${SOMEREPO}.com/node-distributions/${NODE_VERSION}/${NODE_FILENAME}

RUN wget -nv "${NODE_URL}" -O "/tmp/${NODE_FILENAME}" && \
    tar xf "/tmp/${NODE_FILENAME}" -C /usr/local/ --strip=1 && \
    rm "/tmp/${NODE_FILENAME}"

[abhagupta]

How did you do the npm install to make sure your dependencies are loading node-gyp without needing internet connection?

[bdharrington7]

I store my dependencies in source control (the non-compiled ones) so I turn off WiFi and run npm rebuild

[nickboldt]

Simpler solution in https://github.com/nodejs/node-gyp/issues/1133#issuecomment-538767799:

NODE_VER=v10.14.1
curl -sSL https://nodejs.org/download/release/${NODE_VER}/node-${NODE_VER}-headers.tar.gz \
        -o /tmp/node-headers.tgz; \
npm config set tarball /tmp/node-headers.tgz