Js-ipfs: distribute js-ipfs binaries with pkg?

[mkg20001]

There is a really cool tool out there (https://github.com/zeit/pkg) that allows nodejs apps to be packaged as binaries.
This would have the following benefits:

• User would not have to download and install nodejs
• Faster installation (npm is slow)
• Newer nodejs version available

[dryajov]

That would be super cool, really like pkg.

[daviddias]

If we were to do this, it would enable jsipfs and goipfs to be tested more easily against the same sharness tests in some CI (//cc @VictorBjelkholm).

We could also make it available through dist.ipfs.io, although that would be a lesser priority.

[mkg20001]

I've tried it out (just run pkg .)

The cli has some missing commands if the binary is run outside of the source folder (all except add, cat, get, completion) and if a repo exists an error occurs because leveldown.node could not be found (pkg for now does not bundle .node files)

Kommandos:
  add <file>       Add a file to IPFS using the UnixFS data format
  cat <ipfs-path>  Fetch and cat an IPFS path referencing a file
  get <ipfs-path>  Fetch a file or directory with files references from an IPFS
                   Path
  completion       generate bash completion script

Optionen:
  --silent  Write no output                          [boolean] [Standard: false]
  --help    Hilfe anzeigen                                             [boolean]

/snapshot/js-ipfs/node_modules/levelup/lib/leveldown.js:31
    throw requireError(e)
    ^
LevelUPError: Failed to require LevelDOWN (Could not locate the bindings file. Tried:
 → /snapshot/js-ipfs/node_modules/leveldown/build/leveldown.node
 → /snapshot/js-ipfs/node_modules/leveldown/build/Debug/leveldown.node
 → /snapshot/js-ipfs/node_modules/leveldown/build/Release/leveldown.node
 → /snapshot/js-ipfs/node_modules/leveldown/out/Debug/leveldown.node
 → /snapshot/js-ipfs/node_modules/leveldown/Debug/leveldown.node
 → /snapshot/js-ipfs/node_modules/leveldown/out/Release/leveldown.node
 → /snapshot/js-ipfs/node_modules/leveldown/Release/leveldown.node
 → /snapshot/js-ipfs/node_modules/leveldown/build/default/leveldown.node
 → /snapshot/js-ipfs/node_modules/leveldown/compiled/8.0.0/linux/x64/leveldown.node). Try `npm install leveldown` if it's missing
    at requireError (/snapshot/js-ipfs/node_modules/levelup/lib/leveldown.js:37:10)
    at getLevelDOWN (/snapshot/js-ipfs/node_modules/levelup/lib/leveldown.js:31:11)
    at LevelUP.open (/snapshot/js-ipfs/node_modules/levelup/lib/levelup.js:112:34)
    at new LevelUP (/snapshot/js-ipfs/node_modules/levelup/lib/levelup.js:84:8)
    at LevelUP (/snapshot/js-ipfs/node_modules/levelup/lib/levelup.js:45:44)
    at new LevelDatastore (/snapshot/js-ipfs/node_modules/datastore-level/src/index.js:27:15)
    at Object.createBackend [as create] (/snapshot/js-ipfs/node_modules/ipfs-repo/src/backends.js:6:10)
    at __dirname.open.waterfall (/snapshot/js-ipfs/node_modules/ipfs-repo/src/index.js:95:35)
    at nextTask (/snapshot/js-ipfs/node_modules/async/waterfall.js:16:14)
    at next (/snapshot/js-ipfs/node_modules/async/waterfall.js:23:9)

[daviddias]

Oh, the native dependencies!

[victorb]

Yeah, native dependencies + dynamically loading the CLI commands would put a stop on using pkg... Not super valuable for CI though, more for having a nodejs runtime embedded so it's not necessary to install.

[mkg20001]

I've got at least the cli to work. Now all commands appear properly in the help page.

diff --git a/package.json b/package.json
index 778a7b3..9675821 100644
--- a/package.json
+++ b/package.json
@@ -6,6 +6,9 @@
     "jsipfs": "src/cli/bin.js"
   },
   "main": "src/core/index.js",
+  "pkg": {
+    "scripts": "src/cli/commands/**/*.js"
+  },
   "browser": {
     "./src/core/components/init-assets.js": false,
     "./src/core/runtime/config-nodejs.json": "./src/core/runtime/config-browser.json",
@@ -80,6 +83,7 @@
     "mocha": "^3.5.0",
     "ncp": "^2.0.0",
     "nexpect": "^0.5.0",
+    "pkg": "^4.2.4",
     "pre-commit": "^1.2.2",
     "pretty-bytes": "^4.0.2",
     "qs": "^6.5.0",

[mkg20001]

The native dependencies issue can be solved by putting the *.node files in the same folder as the ipfs binary and patching the modules so they load it from that location.
Edit: Or even better: The *.node files could be renamed and embedded into the executable and then temporarily written somewhere.

[mkg20001]

I've got it working. I've created this module that can embed the natives in the binary! Pr coming soon... (first I have to cleanup that messy module)

[mkg20001]

PR #1007

[daviddias]

@mkg20001 thank you for being awesome and proposing an idea and solving the issues to make it happen. I'll close this issue to avoid fragment the discussion and focus on reviewing the PR and idea here https://github.com/ipfs/js-ipfs/pull/1007

I won't make a call on whether or not to support this but I do hope that by the beginning of October there is clarity. Thanks!