Colabtools: Add support for ijavascript (nodejs)

[briancullinan]

Looking at your instructions on colab "Snippets: Importing libraries" (very helpful, thank you!). Here's what I tried:

!npm init -y
!npm install ijavascript zeromq

But it seems there are access permissions:

?25h
> [email protected] install /content/node_modules/zeromq
> node scripts/prebuild-install.js || (node scripts/preinstall.js && node-gyp rebuild)


/bin/sh: 1: prebuild-install: Permission denied

Building libzmq for linux
internal/child_process.js:313
    throw errnoException(err, 'spawn');
    ^

Error: spawn EACCES
    at exports._errnoException (util.js:1022:11)
    at ChildProcess.spawn (internal/child_process.js:313:11)
    at exports.spawn (child_process.js:380:9)
    at buildZMQ (/content/node_modules/zeromq/scripts/preinstall.js:17:15)
    at IncomingMessage.<anonymous> (/content/node_modules/zeromq/scripts/preinstall.js:106:5)
    at emitNone (events.js:91:20)
    at IncomingMessage.emit (events.js:185:7)
    at endReadableNT (_stream_readable.js:974:12)
    at _combinedTickCallback (internal/process/next_tick.js:74:11)
    at process._tickCallback (internal/process/next_tick.js:98:9)
?25hnpm WARN [email protected] No description
npm WARN [email protected] No repository field.
npm ERR! Linux 4.4.86+
npm ERR! argv "/tools/node/bin/node" "/tools/node/bin/npm" "install" "ijavascript" "zeromq"
npm ERR! node v6.10.0
npm ERR! npm  v3.10.10
npm ERR! code ELIFECYCLE

npm ERR! [email protected] install: `node scripts/prebuild-install.js || (node scripts/preinstall.js && node-gyp rebuild)`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] install script 'node scripts/prebuild-install.js || (node scripts/preinstall.js && node-gyp rebuild)'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the zeromq package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node scripts/prebuild-install.js || (node scripts/preinstall.js && node-gyp rebuild)
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs zeromq
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls zeromq
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /content/npm-debug.log

Any chance someone more qualified on the colab environment could add a NodeJS option to "Change runtime type" ? It looks like node is already installed and used in some capacity, I will keep trying.

[blois]

The error you're seeing appears to be https://github.com/creationix/nvm/issues/1407, that npm install is getting a permissions error when installing as 'nobody'.

A workaround is to use --unsafe-perm:

!npm install -g --unsafe-perm ijavascript zeromq

Then:

!ijsinstall

And validate it:

!jupyter-kernelspec list

Then the next step would be to generate a notebook with a kernelspec with a name of 'javascript':

"kernelspec": {
  "name": "javascript",
  "display_name": "Javascript"
}

And then... Colab will fail to connect to the kernel because it tries to do some initialization in Python. But that's certainly much closer, and may be something we can tweak.

[briancullinan]

@blois Nice! Thank you! That does look closer. I have a bit of experience with the kernel spec, maybe from that I can figure it out.

[blois]

Modifying the kernelspec is just manually tweaking the notebook JSON (just File->Download .ipynb, edit, then File -> Upload notebook).

The last bit requires a change in the Javascript for Colab. We may be able to tweak something here, but be warned that this is definitely off the supported path!

[briancullinan]

Played around with it more. Got it to recognize ijavascript in the "global" jupyter path. Opened the javascript notebook and the kernel says "Could not connect to kernel".

Is Colab primarily interested in supporting the PySci? I love node and powershell for DevOps. This is why I bring it up, using this notebook has been excellent in developing algorithms and scripts, and using Colab would make setting up the environment so much easier.

[blois]

Colab's initial focus is on Python-based data analysis though we see a number of requests for R support as well.

At Google we do make heavy use of Colab in the DevOps workflows, ours generally use Python. It is indeed quite invaluable in that regard.

I just submitted a change which should get past this next hurdle (will take a week or so to be live since we're not doing releases over the holidays). It looks like it should be straight-forward to get something working, though there is quite a bit of work to get something polished. The general interest level in ijavascript support would dictate how much we can invest in it.

[briancullinan]

Woohoo! You're the best!

On Thu, Dec 28, 2017 at 2:09 PM, blois [e-mail] wrote:

Colab's initial focus is on Python-based data analysis though we see a
number of requests for R support as well.

At Google we do make heavy use of Colab in the DevOps workflows, ours
generally use Python. It is indeed quite invaluable in that regard.

I just submitted a change which should get past this next hurdle (will
take a week or so to be live since we're not doing releases over the
holidays). It looks like it should be straight-forward to get something
working, though there is quite a bit of work to get something polished. The
general interest level in ijavascript support would dictate how much we can
invest in it.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/googlecolab/colabtools/issues/13#issuecomment-354358304,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AX5Xbpjb3jbFLjWaQWsbC7ZhDCyMSi3uks5tFAOMgaJpZM4ROfYx
.

--
"I studied engineering"

CONFIDENTIALITY NOTICE: The contents of this email message and any
attachments are intended solely for the addressee(s) and may contain
confidential and/or privileged information and may be legally protected
from disclosure. It is then shared with tech companies, bots, hackers,
government agencies, and marketers. The security of this message is none,
and it may be shared on Instagram at anytime. If you are OK with this,
please respond. There isn't really any security or privacy anywhere. If
you disagree you may want to go camping and talk to people face-to-face
like in old times.

[blois]

Connections to ijavascript kernels should now be working, if you have already installed it.

Again- keep in mind that you'll need to install the ijavascript kernel before attempting to open an ijavascript notebook.

[blois]

Example installation: https://colab.research.google.com/notebook#fileId=1bP_ZSdWAP98PWyOcuXqXB9EMseVKpTOM

Example notebook (if you open this before installing then it will connect to a Python kernel):
https://colab.research.google.com/notebook#fileId=1Bj9AM4spC26gtxZGA2ybaCQON1vImAsZ

[briancullinan]

I don't see the material components used for language selection in this
repository. Google modified the menu system from jupyter right? Would it
be possible to add NodeJS to the kernel language menu too?

[briancullinan]

Took me a minute to figure out, after running the install from python:

!npm install -g --unsafe-perm ijavascript zeromq
!ijsinstall --install=global
!jupyter-kernelspec list

I had to delete a notebook I opened previously and copy it back to get it to connect to the right instance. It still says attached to "Python 2 Google Compute Engine" even when attached to javascript. These two side effects I am happy to live with.

This appears to work great for my use case, thank you!

[guy1ziv2]

I ran the above installation and the example - both worked. I dont understand however how to make a new Colab using the javascript kernel ? the Runtime/change runtime type does not list javascript as an option

[sabino]

@guy1ziv2 just set the kernel within the .ipynb file

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "ijavascript.ipynb",
      "version": "0.3.2",
      "provenance": []
    },
    "kernelspec": {
      "name": "javascript",
      "display_name": "Javascript"
    }
  },
  "cells": [
    {
      "metadata": {
        "id": "NDpbv83iNhLp",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "be610b97-49dc-4b09-fbfe-9c28157f944c"
      },
      "cell_type": "code",
      "source": [
        "console.log('here!')"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "here!\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}

[sabino]

BTW I've managed to do the same with R, not sure if this is an optimal way of installing but it works.

Example installation:
https://colab.research.google.com/drive/1P9PvbeXsh50YzGzdhEj3MN43Aa7jrvvn

Example notebook:
https://colab.research.google.com/drive/1NVwZvup2_Me-mYLACfdY1czvW9yehuBL

[blois]

FWIW, you can have a Python cell at the beginning of the notebook which does the installation then kills the current kernel. On reconnect, the R portion will be picked up.

For instance, to upgrade your Python kernel:

!pip install -q --upgrade ipython
!pip install -q --upgrade ipykernel

# Restart with new IPython.
import os
import signal
os.kill(os.getpid(), signal.SIGTERM)

[kaue]

@blois any chance we can add a option to "Change runtime type" to set the kernelspec to javascript ?

"kernelspec": {
      "name": "javascript",
      "display_name": "Javascript"
    }

[jacobfvanzyl92]

I ran the above installation and the example - both worked. I dont understand however how to make a new Colab using the javascript kernel ? the Runtime/change runtime type does not list javascript as an option

Reviving an ageing thread here - but I would just like to confirm if/how its possible to set the runtime to always be javascript ? The scenario I'm investigating is to use Colab for my team to spec our platform development, which is mostly node.js - Vue - cordova.

I've got it working perfectly now for myself using the instructions above, but I guess my real question is: Will my team members (when opening the .ipynb file from the GDrive which we share) also drop into a JS runtime ?

[jacobfvanzyl92]

... in other words: When opening the file, will they hit the same instance as I've installed ijavascript on ?

[craigcitro]

... in other words: When opening the file, will they hit the same instance as I've installed ijavascript on ?

No, distinct users always get distinct runtimes. (In addition, once your runtime is collected after being idle, you'll need to reinstall the ijavascript toolchain.)

[korakot]

@blois Can you make this code in default installation?

!npm install -g --unsafe-perm ijavascript zeromq
!ijsinstall --install=global

So that we don't have to switch to python notebook to install ijavascript.
This is already done with IRkernel and Swift already. Why not javascript?
I believe a few people will want to use TensorFlow + Node.js too.
So, it should benefit the promotion of TensorFlow as well.

[blois]

I'm curious- are there examples of use-cases for ijavascript? I know that it _works_, but I'm really interested in how it's being used. Is there a mybinder-capable docker image available somewhere? I don't see one in the default stacks at https://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html.

Each addition incurs a bit of overhead that we're cautious about undertaking- there was a long discussion preceding the addition of R and it's still not an advertised feature.

[korakot]

I haven't used it myself. But as I understand it. The rationale for Google to provide Colab free for everyone is to promote AI + TensorFlow. So, whichever languages that TensorFlow support, Colab should support as well. All the TensorFlow tutorials should run smoothly without installation. The less barrier the better.

If TensorFlow is going to support Node.js so should Colab.
And TensorFlow.js is going to be a big part of TF deployment, all the more reason to support JavaScript both serverside and clientside. Some code could be shared on both sides, I think.

BTW, thanks a lot for the R support. There are a few cases where I want to call R from Python.
Now I can use rpy2 and %%R to pass data from python to an R library I like (e.g. wordcloud, ggplot)

[blois]

Also discussed in https://github.com/tensorflow/tfjs/issues/268

[jasonhargrove]

Docs for importing modules into your JavaScript Notebook once setup

Working on an iPad Pro today and managed to get this going. Will describe my whole flow since the iPad required a work around which may be useful to someone.

Installing ijavascript

Just as shown above, but with one difference - note the familiar-log module I installed, which is to be used in the final step.

!npm init -y
!npm install ijavascript -g zeromq familiar-log --unsafe-perm
!ijsinstall
!jupyter-kernelspec list
!npm root -g

Modifying your .ipynb file without a computer

My solution was to save the notebook to GitHub Gists, edit the file there to set the kernel as described above, save it, then tap the Open in Colab button to send it back.

Importing npm modules into your script

See in step 1 where I installed the familiar-log module. Note the !npm root -g.

After confirming that JS works in my new notebook console.log(‘hello world’) I imported the module by using the value output by the root command. In my case: /tools/node/lib/node_modules. So instead of require(‘familiar-log’) I specified the location.

var log = require('/tools/node/lib/node_modules/familiar-log').log

log('hello');

log('hello').json({ world: true });

[obenjiro]

I made a simple to bootstrap template that allows you to run TensroflowJS on Google Colab, install modules in notebook, and run long running async code

https://dev.to/obenjiro/silence-of-the-fans-part-1-javascript-quickstart-5f3m

[korakot]

@obenjiro Can you make it support typescript as well?

Does your example still work if I change ijavascript to tslab?

https://github.com/yunabe/tslab