Jupyter_contrib_nbextensions: can't install on jupyter 4.1 system python debian sid

[epifanio]

No matter what I try the nbextensions folder is never found. (404)
I installed everytihing using PIP.
I also tried to remove all the ipytoh/jupyter related files, anf then a clean installation of just the jupter notebook ... and in the js log I was still having thew (404) error for a missing nbextensions directory.
Can yuo help to debug this?
https://gist.github.com/4fac50855c12ada0a6983ad2d8ff4ec1

[epifanio]

maybe useful to debug, when i try to run the single user jupyter notebook:

the link: http://myhost:9000/nbextensions in the js console gives:

require.js?v=6da8be3…:900 TypeError: $ is not a function(…)Module.check @ require.js?v=6da8be3…:900
require.js?v=6da8be3…:900 TypeError: Cannot read property 'extend' of undefined(…)Module.check @ require.js?v=6da8be3…:900
require.js?v=6da8be3…:900 TypeError: $ is not a function(…)Module.check @ require.js?v=6da8be3…:900
require.js?v=6da8be3…:900 TypeError: Cannot read property 'build_page' of undefined(…)

and in the bash shell from where the process is running, i have:

404 GET /static/components/underscore/underscore-min.map (50.187.182.220) 28.79ms referer=None

In the browser a blank page with entitled _Notebook Extension Config_ is displayed.

The code below:

import os
import glob

from jupyter_core.paths import jupyter_config_dir, jupyter_data_dir

pats = {
    'config': os.path.join(jupyter_config_dir(), '*'),
    'extensions': os.path.join(jupyter_data_dir(), 'extensions', '*'),
    'nbextensions': os.path.join(jupyter_data_dir(), 'nbextensions', '*'),
}
for name, pat in pats.items():
    print('{}:\n  {}'.format(name, '\n  '.join(glob.glob(pat))))

gives me:

config:
  /home/epinux/.jupyter/jupyter_nbconvert_config.py
  /home/epinux/.jupyter/migrated
  /home/epinux/.jupyter/jupyter_nbconvert_config.json
  /home/epinux/.jupyter/jupyter_notebook_config.py
  /home/epinux/.jupyter/nbconfig
  /home/epinux/.jupyter/jupyter_notebook_config.json
nbextensions:
  /home/epinux/.local/share/jupyter/nbextensions/codemirrormode
  /home/epinux/.local/share/jupyter/nbextensions/gist.js
  /home/epinux/.local/share/jupyter/nbextensions/config
  /home/epinux/.local/share/jupyter/nbextensions/jupyter-js-widgets
  /home/epinux/.local/share/jupyter/nbextensions/jupyter-leaflet
  /home/epinux/.local/share/jupyter/nbextensions/usability
  /home/epinux/.local/share/jupyter/nbextensions/ipyparallel
  /home/epinux/.local/share/jupyter/nbextensions/styling
  /home/epinux/.local/share/jupyter/nbextensions/slidemode
  /home/epinux/.local/share/jupyter/nbextensions/testing
  /home/epinux/.local/share/jupyter/nbextensions/publishing
extensions:
  /home/epinux/.local/share/jupyter/extensions/pre_pymarkdown.py
  /home/epinux/.local/share/jupyter/extensions/pre_svg2pdf.py
  /home/epinux/.local/share/jupyter/extensions/pre_codefolding.py
  /home/epinux/.local/share/jupyter/extensions/highlight_html_cfg.py
  /home/epinux/.local/share/jupyter/extensions/strip_output_prompt.py
  /home/epinux/.local/share/jupyter/extensions/highlight_latex_cfg.py
  /home/epinux/.local/share/jupyter/extensions/pre_collapsible_headings.py
  /home/epinux/.local/share/jupyter/extensions/nbextensions.py
  /home/epinux/.local/share/jupyter/extensions/__pycache__
  /home/epinux/.local/share/jupyter/extensions/pp_highlighter.py
  /home/epinux/.local/share/jupyter/extensions/post_embedhtml.py

/home/epinux/.jupyter/nbconfig/ edit.json

{
  "load_extensions": {
    "jupyter_cms/editor/main": false
  }
}

/home/epinux/.jupyter/nbconfig/ notebook.json

{
  "load_extensions": {
    "codefolding": true,
    "jupyter-leaflet/extension": true,
    "jupyter_cms/notebook/main": false,
    "jupyter-js-widgets/extension": true
  }
}

/home/epinux/.jupyter/nbconfig/ tree.json

{
  "load_extensions": {
    "jupyter_cms/dashboard/main": false,
    "ipyparallel/main": true
  }
}

/home/epinux/.jupyter/nbconfig/ usability.json

{
  "load_extensions": {
    "codefolding": true
  }
}

/home/epinux/.jupyter/ / jupyter_nbconvert_config.json

{
  "Exporter": {
    "preprocessors": [
      "pre_codefolding.CodeFoldingPreprocessor",
      "pre_pymarkdown.PyMarkdownPreprocessor"
    ],
    "template_path": [
      ".",
      "/home/epinux/.local/share//jupyter/templates"
    ]
  },
  "NbConvertApp": {
    "postprocessor_class": "post_embedhtml.EmbedPostProcessor"
  },
  "version": 1
}

/home/epinux/.jupyter/ / jupyter_nbconvert_config.py

#--- nbextensions configuration ---
import sys
sys.path.append('/home/epinux/.local/share/jupyter/extensions')
#--- nbextensions configuration ---

/home/epinux/.jupyter/ / jupyter_notebook_config.json

{
  "NotebookApp": {
    "extra_template_paths": [
      "/home/epinux/.local/share/jupyter/templates"
    ],
    "nbserver_extensions": {
      "nbextensions": true
    }
  },
  "version": 1
}

/home/epinux/.jupyter/ / jupyter_notebook_config.py

#--- nbextensions configuration ---
import sys
sys.path.append('/home/epinux/.local/share/jupyter/extensions')
#--- nbextensions configuration ---

/home/epinux/.jupyter/ / migrated

2016-06-04T22:39:39.713878

ls /home/epinux/.local/share/jupyter/extensions

highlight_html_cfg.py   post_embedhtml.py   pre_collapsible_headings.py  __pycache__
highlight_latex_cfg.py  pp_highlighter.py   pre_pymarkdown.py            strip_output_prompt.py
nbextensions.py         pre_codefolding.py  pre_svg2pdf.py

ls /home/epinux/.local/share/jupyter/nbextensions

codemirrormode  gist.js      jupyter-js-widgets  publishing  styling  usability
config          ipyparallel  jupyter-leaflet     slidemode   testing

ls /home/epinux/.local/share/jupyter/templates/

highlighter.tpl   nbextensions.html  nbextensions.tplx    rendermd.html
highlighter.tplx  nbextensions.tpl   printviewlatex.tplx

[epifanio]

The nbextension and extension are also found in: /root/.jupyter/

[epifanio]

While this:

import notebook
notebook.nbextensions.check_nbextension('usability/codefolding', user=True)
notebook.nbextensions.check_nbextension('usability/codefolding/main.js', user=True)

returns True as expected.

This:

import notebook
E = notebook.nbextensions.EnableNBExtensionApp()
E.enable_nbextension('usability/codefolding/main')

Returns:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-2-79dd84cd75e5> in <module>()
      1 import notebook
      2 E = notebook.nbextensions.EnableNBExtensionApp()
----> 3 E.enable_nbextension('usability/codefolding/main')

AttributeError: 'EnableNBExtensionApp' object has no attribute 'enable_nbextension'

[epifanio]

After a long debug I think I fixed, most of the isse. I'll clode this and open a new one for the extensions which are not running/loadig on the jupyterhub

[jcb91]

apologies for late reply, just seen this. I guess this was partly down to notebook versioning issues with your enable call, and partly down to the fact that our current install scripts only ever put files in jupyter --data-dir...

[jcb91]

and what does

import notebook
print(notebook.version_info)

give?

[epifanio]

Hi, I just cleaned up all my config, reinstalled python using conda and reinstalled the extension. I was finally able to load the configuration page using jupyter-notebook.
I then installed jupyterhub (my target is to have the extensions working for all the users) .
Running the hub as normal user, I can access to the nbextensions as I do with the jupyter-notebook (single user).
The problem arise when the the hub is started via sudo in that case the nbextensions page resultgs in a blank page.

[epifanio]

import notebook
print(notebook.version_info)
(5, 0, 0, '.dev')

[jcb91]

ah, ok. So, the fact that you get a blank page and not a 404 implies that the server responds, but doesn't manage to find the appropriate javascript

[jcb91]

Ah, 5.0! There be dragons (maybe)

[epifanio]

yes! unfortunately I'm using PAM authentication method and sudo is required to run a multi user server

[jcb91]

So, currently, all of our javascript gubbins gets installed into the per-user jupyter --data-dir. So, to get round this, you can:

• install this repo using sudo, to (hopefully) get the same jupyter --data-dir that the server will search
• set environment variable JUPYTER_DATA_DIR before installing, in order to install to a custom location, which you can set to one of the locations in the jupyter path that the server searches

[jcb91]

You can list the nbextensions directories which the server will search using

import os.path
from jupyter_core.paths import jupyter_path
for base in jupyter_path():
    print(os.path.join(base, 'nbextensions'))

(since your server is running as super-user, you could run this python script as super-user to get appropriate results, or alternatively, use the system jupyter path defined at jupyter_core/paths.py#L97-L107)

[jcb91]

Presumably, you should be able to get js installed into an appropriate location using

JUPYTER_DATA_DIR='/usr/share/jupyter'
pip install https://github.com/ipython-contrib/IPython-notebook-extensions/archive/master.zip

since jupyter_core.paths used in our install script checks the JUPYTER_DATA_DIR envrionment variable. This should all get easier once we switch to a proper pip package (see my master branch for details, if you're interested)

[epifanio]

This is what I use in the jupyterhub configuration file:

c = get_config()

os.environ['JUPYTER_DATA_DIR'] = '/usr/local/share/jupyter/'

When I try to install the nbextensions using sudo (or root) it tries to install in /root/.jupyter

root@Debian-70-wheezy-64-minimal:/home/epinux/dev/IPython-notebook-extensions# export JUPYTER_DATA_DIR=/usr/local/share/jupyter/
root@Debian-70-wheezy-64-minimal:/home/epinux/dev/IPython-notebook-extensions# python setup.py install
Running source install...
Extensions and templates path: /usr/local/share/jupyter/
Installing Jupyter notebook extensions.
Done!
Configuring extensions...
Configuring the Jupyter notebook extensions.
Configuration files directory: /root/.jupyter
Done!
..
...

From the hub running under normal user I have:

/usr/local/share/jupyter/nbextensions
/home/epinux/anaconda3/share/jupyter/nbextensions
/usr/local/share/jupyter/nbextensions
/usr/share/jupyter/nbextensions

and from the Hub running under sudo:

/usr/local/share/jupyter/nbextensions
/usr/local/share/jupyter/nbextensions
/usr/share/jupyter/nbextensions

When running with sudo I got the blank page with the foillowing js log in the console:

require.js?v=6da8be3…:900 TypeError: $ is not a function(…)Module.check @ require.js?v=6da8be3…:900
require.js?v=6da8be3…:900 TypeError: Cannot read property 'extend' of undefined(…)Module.check @ require.js?v=6da8be3…:900
require.js?v=6da8be3…:900 TypeError: $ is not a function(…)Module.check @ require.js?v=6da8be3…:900
require.js?v=6da8be3…:900 TypeError: Cannot read property 'build_page' of undefined(…)Module.check @ require.js?v=6da8be3…:900

[epifanio]

when installing with sudo the data goes in the right place /usr/local/share/jupyter while the config goes in :

 Configuration files directory: /root/.jupyter

[jcb91]

Ah, ok, so the console errors about $ mean that this is something
different to what I was thinking.

I thought that the server was just throwing back a 404 for the script
required to load the nbextensions config page, but it seems that actually
it's more complicated than that.

It seems as though something in the page is breaking related to a jquery
requirement.

I have a feeling that this may actually be down to some change in how
jquery is loaded in notebook 5.0 - would it be possible to try running with
4.x?

If you could run the server with the --debug flag, we might see in the
server logs what's getting requested/404'ed, although it seems that may
have been a red herring. I'm going to sleep now, but will try to check this
in the morning...
On 6 Jun 2016 2:58 a.m., "epifanio" [email protected] wrote:

when installing with sudo the data goes in the right place, while the
config goes in :

Configuration files directory: /root/.jupyter

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/ipython-contrib/IPython-notebook-extensions/issues/617#issuecomment-223853687,
or mute the thread
https://github.com/notifications/unsubscribe/AFXq9kFQ4CTuEt0S6lx-HI-Y7Y_jXaKWks5qI37dgaJpZM4IuPpI
.

[epifanio]

Thanks for the help! I'll try a downgrade and see if changes (but I remember 4.x was giving me trouble as well, I upgraded to 5.0 today. I open this issue running 4.1)
I'm using master brach from your git repo, If you want debug this on a specific branch I can switch to it.

For what I saw I guess the issue is related to the config files not the widgets data, I run the hub without sudo I have access to the configuration page (but can't run multiuser) I enabled some widget for just my user. Then I stop the hub, re-run with sudo and the widgets I enabled before are still there and working properly. Is just the configuration-page which doesn't load.
I'll enable debug and post the log (I actually installed the hub with pip install -e. from the git repo, I should be able to open a shell and see the js logs as well.)
I'll post more tomorrow, Thanks!

[epifanio]

When I refersh the browser page, pointing to the NBEXTENSIONS configuration page ( which is rendering a blank page) I saw this log in the jupyter shell outup:

[W 2016-06-06 04:34:19.345 epinux log:47] 404 GET /user/epinux/static/components/underscore/underscore-min.map (132.177.103.226) 1.91ms referer=None

[epifanio]

More from the js console:

require.js:900 TypeError: Cannot read property 'extend' of undefined
    at http://epinux.com:8888/user/epinux/static/base/js/keyboard.js?v=20160606043728:65:10
    at Object.context.execCb (http://epinux.com:8888/user/epinux/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce:1690:33)
    at Object.Module.check (http://epinux.com:8888/user/epinux/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce:865:51)
    at Object.<anonymous> (http://epinux.com:8888/user/epinux/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce:1140:34)
    at http://epinux.com:8888/user/epinux/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce:131:23
    at http://epinux.com:8888/user/epinux/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce:1190:21
    at each (http://epinux.com:8888/user/epinux/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce:56:31)
    at Object.Module.emit (http://epinux.com:8888/user/epinux/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce:1189:17)
    at Object.Module.check (http://epinux.com:8888/user/epinux/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce:940:30)
    at Object.Module.enable (http://epinux.com:8888/user/epinux/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce:1177:22)Module.check @ require.js:900

require.js:900 TypeError: $ is not a function
    at Object.get_body_data (http://epinux.com:8888/user/epinux/static/base/js/utils.js?v=20160606043728:521:19)
    at http://epinux.com:8888/user/epinux/nbextensions/config/main.js?v=20160606043728:31:26
    at Object.context.execCb (http://epinux.com:8888/user/epinux/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce:1690:33)
    at Object.Module.check (http://epinux.com:8888/user/epinux/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce:865:51)
    at Object.<anonymous> (http://epinux.com:8888/user/epinux/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce:1140:34)
    at http://epinux.com:8888/user/epinux/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce:131:23
    at http://epinux.com:8888/user/epinux/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce:1190:21
    at each (http://epinux.com:8888/user/epinux/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce:56:31)
    at Object.Module.emit (http://epinux.com:8888/user/epinux/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce:1189:17)
    at Object.Module.check (http://epinux.com:8888/user/epinux/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce:940:30)Module.check @ require.js:900

require.js:900 TypeError: Cannot read property 'build_page' of undefined
    at http://epinux.com:8888/user/epinux/nbextensions/:204:23
    at Object.context.execCb (http://epinux.com:8888/user/epinux/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce:1690:33)
    at Object.Module.check (http://epinux.com:8888/user/epinux/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce:865:51)
    at Object.<anonymous> (http://epinux.com:8888/user/epinux/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce:1140:34)
    at http://epinux.com:8888/user/epinux/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce:131:23
    at http://epinux.com:8888/user/epinux/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce:1190:21
    at each (http://epinux.com:8888/user/epinux/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce:56:31)
    at Object.Module.emit (http://epinux.com:8888/user/epinux/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce:1189:17)
    at Object.Module.check (http://epinux.com:8888/user/epinux/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce:940:30)
    at Object.<anonymous> (http://epinux.com:8888/user/epinux/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce:1140:34)Module.check @ require.js:900

[jcb91]

For what I saw I guess the issue is related to the config files not the widgets data

Apologies, yes, from your description (works as regular user, not as super), I think you're right. I was clearly getting a little over-tired 😝

So, there seem to be a few errors relating to missing javascript files, but they all seem to relate to components of notebook, rather than anything from this repo, so I'm a bit mystified.

The server console error is just a sourcemap for the minified version of the underscore library missing from somewhere, so probably not a big deal that it's missing, as the minified version should still be available.

The set of javascript console errors seem to be caused by missing notebook components:

require.js:900 TypeError: Cannot read property 'extend' of undefined
    at http://epinux.com:8888/user/epinux/static/base/js/keyboard.js?v=20160606043728:65:10

seems to result from a missing/undefined jquery in notebook/static/base/js/keyboard.js#L62-64

require.js:900 TypeError: $ is not a function
    at Object.get_body_data (http://epinux.com:8888/user/epinux/static/base/js/utils.js?v=20160606043728:521:19)

seems to be another missing jquery ($), I think from notebook/static/base/js/utils.js#L581.

Finally,

require.js:900 TypeError: Cannot read property 'build_page' of undefined
    at http://epinux.com:8888/user/epinux/nbextensions/:204:23

is just a result of the previous two errors, which mean the nbextensions configurtor-page javascript didn't define a proper module because it threw errors on missing required notebook components (utils and keyboard).

I'm still none the wiser why they're missing a jquery.

Could you try in a (multi-user) notebook window running from a javascript console

require('base/js/utils', function () { console.log('success:', arguments); }, function () { console.log('error:', arguments); });

?

[jcb91]

also, from your gist https://gist.github.com/epifanio/4fac50855c12ada0a6983ad2d8ff4ec1/70dd7aeedc8f78f968930dc3188078b8fd9b36cf, it seems that you're running

sudo jupyter nbextension enable codefolding

when actually, you need the full require path to the nbextension entry-point, which would be

sudo jupyter nbextension enable usability/codefolding/main

[epifanio]

Ok, seems I find the culprit. Switching between session and users from the same ssh login shell is not as safe as I thought. The environment between the users get confused.
The situation now is:

• I can load the extensions for all the users
• I can have access to the NBEXTENSIONS configuration page for all the users
• The extensions are listed as incompatible, but once activated they works fine
• in the NBEXTENSIONS configuration page the extensions are listed twice

To have this working I installed as sudo the main repository:

git clone https://github.com/ipython-contrib/IPython-notebook-extensions
cd IPython-notebook-extensions
sudo python3 setup.py install

At this point the extensions are on the server but the other users can't access to the NBEXTENSIONS configuration page, it gives a 404 error. To fix this I opened a new terminal and I logged in a new ssh session with the new user and I installed the extensions with:

pip install https://github.com/ipython-contrib/IPython-notebook-extensions/archive/master.zip --user

While this fix the availability of extensions for the user, it will add a bouch of duplicate libraries in the usser's local environment see installation log
then stop the user notebook server. log-out, log-in and now the user has access to the configuration page:

Uncheck the box to enable incompatible extensions:

As you can see all the extensions gets duplicated, I thought because it is finding the extensions system wide and user side (.local/share/jupyter/nbextensions) so I tried to remove the user's extensions running from the user's ssh shell:

rm -rf .local/share/jupyter/nbextensions/publishing 
rm -rf .local/share/jupyter/nbextensions/slidemode
rm -rf .local/share/jupyter/nbextensions/styling
rm -rf .local/share/jupyter/nbextensions/testing
rm -rf .local/share/jupyter/nbextensions/usability

But also after stopping the server and logging in again, I still have the extensions listed twice.

I tested some of the extensions and they works (note: I have to check both entries to have them enabled), except for the latex_envs which requires the changes you have in the relative branch.
I tried to apply them, this is the log trying to load the latex_envs . the log points to http://epinux.com:8888/user/epinux/nbextensions/usability/latex_envs/latex_envs.js to which I already applied the changes you suggested (seems they do not work anymore)

For the duplicate etryies, my best guess is that system wide, the extensions get installed in different location and are loaded twice but after:

locate latex_envs

I got:

/home/epinux/anaconda3/conda-bld/work/nbextensions/usability/latex_envs/latex_envs.js
/home/epinux/dev/IPython-notebook-extensions/nbextensions/usability/latex_envs/latex_envs.js
/usr/local/share/jupyter/nbextensions/usability/latex_envs/latex_envs.js

from:

import os.path
from jupyter_core.paths import jupyter_path
for base in jupyter_path():
    print(os.path.join(base, 'nbextensions'))

I got:

/usr/local/share/jupyter/nbextensions
/usr/local/share/jupyter/nbextensions
/usr/share/jupyter/nbextensions

Here you can see the system directory is printed twice .. can be a clue? At this point I think the duplication depends on the configuration files. can it be?

From the main user notebook:

import os
import glob
from jupyter_core.paths import jupyter_config_dir, jupyter_data_dir
ls {jupyter_config_dir()}

returns:

jupyter_nbconvert_config.json  jupyter_notebook_config.json  migrated
jupyter_nbconvert_config.py    jupyter_notebook_config.py    nbconfig/
jupyter_nbconvert.json         jupyter_notebook.json

This is the content of

jupyter_notebook_config.json

{
  "NotebookApp": {
    "extra_template_paths": [
      "/home/epinux/.local/share/jupyter/templates"
    ],
    "nbserver_extensions": {
      "nbextensions": true
    },
    "server_extensions": [
      "nbextensions"
    ]
  },
  "version": 1
}

jupyter_notebook_config.py

#--- nbextensions configuration ---
import sys
sys.path.append('/home/epinux/.local/share/jupyter/extensions')
#--- nbextensions configuration ---
from jupyter_core.paths import jupyter_config_dir, jupyter_data_dir
import os 
import sys

sys.path.append(os.path.join(jupyter_data_dir(), 'extensions'))
c = get_config()
c.NotebookApp.extra_template_paths = [os.path.join(jupyter_data_dir(),'templates') ]
c.Exporter.template_path = [os.path.join(jupyter_data_dir(), 'templates') ]

and jupyter_notebook.json

{
  "Exporter": {
    "preprocessors": [
      "pre_codefolding.CodeFoldingPreprocessor",
      "pre_pymarkdown.PyMarkdownPreprocessor"
    ]
  },
  "NbConvertApp": {
    "postprocessor_class": "post_embedhtml.EmbedPostProcessor"
  },
  "NotebookApp": {
    "server_extensions": [
      "nbextensions"
    ]
  },
  "version": 1
}

[epifanio]

btw, .. @jcb91 I'm so happy to have them back working and for all the users! thanks a lot for all this work and to taking the time to look into my issue! :)

[jcb91]

Ah, you're welcome @epifanio, it's always nice to know that somebody's using the stuff :)

So, I think the duplicates might be an unintended result of my earlier suggestion in https://github.com/ipython-contrib/IPython-notebook-extensions/issues/617#issuecomment-223852006, that of setting the JUPYTER_DATA_DIR variable to be one of the directories from the system path. From

I got:

/usr/local/share/jupyter/nbextensions
/usr/local/share/jupyter/nbextensions
/usr/share/jupyter/nbextensions

We can see that /usr/local/share/jupyter/nbextensions appears twice over, so I think this is where the duplicates are coming from. From

This is what I use in the jupyterhub configuration file:

c = get_config()

os.environ['JUPYTER_DATA_DIR'] = '/usr/local/share/jupyter/'

I guess what's happening is that the environment-variable path gets blindly prepended to the system path, without checking whether it's already in there. So, you should hopefully be able to fix it removing the relevant line in your jupyterhub configuration.

[jcb91]

As for each user being able to see the nbextensions configuration page, this relies on the server extension being specified in a relevant jupyter_notebook_config.json, for which we can use a similar trick to setting the JUPYTER_DATA_DIR environment variable to spoof the current install script. I should really have mentioned that when I suggested the previous trick.

[jcb91]

A full system-wide install of the repo should be possible using (bash):

JUPYTER_CONFIG_DIR=$(python -c "from jupyter_core.paths import SYSTEM_CONFIG_PATH as p; print(p[0])")
JUPYTER_DATA_DIR=$(python -c "from jupyter_core.paths import SYSTEM_JUPYTER_PATH as p; print(p[0])")
export JUPYTER_CONFIG_DIR JUPYTER_DATA_DIR
pip install https://github.com/ipython-contrib/IPython-notebook-extensions/archive/master.zip
unset JUPYTER_CONFIG_DIR JUPYTER_DATA_DIR

(which of course requires write permissions to the relevant directories , which are /usr/local/etc/jupyter and /usr/local/share/jupyter for unix-based systems)

[epifanio]

I upgraded all my installations to latest git master for ipython, notebook, jupyter, jupyterhub, ipywidgets I confirm the nbextensions are working fine for all the users (i'll try to debug latex_envs later on) and the config page loads for all users as well (with the duplicate effect)

@jcb91 what do you suggest? should I just run the commands below without baking-up or removing existent installation?

[jcb91]

Well, if it for you now, there's no pressing need to change. The only real difference will be if/when you need to add new users: with the existing install, you'll need to make sure their config files are modified as with those you've already done. Whereas with a system -wide config in place, you wouldn't need to do anything extra...

[epifanio]

I ran as root:

JUPYTER_CONFIG_DIR=$(python -c "from jupyter_core.paths import SYSTEM_CONFIG_PATH as p; print(p[0])")
JUPYTER_DATA_DIR=$(python -c "from jupyter_core.paths import SYSTEM_JUPYTER_PATH as p; print(p[0])")
export JUPYTER_CONFIG_DIR JUPYTER_DATA_DIR
pip install https://github.com/ipython-contrib/IPython-notebook-extensions/archive/master.zip
unset JUPYTER_CONFIG_DIR JUPYTER_DATA_DIR

then added a new unix user on the server and use it to login on the hub, I got the same behaviour. 404 at the config nbextension page. Logging in with previous users still has the duplicated entryies in the config page (which is accessible, no-404)

[jcb91]

I ran as root:

Could you post any logs form the install - what file(s) got written?

I got the same behaviour. 404 at the config nbextensions page

Hmm. Ok, I'm not exactly sure what's going on there. In the server logs (when run with --debug flag), you should be able to find which config files get loaded - could you check? Do you get any warning logs about being unable to import the extension (module name is nbextensions)? If not, then it seems that the relevant config files aren't requesting it be loaded...

[jcb91]

also, I assume you restart the server after the install?

[epifanio]

@jcb91 yes I stopped the hub including the http_proxy and restart it from the shell. p.s. If you think can be useful I can add you as user on the hub. I'm on gitter #ipywidgets #ipython #jupyter

[epifanio]

I'll rerun the installation, save the shell output then I'll restart the server and check the logs in both shell and js console.

[jcb91]

okey doke 👍

[jcb91]

I'm not sure how single-user server logs get handled - do they go to the same place as the hub server logs?

[epifanio]

yes, each action on the hub (any user) is notified in the shell. I usually start the server with:

nohup sudo [my-jupyter-start-script.sh] &

so all the logs are stored in the nohup.out file.

[jcb91]

Righto, I'll await logs with interest. I've never used gitter, so don't really know where to start with that, but feel free to move conversation there if it's more convenient.

If you think can be useful I can add you as user on the hub

Possibly, if all the kernels are running on the same machine as the hardware I guess I could read relevant config files more quickly? Not sure.

[epifanio]

I made a pr to add a gitter badge https://github.com/ipython-contrib/IPython-notebook-extensions/pull/624 generating the log, I'll post here in 5'

[jcb91]

👍 merged

[epifanio]

Installation logs

HUB restart

root@Debian-70-wheezy-64-minimal:~# sh /usr/local/bin/geopy.sh
root@Debian-70-wheezy-64-minimal:~# nohup: appending output to 'nohup.out'

Output in nohup.out after the following actions:

• adduser jcb91
• log-in as admin in the hub -> admin control panel -> add user jcb91 -> start jcb91 server -> log out
• log-in as jcb91 -> start an empty python3 notebook -> go in the browser address and try to load http:my.hub.domain:8888/user/jcb91/nbextensions

this the nohup.out log: https://gist.github.com/c42ed8cfe84c684621cfc9c39b3172f2 (ignore the error frol L43-L187 was my mistake, I forgot to add the directory used to store the notebooks in the user home)

this the js console while loading the first notebook and then trying to load the nbextensions configuration page: https://gist.github.com/e5f47b42ad90e1fefc3746ac36a478e1

[jcb91]

this should now be a lot simpler with our new 2-part installation method:

pip install jupyter_contrib_nbextensions
jupyter contrib nbextension install --system

(as always, writing to system config files and/or jupyter data directories may require root privileges).
So, I'll close this assuming it's fixed. Please open a new issue if the new method doesn't work as intended...