Jupyterlab: No --user install of Lab extentions

[tam203]

Hi, I've authored a pair of extensions, one for Jupyter Server and one for Jupyter Lab (front end). The way we are running the Jupyter system I think is common to many in that we have JupyterHub using the docker/kubernetes spawner to create containers where the user is jovyan (not root and no sudo ability).

In this set up I can install my server extension(s)

> pip install --upgrade git+https://github.com/met-office-lab/jubo.git --user
> jupyter serverextension enable --py jubo --user

But am unable to install my juptyerlab extentions:

> jupyter labextension install jubo_lab --user
...
Bad config encountered during initialization:
Unrecognized flag: '--user'

Without --user:

Enabling: jubo.jubo_server_ext
- Writing config: /home/jovyan/.jupyter
    - Validating...
      jubo.jubo_server_ext  OK
jovyan@jupyter-tam203:~$ jupyter labextension install jubo_lab
> /opt/conda/bin/npm pack jubo_lab
jubo_lab-0.1.0.tgz

Errored, use --debug for full output:
PermissionError: [Errno 1] Operation not permitted: '/opt/conda/share/jupyter/lab/staging/index.js'

So three questions:

A) is there a work around?
B) is there planed to be a solution?
C) What does jupyter labextension install jubo_lab actually do? Do extensions get tanspiled in to one central JavaScript app or are extensions kept separate from the "official" Lab code, with just settings/flags pointing to there install location.

Ta!

[blink1073]

Hi @tam203, what does jupyter lab paths show? jupyter labextension install foo installs to the Application directory (the first item in jupyter lab paths). You can change the application directory using the JUPYTERLAB_DIR environment variable to be a location of your choice, so that jupyter labextension install and jupyter lab will use that directory.

[jacobtomlinson]

Thanks for the response @blink1073. I'm working on this with @tam203.

The command outputs looks like this:

$ jupyter lab paths
Application directory:   /opt/conda/share/jupyter/lab
User Settings directory: /home/jovyan/.jupyter/lab/user-settings
Workspaces directory /home/jovyan/.jupyter/lab/workspaces

I tried setting the env var you suggest however that causes the following issue.

I'm guessing we want to prepend the path with our custom dir instead of replace it? I also tried a format like JUPYTERLAB_DIR: /home/jovyan/.lab:/opt/conda/share/jupyter/lab but that didn't work either.

[tam203]

To follow up on @jacobtomlinson message. I'm sure if we set JUPYTERLAB_DIR=/home/jovyan/.lab and run jupyter lab build then this problem would go away but then each of our users would have a copy of lab in their home. This would make it more difficult to upgrade as new versions/releases come out (we don't to edit our users home dirs). Ideally we want a base install as we currently have at /opt/conda/share/jupyter/lab and users can install in their home extra extensions in a location such as ~/.lab.

[blink1073]

JupyterLab is a single bundle using WebPack. A single version can be used if a base set of extensions is to be used by all users, but if each user has custom extensions then they will need their own JuptyerLab install.

[jacobtomlinson]

Hmm that's a shame.

When you do a jupyter labextension install <package> does that automatically rebuild the bundle? Or do I need to also run jupyter lab build?

[blink1073]

It automatically rebuilds unless you use jupyter labextension install <package> --no-build

[twolodzko]

Same problem for me. Jupyter is installed in ~/.local, while JupyterLab wants to install the extensions in /usr/local. Notice that this may be a problem for uses with non-root privileges.

[cancan101]

I would add that we also have this use case where we want to pre install a set of extensions for users but then allow them to add their own. The user accounts do not have sudo access (we are also using docker).

[ianabc]

We rely heavily on the various --user flags (to pip, serverextension etc.). The ability to augment a default install with a small number of user extensions is extremely useful.

[timkpaine]

I don't see it being terribly difficult to what most people want here. You can provide a base set of extensions (as tarballs) in staging, then users can set up the path to point to somewhere local/writable. You can provide the base set to be copied into their staging and then they can build into their own static. No sudo, base packages pulled from a common set (so not out of sync), and still able to install extras.

[timkpaine]

@ianabc @cancan101 see the above

[timkpaine]

@twolodzko there are many jupyter flags you can set. Have you tried running JUPYTERLAB_DIR=~/.jupyter jupyter lab build? There is definitely no requirement to have root, and you can install jupyterlab wherever you like. I do so for my reproducible build, it just makes a directory ./build and compiles jlab into there.

[timkpaine]

@twolodzko I also have custom directories for settings, kernels, configs, etc. Its all overridable with environment variables (I think I use 5 for the backend assets and 2 for the frontend assets/settings/keymaps/etc)

[ianabc]

Thanks @timkpaine, I'm looking at this now. Should I expect something like

!JUPYTERLAB_DIR=jupyter labextension install @jupyterlab/plotly-extension
!JUPYTERLAB_DIR=~/.jupyter jupyter lab build

To work similarly to --user? I'm running into problems at the moment (webpack being SIGKILL'd) but that could be a problem with my system. If this is the intended behaviour would --user be a useful flag to these commands?

[timkpaine]

@ianabc yep, for ref I set the following at build time (probably not all necessary for installing extensions):

mkdir -p build
cd build
export JUPYTER_CONFIG_DIR=`pwd`
export JUPYTER_PATH=`pwd`
export JUPYTER_RUNTIME_DIR=`pwd`
export JUPYTERLAB_DIR=`pwd`
cd ..

When I have my jupyter deployed (which consists for me of copying build/static/* to wherever my jupyterlab python package is installed), I set the following:

import jupyterlab
os.environ['JUPYTERLAB_DIR'] = os.path.abspath(os.path.dirname(jupyterlab.__file__))

[timkpaine]

For control over where configs/kernels/profiles/etc might be stashed you can check out JUPYTER_PATH, IPYTHONDIR, JUPYTER_CONFIG_DIR, JUPYTER_DATA_DIR, JUPYTER_RUNTIME_DIR, JUPYTER_STATIC_DIR according to here

[cancan101]

It might be helpful to see what it looks like to add these settings, etc to a Docker file which is built using root but will ultimately be run as a user.

[jacobtomlinson]

From our point of view it would be ideal if JUPYTERLAB_DIR behaved like a path. For example we could set it to JUPYTERLAB_DIR=~/.jupyterlab:/opt/conda/share/jupyter/lab which would allow users to install lab with custom plugins in their home directory if they wish, but if they don't (or on first run) it will pick up the system lab.

We are using zero-to-jupyterhub where all users get a docker image where lab has been installed at system level, then they get a home directory mounted in as a volume. I'd rather not have to pre-populate the home directories with a default version of Jupyter Lab.

[timkpaine]

That might be possible, but it's a bit misleading as it won't function like a union of paths (since there's only 1 webpacked asset). So users would have to install all of the system extensions in addition to their custom one.

[jacobtomlinson]

Yes, they would basically create a whole new lab bundle in their home directory.

It doesn't feel ideal but given the architecture of Jupyter Lab and the fact that you can't neatly install custom user specific plugins it is one of the only things I can think of.

[timkpaine]

It doesn't feel ideal but given the architecture of Jupyter Lab and the fact that you can't neatly install custom user specific plugins it is one of the only things I can think of.

"Neatly" is a matter of opinion, you have 1 neatly bundled asset as opposed to a bunch of random js files. It might be possible to just shade the staging, but create a new static folder. This way all the different packages can be gathered from a union of staging dirs, and bundled into a single user static dir. any thoughts on this @ellisonbg @blink1073 @ian-r-rose

[cancan101]

Right now I have the following in my Dockerfile:

RUN jupyter serverextension enable --py jupyterlab --sys-prefix

What env vars should I add before calling this such that after the build process additional extensions can be installed? ie how do I make sure that the bundle is created in an an appropriate location?

[timkpaine]

So that line configures your jupyter_notebook_config.json. If you want to change that its JUPYTER_CONFIG_DIR. To change the client assets, do something like JUPYTERLAB_DIR=~/.jupyter jupyter lab build

[mdeff]

Another use case for having multiple extension directories is when jupyterlab is installed system-wide (say by the package manager of a distribution). In such a case, the app dir would default to /usr/share/jupyter/lab/ so that all existing and new users can launch jupyterlab right away without the need to build their own app. The problem now is that if a user want to install an extension, he has to set JUPYTERLAB_DIR (e.g. in his .bashrc), build jupyterlab with jupyter lab build before he'll be able to install with jupyter labextension install <my-extension>.

That process could be made much simpler if we'd have a hierarchy of directories, as is the case with the jupyter notebook (shown by jupyter --paths). The shared directory would contain the core extensions forming jupyterlab and eventually some system-specific ones, and each user would have its own directory for his personal extensions.

[jacobtomlinson]

Yes! Thanks for that @mdeff that's exactly what I was trying to say before. This if definitely an important use case!

[garu57]

If I'm allowed to add my 5 cents from my forty years experience as a system engineer and system integrator (yes, I'm a dinosaur that still remember punched cards and ferrite cores memories :) in very large sites, i can assure you that the only way to ensure that all users obtains consistent and compatible results is to manage a centralized installation, while still allowing users to add their own extensions.

Once you force users to rebuild locally, part of the installation, as soon as you update the central install you will have lost control.
I can assure, by way too many direct experiences, that in such cases, either inviting or forcing you'll never obtain all users to rebuild their part and if they need to share or exchange data, that is a sure receipt for a big mess.

So referring to the comment of @blink1073, if you consider lab no more than a single user toy, that's ok.
But if you are thinking of lab as a system wide tool to be used by teams of scientists PLEASE don't do that. Think system wide, convenience of packaging should be never preferred to consistency, usability, availability and maintainability.

English is not my mother language so if anybody feels hurt by my words i apologize in advance, it was not my intention.

[mdeff]

That's worth much more than 5 cents @garu57 :+1:

[mdeff]

Yet another use case: if I'm a single user managing multiple environments with venv or conda, my settings and workspaces are shared across environments (as they are stored in ~/.jupyter/lab/{user-settings,workspaces} while my extensions are not (as they are stored in <env-name>/share/jupyter/lab). Storing core and system-wide extensions in /usr/share/jupyter/lab/ (or in whatever venv or conda environment folder) and user extensions in ~/.local/share/jupyter/lab/ would solve this consistency issue.

Moreover, that's exactly how the notebook works, with a folder hierarchy of settings and data:

$ jupyter --paths
config:
    /home/user/.jupyter
    /usr/etc/jupyter
    /usr/local/etc/jupyter
    /etc/jupyter
data:
    /home/user/.local/share/jupyter
    /usr/local/share/jupyter
    /usr/share/jupyter
runtime:
    /run/user/1000/jupyter

I have no experience in web development, so maybe there is advantages in a single app directory that I don't see. But from a user perspective, the way the lab manages extensions is a regression from how the notebook did it.

[jasongrout]

i can assure you that the only way to ensure that all users obtains consistent and compatible results is to manage a centralized installation, while still allowing users to add their own extensions.

To ensure all users obtain consistent and compatible results, shouldn't there be a single centralized installation with no custom extensions, so everyone is using the exact same platform? Note that each user can have different settings in the current JLab, but the actual software components should be the same, right?

Once you force users to rebuild locally, part of the installation, as soon as you update the central install you will have lost control.
I can assure, by way too many direct experiences, that in such cases, either inviting or forcing you'll never obtain all users to rebuild their part and if they need to share or exchange data, that is a sure receipt for a big mess.

The web application served to the browser is a single webpacked (compiled) js file - this is so that extensions and all dependencies can be deduped and version-matched across all installed plugins, so you have a coherent set of installed extensions. Your warnings here sound to me like they are reasons for not allowing a user to install custom extensions, but instead having a single system-wide installation.

[jasongrout]

Moreover, that's exactly how the notebook works, with a folder hierarchy of settings and data:

The notebook hierarchy is also a constant source of frustration for many people. For example, user-level environments (like a user-specific installation of conda) turn some of the hierarchy inside-out, so that the sys-prefix level is now more specific than the user level. We have regular confusion with people installing the ipywidgets extension and having an old version in another level of the hierarchy superseding it.

In general, the idea of a hierarchy of directories is nice, but the current notebook system is certainly not without fault. Again, as @blink1073 mentioned, in order to make sure that the collection of extensions and their dependencies formed a coherent whole, it would involve each user essentially compiling their own version of jlab from the union of extensions in the different directories. That's not impossible to implement, but we should be careful in how we design the hierarchy to try to correct some of the problems with the notebook extension system, and also be mindful of some of the issues that @garu57 raised.

[jasongrout]

Yet another use case: if I'm a single user managing multiple environments with venv or conda, my settings and workspaces are shared across environments (as they are stored in ~/.jupyter/lab/{user-settings,workspaces} while my extensions are not (as they are stored in /share/jupyter/lab). Storing core and system-wide extensions in /usr/share/jupyter/lab/ (or in whatever venv or conda environment folder) and user extensions in ~/.local/share/jupyter/lab/ would solve this consistency issue.

Very good point about prioritization of directories - this a huge point of confusion in the classic notebook too. Again, this is showing how the notion of one user having many environments inverts the priority order the notebook extensions assume (i.e., user is more specific than sys-prefix is more specific than global), which causes lots of confusion about classic notebook extensions.

In your example, user-level settings still apply across all virtual envs even with user-specific extensions, so it doesn't solve the issue in this case.

[jasongrout]

@mdeff - at the same time, I hear your point - it would be nice to have a hierarchical set of directories to look for extensions. Each user would have to build their custom version of JLab from their unique combination of extensions, presumably in the most specific directory.

[jacobtomlinson]

Your warnings here sound to me like they are reasons for not allowing a user to install custom extensions, but instead having a single system-wide installation.

Allowing for flexibility and customisation is hard, but that's not a reason not to do it.

One nice thing about having a path system is that you can provide a system install which the sysadmins or whoever can gurantee and support. But at the same time users can customise within their home directory. And if they hose their install by breaking something it is their problem and theirs alone.

Each user would have to build their custom version of JLab from their unique combination of extensions, presumably in the most specific directory.

I still think this is the best option. If a user does nothing and runs jupyter lab they get the system version. If they install an extention in their home directory and run jupyter lab build it builds a custom version in their home directory. Then next time they run jupyter lab they get their custom version.

[jasongrout]

Allowing for flexibility and customisation is hard, but that's not a reason not to do it.

Given that's one of the chief features of JupyterLab over the notebook (flexibility and customization in the extension system, even though it's taken us years to build this foundation), we certainly agree. For any specific feature, I'd also add the pragmatic constraints of resources available to do it and that it's the highest priority item, of course.

I still think this is the best option. If a user does nothing and runs jupyter lab they get the system version. If they install an extention in their home directory and run jupyter lab build it builds a custom version in their home directory. Then next time they run jupyter lab they get their custom version.

Sounds like a great next step to me. Is there anyone available to try this and submit a concrete PR we can iterate on and discuss? We're more than happy to help anyone to get started.

[jasongrout]

Your warnings here sound to me like they are reasons for not allowing a user to install custom extensions, but instead having a single system-wide installation.

Allowing for flexibility and customisation is hard, but that's not a reason not to do it.

My point here was that it sounded to me like the ideal of "consistent and compatible results" with a "centralized installation" was contradicting the idea of user-installed extensions (nothing to do with implementation details, just that the ideas themselves seemed in conflict). I'd love to hear more about how those two ideas support each other.

[jasongrout]

Another thought: the main reason for building all of the js into one 'statically linked binary' of sorts, is that we need to make sure that packages are deduplicated and have compatible versions (for example, certain libraries assume they can store state, so you can't have multiple copies working simultaneously on the same page).

However, which plugins are enabled and active on the page is independent of the build step, so it could be a user setting. Would it be sufficient for the usecases here to have a system jupyterlab installation that includes the possible plugins a user might want, possibly with many of them disabled, and then the user can selectively enable/disable extensions?

That way, tracking down compatibility issues and making sure extensions work together are the job of the system admin, while enabling/disabling functionality is open to the users. This allows the system admin to provide a platform that goes far beyond the stock install in potential, but lets the users selectively enable the features they want.

[dhirschfeld]

I'm interested in this discussion as I have different users who may possibly want different extensions. I've currently taken the approach of bundling everything for everyone which is fine for now but I think it would be good if each user could chose easily in the UI which of the bundled extensions they personally want enabled and for JupyterLab to remember that setting for them - which is what I think @jasongrout is suggesting.

AFAIK there's no way currently in the UI to enable or disable extensions on-the-fly?

[mdeff]

Thanks @jasongrout for your detailed answers. It looks like we all agree that it would be nice to have a hierarchical set of directories to look for extensions. The problem I see with the build step (in the solution proposed by @jacobtomlinson and @jasongrout) is what will happen when jlab (or any system extension) will be upgraded. Will the users have an outdated build in their home and need to manually rebuild? Should jlab check when it launches if its assets are out-of-date and automatically rebuild or emit a warning?

Another solution would be to avoid the build step. Is there any way to prevent duplications and version clashes without building? (again, I have no experience in web dev)

If a user does nothing and runs jupyter lab they get the system version. If they install an extention in their home directory and run jupyter lab build it builds a custom version in their home directory. Then next time they run jupyter lab they get their custom version.

it would be nice to have a hierarchical set of directories to look for extensions. Each user would have to build their custom version of JLab from their unique combination of extensions, presumably in the most specific directory.

[mdeff]

However, which plugins are enabled and active on the page is independent of the build step, so it could be a user setting. Would it be sufficient for the usecases here to have a system jupyterlab installation that includes the possible plugins a user might want, possibly with many of them disabled, and then the user can selectively enable/disable extensions?

That's certainly a useful setting. It solves the issue of selecting extensions. Though it doesn't solve the use case where users want to install an extension which is not included in the system managed jlab installation. It's a great first step, but to me it only solves part of the issue.

[jacobtomlinson]

@jasongrout your suggestions are sensible and pragmatic.

I am concerned as I have seen many environments where "everything the user wants" is provided by default, and it never contains everything the user wants. I think it is really important for people to install their own plugins if they wish.

I'm keen to create an environment where the deal is if you wish to install your own extensions you are on your own, and that means managing your own update cycle. I expect that it will be the minority of users which manage their own setup, the rest will just use the system environment which you suggest. A message to let them know there is a more up to date lab on in the system install would be nice, but not critical.

[garu57]

Hi all,
sorry if I didn't get back before but I was rather busy with my job.

Anyhow in the meanwhile I played a bit with lab, nothing serious, nothing more than a few demos, but boy! it's amazing. My compliments to you all, no flattery, I'm sincere. If I only had this when I was younger and still had some spare neurons!

But more than this I tried to understand how it is built, I mean the build plumbing not what the code is doing. No previous exposure to JavaScript other than hunting for bugs in some Ajax jquery pages and configuring some applications, so I had to dig a bit about npm, yarn, webpackand their universe. As you may imagine I just scratched the surface so may be what I'm gonna say could be easily proved wrong (and I really hope so) by someone more knowledgeable. Moreover, I always worked with Winpyton since its inception (at that time conda was in its early days and was just creating more problems than those it solved), but since Winpyton ships lab preconfigured with some extensions and since I see that the most part of issues are from Anaconda users I switched to it to have a base jupyter environment like most users.

For a number of reasons I have to stay on Windows and had no time to try this on linux, but I hardly can imagine a more involute platform to develop than it, so if it works in windows it will work anywhere else, better. I simply tried jupyter lab build without any extension, but those shipped with the product, with JUPYTERLAB_DIR set to avoid clobbering conda installed package, on a couple of pc (Win 7 and 10, no difference of behavior so I won't mention it further) and here is what I can see.

yarnis touted as a tool to guarantee stable and reproducible builds, but the way it is engineered now, gives exactly opposite results. Given the large and nested set of dependencies, and given that most package.json specify equal or higher than version for dependencies, each time you rebuild, yarn will resolve and download the latest version of each requisite package and you will never obtain neither something similar to what is shipped nor reproducible builds.
Pls see for example discussions in Running Yarn offline and https://github.com/jupyterlab/jupyterlab/issues/2065 to see what i mean.

I took a casual sample of the involved packages (don't ask, I don't remember which and given what i found it is not significant) and made a search for issues to find about version to version stability and, as I imagined, for almost all version changes of many packages there was a surge of issues often for subtle incompatibilities.
In my previous post it was more a sensation due to a long-standing experience, but now I'm certain that forcing users to rebuild the whole app, just to add some data view extensions, YOU will lose control and in the long run it can only produce a number of unsatisfied users and a flood of difficult to diagnose issues.
@jasongrout said that prioritization of directories is a point of confusion with notebook, i may agree with you, but if you got an issue in that area, at least you know that the application is the well tested and stable one that was shipped and 90% of the times it is a user error, easy to diagnose. But if user rebuilds the whole lab and he gets a problem in the base app, how are you going to diagnose it if you no longer know what the user packed in the app?
As a side note, if you have write permission over the python installation, following the first part of instructions for installing extensions before reading the Advanced Usage part where it talks about JUPYTERLAB_DIR, you will end clobbering the installation and if anything goes wrong...
Well, you know I'm an unfortunate guy, it happened to me during the first attempt. Network outage during the build and the staging got somehow screwed up. Following builds were steadily failing with arcane errors. I was in a hurry so conda uninstall and reinstall cured it.

I don't think that --offline is going to solve all the issues either. Let me tell you a tale.
As you may know, rebuilding on windows fails the canvas compile unless you have installed GTK2 and libjpeg. They pretend to be installed in C:\. No way, if I would have installed in C:\ everything that wanted to be installed there now it would be a cesspool (do you know of any application that in unix wants to be installed in root dir?).
I know it is optional but i wanted to see what it costs to obtain a clean build, so I installed in their proper place and npm install --GTK_Root=xxxxxx --jpeg_root=xxxxxxx canvas worked like a charm. Then, since i'm a clever guy, i thought, let's put a prebuilt canvas in staging\node_modules and add --GTK_Root= and --jpeg_root= to "install": "node-gyp rebuild" script in package.json so that yarn can pick it from there and compile properly.
Obviously I didn't understand a heck of how yarn works. canvas is required by a couple of vega* and d3* packages that are required by @jupiterlab/vega2-extension, so, unless i alter package.json of all the requiring packages tree into installation staging to point to a local built canvas, yarn will still download it overwriting the prebuilt one and the compile will still fail.
Well, not exactly a neat solution to recommend to a casual user and it is not sure it works either, when yarn find a more recent version of any of the packages requiring canvas, again it will happily download them and overwrite whatever you may have put in staging.
So... i could download in advance latest version of the involved packages, then alter them, then...
Well I surrender, may be I can put temporary links in C:\ for the time of build. Voilà, done! But then a thougth stroke me, what if I a were a poor user with no administrative privileges to write on system dirs and I would need both canvas and some user extensions? Well I would have to choose, either a base installation with canvas but no user extensions or the new build with user extensions but no canvas. Ah, good old days when it was enough to add a directory to JUPYTER_PATH, register the new extensions et voilà, notebook was immediately picking them without touching the base installation...

I really hope that someone will tell me that I didn't understand nothing, here is a magic yarn option that make what you need, but if I have to judge from yarn opened issues I'm not the only one to hit problems like this.

I know, I know, I'm too much verbose, it's my defect, so now i stop, but for one last consideration. Again, please find a solution to allow adding user extensions, like with notebook, without having to rebuild everything. For the moment it's not a real problem for me, I mean, right now, I'm using it only for playing, and since for my job it is a whole life that I have to take bunches of unrelated things and hack solutions to have them working together, whatever solution you choose, in a way or another, with prayers or hammer, I will bent it to work the way I want, but IMO once lab replaces notebook and starts to diffuse you are going to drown in swamp of issues.

Cheers,
Gabriele

[ianabc]

Hi @garu57,

Thanks for the (long!) read. I agree with almost all of the points you are making and am also really hoping that I'm missing something.

I don't find the prioritization of directories a source of confusion. It implements a well established pattern of inheritance, where system-wide installations can be customized by users. It's familiar to almost all of my users in the form of --user package installations (for pip) as well as installation of notebook extensions etc. A lot of them are also familiar with the same concept through adjusting their PATH and other environment variables.

My main concern is for JupyterHub installations where there's a large and varied user base. Someone running a hub can provide a system-wide installation to satisfy the majority of users, but some users will want/need to augment the installation with their own stuff e.g. when testing new extensions.

The removal of inline javascript and the drive to have things implemented as (lab)extensions really exacerbates the problem: When a user wants to try out a new extension they need a reliable path to do it themselves (on top of the existing installation). They won't/can't always wait for the hub admin to update the system-wide installation, and if they did, it would end up bloated by everything that anyone has ever requested. I've had trouble doing this for myself as a user (using environment variables as above) so I'm not really sure what to tell my users.

I'll try digging around in the webpack documentation to see if I can find what I'm missing.

[dhirschfeld]

When a user wants to try out a new extension they need a reliable path to do it themselves on top of the existing installation

I'll just note here that since JupyterLab provides a terminal a user can quite easily install a new extension into their running JupyterLab and just refresh their browser. Seems to work great for testing purposes though, at least with my setup, the changes wouldn't survive after the container was destroyed.

I don't however automatically destroy containers after the user exits jupyterlab - they'll simply reuse the same container unless they explicitly choose to shut it down themselves so this is a quiet useful testing workflow.

[pybokeh]

Commenting here just in case my issue is the same or similar. I tried to install a jupyterlab extension, but getting permission error:

(.venv) λ jupyter labextension install jupyterlab_voyager
> D:\nodejs\npm.CMD pack jupyterlab_voyager
jupyterlab_voyager-0.2.0.tgz

Errored, use --debug for full output:
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\blah_blah

I am on Windows 7, jupyterlab version==0.34.1

I set JUPYTERLAB_DIR environment variable, but still getting same error.

I tried to find a workaround in this discussion thread, but haven't found one.

[blink1073]

Hi @pybokeh, can you please post the output of jupyter labextension install --debug jupyterlab_voyager?

[pybokeh]

@blink1073 After ensuring there were no other Python processes that were running concurrently and still having that error, I decided to reboot my computer. After rebooting, I am now able to install the voyager extension. I can only guess that perhaps there my company's anti-virus was running at that time and was perhaps preventing file system access. Sorry for the false alarm!

[blink1073]

Been there, glad you got sorted!

[itcarroll]

This thread began with a jupyterhub use case ... has a solution/workaround (depending on your point of view I guess) been documented for this case? The workaround I'm barely grasping is to allow users (me) to set ~/.local as the application directory, install my own jupyter labextensions and build my own webpack, and ask jupyterhub admins (not me) to make the hub launch my webpack rather than whatever's in /usr/local? Are these steps (or the right ones, if these aren't) documented somewhere that's easy to follow?

[ianabc]

Sorry for the delay, but I don't have a satisfactory workaround at the moment.

The process you described "kinda" worked when I tried it, but it was really fragile so I didn't pursue it. It was quite "all or nothing"; you had to commit yourself to rebuilding all of the extensions (including any that might be _required_ by the system). There isn't really a concept of augmenting an existing system-wide installation with your own extras as far as I know. It's frustrating because it that behaviour is one of the real strengths of python in general and the notebook specifically, and to be honest this is the main thing holding up our migration to lab.

[skrat]

It's IMO plain broken, you have a command there (jupyter labextension install), and you have UI there, and, out of the box, on a multi-user system, it's broken. So do you need to rebuild jupyter frontend after installing an extension? If not, then what's the problem with layering extension storage paths into system, local and user ones?

[jasongrout]

So do you need to rebuild jupyter frontend after installing an extension?

Yes.

We are reworking the extension system for jlab 3.0 with the goal of not needing a rebuild, in which case we should have the capacity to support layered extension paths.