Machinelearningnotebooks: Make Widget available for VSCode & Jupyter Lab

[swanderz]

Request

Make the widget available for other IDEs besides Jupyter Notebooks

Rationale

The widget is a great because it:
I love the widget. So much so that I use Jupyter notebooks as my IDE for the AML SDK even though I would prefer to use VSCode or Jupyter Labs.

• saves A LOT of time. Not necessary to switch to the Portal and go clicking for find the run that was just submitted for monitoring.

The widget is not great because it:

• is can't be used in other IDEs like VSCode or Jupyter Labs
• requires developing in the .ipynb's which are hard to version control (and therefore collaborate on). My current work around is to use the VSCode Python extension to port notebooks to .py's for version control. Then export the script back to a notebook to there are changes in source.

potential "quick fixes"

A method for Run's that returns a URL of the run for monitoring within the portal

[j-martens]

@sdgilley - do you have the steps for JupyterLab to share?

[BillmanH]

Repro steps are very easy: Run any widget in Jupyter Labs, or Visual Studio Code

from azureml.widgets import RunDetails

{{ your ML experiment here }}

run = experiment.submit(config=automl_config)
RunDetails(run).show()

You get no visual.

output:
_AutoMLWidget(widget_settings={'childWidgetDisplay': 'popup', 'send_telemetry': False, 'log_level': 'INFO', 's…

[sdgilley]

The widget does work in Jupyter Labs, but it need an extension installed before it functions correctly. See https://ipywidgets.readthedocs.io/en/stable/user_install.html#installing-the-jupyterlab-extension

There are also methods for run that return a URL to the experiment: see https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-track-experiments#view-the-experiment-in-the-azure-portal

Hope one or both of these help.

[BillmanH]

I tried this and got a new error:

from azureml.train.automl import AutoMLConfig

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)

automl_config = AutoMLConfig(task = 'regression',
                            X=X_train.values,
                            y=np.asarray(y_train.values.flatten()),
                            model_explainability=True,
                            iterations=10,
                            iteration_timeout_minutes=5,
                            primary_metric="spearman_correlation")

from azureml.widgets import RunDetails

run = experiment.submit(config=automl_config)
RunDetails(run).show()

returns:
Error displaying widget: model not found

Same notebook. I just conda installed the modules recommended above. Just to confirm I ran the same notebook again in jupyter notebook and the widget loaded fine.

[sdgilley]

did you restart the kernel after installing in JupyterLab?

[BillmanH]

I believe so, it was a while ago and I've been doing other things since then. Hold on I'll just run the notebook in labs again. I've been working in notebooks all day so I'll just pop over.
[.. walks over to other computer, fires same env, new kernel, runs lab, opens the same .ipynb file that I've been using successfully in notebook, run, wait ...]
thanks for looking into this btw, we really appreciate it. :)
[.. wait ..]
Yup, still crashes. I've been running the same notebook all day from the same virtual env. Runs fine in notebooks, not in labs.

[ahcan76]

Could you try these steps:

Open jupyter lab
Open new Terminal window from the launcher and run below commands.

sudo -i pip install azureml-widgets
sudo -i jupyter labextension install @jupyter-widgets/jupyterlab-manager

To verify run below command and check output:

jupyter labextension list

Output should be:

JupyterLab v0.34.12
Known labextensions:
app dir: /data/anaconda/envs/py35/share/jupyter/lab
@jupyter-widgets/jupyterlab-manager v0.37.4 enabled OK
@jupyterlab/git v0.2.2 enabled OK
@jupyterlab/hub-extension v0.11.0 enabled OK
azureml_widgets v1.0.1 enabled OK

Then reopen jupyterlab or reload the existing jupyter lab window by clicking F5 for extensions to be reloaded.

[BillmanH]

Converting to windows cmd:
pip install azureml-widgets
[requirement already satisfied]
jupyter labextension install @jupyter-widgets/jupyterlab-manager
no response

jupyter labextension list
(env-projectx) C:\Users\william.harding\Desktop\MLResearch>jupyter labextension list
JupyterLab v1.0.1
Known labextensions:
app dir: C:\Users\william.harding\AppData\Local\Continuum\anaconda3\envs{{redacted}}\share\jupyter\lab
@jupyter-widgets/jupyterlab-manager v1.0.1 enabled ok
azureml_widgets v1.0.1 enabled ok

* note * I think I'm seeing that my jypyter lab is ahead of the version you posted
* additional note * I'm off for vacation for two weeks so I might not jump back in to do repro steps for a while.

[skunkwerk]

If you run the widget in Google Colab, it doesn't display anyting beyond:

[swanderz]

@sdgilley I can confirm that this works (but not with dark theme lol)

conda install -c conda-forge nodejs  # took 2 min
conda update jupyterlab  # 30 s
jupyter labextension install @jupyter-widgets/[email protected]  # 20 min for some reason