Papermill: Cannot run papermill from a virtual environment - NoSuchKernel error

Created on 12 Nov 2018 · 7Comments · Source: nteract/papermill

Hello,

I am trying to execute a ipython notebook using the CLI (or papermil.execute_notebook) but get a NoSuchKernel error. Is there anything to do to configure the Kernel before running papermill?

The virtual environment was created with pipenv install papermill and entered using pipenv shell

(test_papermill) dpetitfils@home:~/test_papermill$ papermill /home/dpetitfils/the-decorators-they-wont-tell-you-about.ipynb /home/dpetitfils/the-decorators-they-wont-tell-you-about_out.ipynb
Input Notebook:  /home/dpetitfils/the-decorators-they-wont-tell-you-about.ipynb
Output Notebook: /home/dpetitfils/the-decorators-they-wont-tell-you-about_out.ipynb
  0%|                                                                                                                                                                                                                                          | 0/61 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/home/dpetitfils/.local/share/virtualenvs/test_papermill-mTaqzCVH/bin/papermill", line 11, in <module>
    sys.exit(papermill())
  File "/home/dpetitfils/.local/share/virtualenvs/test_papermill-mTaqzCVH/lib/python3.5/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/dpetitfils/.local/share/virtualenvs/test_papermill-mTaqzCVH/lib/python3.5/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/dpetitfils/.local/share/virtualenvs/test_papermill-mTaqzCVH/lib/python3.5/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/dpetitfils/.local/share/virtualenvs/test_papermill-mTaqzCVH/lib/python3.5/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/dpetitfils/.local/share/virtualenvs/test_papermill-mTaqzCVH/lib/python3.5/site-packages/papermill/cli.py", line 124, in papermill
    report_mode=report_mode,
  File "/home/dpetitfils/.local/share/virtualenvs/test_papermill-mTaqzCVH/lib/python3.5/site-packages/papermill/execute.py", line 66, in execute_notebook
    start_timeout=start_timeout,
  File "/home/dpetitfils/.local/share/virtualenvs/test_papermill-mTaqzCVH/lib/python3.5/site-packages/papermill/engines.py", line 50, in execute_notebook_with_engine
    return self.get_engine(engine_name).execute_notebook(nb, kernel_name, **kwargs)
  File "/home/dpetitfils/.local/share/virtualenvs/test_papermill-mTaqzCVH/lib/python3.5/site-packages/papermill/engines.py", line 279, in execute_notebook
    nb = cls.execute_managed_notebook(nb_man, kernel_name, log_output=log_output, **kwargs)
  File "/home/dpetitfils/.local/share/virtualenvs/test_papermill-mTaqzCVH/lib/python3.5/site-packages/papermill/engines.py", line 330, in execute_managed_notebook
    preprocessor.preprocess(nb_man, kwargs)
  File "/home/dpetitfils/.local/share/virtualenvs/test_papermill-mTaqzCVH/lib/python3.5/site-packages/papermill/preprocess.py", line 24, in preprocess
    with self.setup_preprocessor(nb_man.nb, resources, km=km):
  File "/usr/lib/python3.5/contextlib.py", line 59, in __enter__
    return next(self.gen)
  File "/home/dpetitfils/.local/share/virtualenvs/test_papermill-mTaqzCVH/lib/python3.5/site-packages/nbconvert/preprocessors/execute.py", line 304, in setup_preprocessor
    self.km, self.kc = self.start_new_kernel(cwd=path)
  File "/home/dpetitfils/.local/share/virtualenvs/test_papermill-mTaqzCVH/lib/python3.5/site-packages/papermill/preprocess.py", line 37, in start_new_kernel
    km, kc = super(PapermillExecutePreprocessor, self).start_new_kernel(**kwargs)
  File "/home/dpetitfils/.local/share/virtualenvs/test_papermill-mTaqzCVH/lib/python3.5/site-packages/nbconvert/preprocessors/execute.py", line 253, in start_new_kernel
    km.start_kernel(extra_arguments=self.extra_arguments, **kwargs)
  File "/home/dpetitfils/.local/share/virtualenvs/test_papermill-mTaqzCVH/lib/python3.5/site-packages/jupyter_client/manager.py", line 246, in start_kernel
    kernel_cmd = self.format_kernel_cmd(extra_arguments=extra_arguments)
  File "/home/dpetitfils/.local/share/virtualenvs/test_papermill-mTaqzCVH/lib/python3.5/site-packages/jupyter_client/manager.py", line 170, in format_kernel_cmd
    cmd = self.kernel_spec.argv + extra_arguments
  File "/home/dpetitfils/.local/share/virtualenvs/test_papermill-mTaqzCVH/lib/python3.5/site-packages/jupyter_client/manager.py", line 82, in kernel_spec
    self._kernel_spec = self.kernel_spec_manager.get_kernel_spec(self.kernel_name)
  File "/home/dpetitfils/.local/share/virtualenvs/test_papermill-mTaqzCVH/lib/python3.5/site-packages/jupyter_client/kernelspec.py", line 236, in get_kernel_spec
    raise NoSuchKernel(kernel_name)
jupyter_client.kernelspec.NoSuchKernel: No such kernel named conda-root-py

Source

eapetitfils

Most helpful comment

I ran into the same type of problem using conda environments, so in case somebody is interested, here is how I made papermill work with a conda environment specific kernel.

My setup is the following:

I have a dedicated jupyter environment:
conda create -n jupyter jupyterlab nb_conda papermill

nb_conda is used to easily set conda environment per notebook from within jupyterlab.

Then for a particular analysis I create a separate environment with all the dependencies necessary for my analysis, for example, for a simple R notebook:

conda create -n analysis_1 r-essentials papermill

With this setup, my notebooks get kernel metadata like so (when this analysis_1 kernel is selected for a notebook in jupyterlab):

"metadata": {
"kernelspec": {
"display_name": "R [conda env:analysis_1]",
"language": "R",
"name": "conda-env-analysis_1-r"
},

I tried to use papermill from my jupyter environment on a notebook using this kernel, but got a similar error:

jupyter_client.kernelspec.NoSuchKernel: No such kernel named conda-env-analysis_1-r

This can be fixed by installing jupyter and papermill in the analysis environment analysis_1:

conda install -n analysis_1 papermill

And specifying the name of the kernel to use from within this environment analysis_1 in the papermill command (-k option):

To find the kernel available from this environment (as noted by willingc):
conda activate analysis_1
(analysis_1)$ jupyter kernelspec list

Available kernels:
ir PATH/TO/ir
python3 PATH/TO/python3

And using the kernel needed from within the analysis_1 environment:
conda activate analysis_1
(analysis_1)$ papermill my_notebook.ipynb output_notebook.ipynb -k ir

Hope this can help ! Thanks a lot for this great tool !

khourhin on 2 May 2019

👍5 🎉2

All 7 comments

This error is saying that your virtualenv doesn't have the conda-root-py kernel installed which your notebook references:

No such kernel named conda-root-py

I am guessing that kernel is conda specific given the name. If this is pure python you can probably easily change your kernel to python2/3 and install the ipykernel in your virtualenv to execute it

MSeal on 12 Nov 2018

Hi @eapetitfils,
As @MSeal mentions, you probably do not have the desired kernel installed within your virtual environment.

To see currently installed kernels: jupyter kernelspec list
See help for more info on jupyter kernelspec -h.

I'm going to close this issue as there is no actionable development task, but please feel free to ask additional questions if needed and we will do our best to answer them.

willingc on 14 Nov 2018

❤1

I ran into the same type of problem using conda environments, so in case somebody is interested, here is how I made papermill work with a conda environment specific kernel.

My setup is the following:

I have a dedicated jupyter environment:
conda create -n jupyter jupyterlab nb_conda papermill

nb_conda is used to easily set conda environment per notebook from within jupyterlab.

Then for a particular analysis I create a separate environment with all the dependencies necessary for my analysis, for example, for a simple R notebook:

conda create -n analysis_1 r-essentials papermill

With this setup, my notebooks get kernel metadata like so (when this analysis_1 kernel is selected for a notebook in jupyterlab):

"metadata": {
"kernelspec": {
"display_name": "R [conda env:analysis_1]",
"language": "R",
"name": "conda-env-analysis_1-r"
},

I tried to use papermill from my jupyter environment on a notebook using this kernel, but got a similar error:

jupyter_client.kernelspec.NoSuchKernel: No such kernel named conda-env-analysis_1-r

This can be fixed by installing jupyter and papermill in the analysis environment analysis_1:

conda install -n analysis_1 papermill

And specifying the name of the kernel to use from within this environment analysis_1 in the papermill command (-k option):

To find the kernel available from this environment (as noted by willingc):
conda activate analysis_1
(analysis_1)$ jupyter kernelspec list

Available kernels:
ir PATH/TO/ir
python3 PATH/TO/python3

And using the kernel needed from within the analysis_1 environment:
conda activate analysis_1
(analysis_1)$ papermill my_notebook.ipynb output_notebook.ipynb -k ir

Hope this can help ! Thanks a lot for this great tool !

khourhin on 2 May 2019

👍5 🎉2

Thanks @khourhin, your explanations are frankly superb and worthy of being included in the doc!

NicolasRouquette on 21 May 2019

👍1

Glad this can help @NicolasRouquette ! @MSeal @willingc if you think so too feel free to reuse the post or let me know if you would be interested by a PR.

khourhin on 21 May 2019

@khourhin Please feel free to open a documentation PR. :smile: It would probably make sense to make it a page titled Troubleshooting: no such kernel error.

willingc on 21 May 2019

👍1

I run into the same problem with anaconda as well in Windows 10. I have solved it by changing the name in ipynb file from "conda-root-py" to "python3" according kernelspec list I have.

"metadata": { "celltoolbar": "Tags", "kernelspec": { "display_name": "Python [conda env:root] *", "language": "python", "name": "python3" },