Papermill: KeyError: 'language_info'

[yogevyuval]

I'm sorry for the lack of information or way to reproduce, but that's all the information I have:

While running a notebook we run many times, we got the following exception:

File "/usr/local/lib/python3.7/site-packages/papermill/execute.py", line 106, in execute_notebook
    **engine_kwargs
  File "/usr/local/lib/python3.7/site-packages/papermill/engines.py", line 49, in execute_notebook_with_engine
    return self.get_engine(engine_name).execute_notebook(nb, kernel_name, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/papermill/engines.py", line 343, in execute_notebook
    cls.execute_managed_notebook(nb_man, kernel_name, log_output=log_output, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/papermill/engines.py", line 402, in execute_managed_notebook
    return PapermillNotebookClient(nb_man, **final_kwargs).execute()
  File "/usr/local/lib/python3.7/site-packages/papermill/clientwrap.py", line 40, in execute
    self.nb.metadata['language_info'] = info_msg['content']['language_info']

Any idea why content would not have that key?

[MSeal]

Yes, so the metadata format technically doesn't require that the language or kernel fields be provided. However without them we can't A) choose what kernel to run (the UI prompts you) and B) can't know what language to translate parameters into.

Ideally we should raise a clearer message to this effect. I'll mark it as an enhancement issue for new contributors to be able to tackle.

[rgbkrk]

To help diagnose this can you:

• post the metadata field from the notebook (you don't have to include the whole notebook)
• run jupyter kernelspec list --json and paste the output here

[alex-treebeard]

I was looking at picking this up as my first contribution but I haven't been able to reproduce with missing metadata fields as @MSeal describes.

It appears the line that threw the exception runs after executing the notebook:
https://github.com/nteract/papermill/blob/main/papermill/clientwrap.py#L47

Is there still an error message fix needed here as described?

[yogevyuval]

This did not happen since posting this, so I have no new information to share. I will close this for now and reopen if issue rises again

[mgasner]

We are also hitting this in CI, with no apparent relevant changes.

[nannau]

I just recently had this. For me, I think ipykernel v5.4.0 breaks it.

Reverting to ipykernel==5.3.4 seems to resolve it for me. That might give some hints. :thinking: