Doom-emacs: EIN inline plots are not shown

[ashkan-leo]

What did you expect to happen?
EIN inline plots to be shown in the notebook.

What actually happened?
EIN doesn't show the plots in the opened notebooks -- see the screenshot below.

Steps to reproduce:

1. M-x ein:jupyter-server-start
2. Open or create a notebook
3. type "import matplotlib.pyplot as plt" and "%matplotlib inline" and execute the cells.
4. plot something like "plt.plot([1, 2], [1, 2])" and execute the cell.

System information:

((emacs
  (version . "26.3")
  (features . "JPEG RSVG IMAGEMAGICK GLIB NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS NS MODULES THREADS LCMS2")
  (build . "Oct 19, 2019")
  (buildopts "--disable-dependency-tracking --disable-silent-rules --enable-locallisppath=/usr/local/share/emacs/site-lisp --infodir=/usr/local/Cellar/emacs-plus/26.3/share/info/emacs --prefix=/usr/local/Cellar/emacs-plus/26.3 --with-xml2 --without-dbus --with-gnutls --with-imagemagick --with-modules --with-rsvg --with-ns --disable-ns-self-contained")
  (windowsys . batch)
  (daemonp . server-running))
 (doom
  (version . "2.0.9")
  (build . "HEAD -> develop b0978a452 2020-02-14 17:32:45 -0500")
  (dir . "~/.doom.d/"))
 (system
  (type . darwin)
  (config . "x86_64-apple-darwin19.0.0")
  (shell . "/usr/local/bin/fish")
  (uname . "Darwin 19.3.0 Darwin Kernel Version 19.3.0: Thu Jan  9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64 x86_64")
  (path "~/.nix-profile/bin" "/nix/var/nix/profiles/default/bin" "~/.jenv/shims" "~/.local/bin" "~/bin/" "/usr/local/sbin" "~/.emacs.d/bin" "/usr/local/bin" "/usr/bin" "/bin" "/usr/sbin" "/sbin" "/opt/X11/bin" "/Library/TeX/texbin" "." "~/.cask/bin" "/usr/local/Cellar/emacs-plus/26.3/libexec/emacs/26.3/x86_64-apple-darwin19.0.0"))
 (config
  (envfile . envvar-file)
  (elc-files . 0)
  (modules :completion (company +childframe) (ivy +prescient +childframe +icons) :ui workspaces deft doom doom-dashboard doom-quit ophints hl-todo indent-guides modeline nav-flash treemacs (popup +all +defaults +icons) pretty-code unicode vc-gutter vi-tilde-fringe window-select :editor (evil +everywhere) file-templates snippets fold (format +onsave) multiple-cursors parinfer rotate-text :emacs (dired +ranger +icons) electric ibuffer vc :term eshell term :tools (lookup +docsets +devdocs) eval direnv docker ein :checkers (syntax +childframe +childframe) spell :tools gist (lsp +company) macos magit make pass pdf tmux upload :lang clojure data emacs-lisp (haskell +dante) (java +meghanada) javascript julia latex markdown (org +attach +babel +capture +export +present +publish) plantuml purescript (python +pyenv +lsp) racket rest (scala +lsp) (sh +fish) web :app (rss +org) :config literate (default +bindings +smartparens))
  (packages . "<(void-function sp-point-in-string)>")
  (elpa "live-py-mode")
  (unpin "n/a")))

[SidharthArya]

@ashkan-leo Try adding
(setq ein:output-area-inlined-images t)
to your config

[jeyj0]

I am having a related, but different issue where the image is split (the image shows that best):

(that is with the suggested (setq ein:output-area-inlined-images t), but it looked the same before that as well)

[UndeadKernel]

The problem with the wrongly sliced images is due to display-line-numbers-mode. Try disabling it (e.g., M-x display-line-numbers-mode) and see if that fixes the problem.

[jeyj0]

Thanks, that works!

[ashkan-leo]

@SidharthArya I added that to my configs. It didn't help

[dickmao]

Hello, I am the primary developer of EIN.

My longstanding impression has been that the EIN Doom module has effectively zero chance of working out given the volume and severity of upstream changes. Moreover, there appears to be no regression testing.

My inclination is to abolish the EIN Doom module, and have Doom users install EIN in the conventional way. I imagine some nice vim-style keybindings are lost in return for a reasonable semblance of functionality.

I notice the EIN spacemacs layer is also constantly playing "catch-up" https://github.com/syl20bnr/spacemacs/issues/13243.

I don't believe such "catch-up" strategies are viable, but I am not one to impose that belief especially as Doom/Spacemacs community members appear to be willing to do this work.

[dickmao]

A third alternative (first is status quo, second is abolish) is to bring the Doom module upstream, test it there and ensure our tests pass. This is the most work (for me). It would help to know if any other upstream packages seamlessly insert a Doom module upon package-install when the symbol doom-version is bound.

[UndeadKernel]

Now that Doom supports pinning specific packet commits, I think we should have no problem supporting the EIN module. We could have EIN pinned down to a version that we know works and update the EIN module at our leisure.

Currently, we probably need to update the module to catch up to the latest changes though.

[dickmao]

The curated stability of Doom is a huge selling point,
I understand, but the bitrot in modules/tools/ein is rather extreme. That
ein:slice-image defaults true betrays a basic misunderstanding why that
boolean exists.

Doom users, I presume, can uncomment :tools ein in their init.el or
independently download EIN from MELPA. I suspect they're more likely to do
the former, which is a shame since 1. the bb97c11d11 pin is on an inferior
and buggy fork and 2. much of the curation is suspect.

So I'm quite convinced that abolishment is best for the EIN brand name, and
Doom users. It would take me three minutes to make a PR effecting this, but I
fear ruffling feathers.

[hlissner]

Hey @dickmao. Thanks for taking the time to chime in! I don't use ein, so I admit, the risk of bitrot with our ein module is higher than others. We certainly don't have regression tests (yet). I'm working on it!

In the meantime, we _do_ have pinning, which @UndeadKernel mentioned, but you didn't seem to acknowledge it as a solution. Only that the pinned commit is inferior (which I'll come to later).

Pinning was developed specifically to address volatile packages. At least long enough that we can catch up. Spacemacs doesn't have this, so they've got more catch to do, but perhaps they have the manpower to make up for it?

My inclination is to abolish the EIN Doom module, and have Doom users install EIN in the conventional way. I imagine some nice vim-style keybindings are lost in return for a reasonable semblance of functionality.

This seems a bit baby-with-the-bathwater. Rather than dump the module entirely, wouldn't the next logical step be to remove our (trivial amount of) conflicting configuration, so we'd at least be left with our keybinds?

Doom users, I presume, can uncomment :tools ein in their init.el or independently download EIN from MELPA. I suspect they're more likely to do the former, which is a shame since 1. the bb97c11d11 pin is on an inferior and buggy fork and...

Perhaps so. That's certainly something worth fixing. I wasn't aware that bb97c11d11 was inferior, but it seems like the fix would be simply to bump it to one that isn't. Is there one you'd recommend?

1. much of the curation is suspect.
   I understand, but the bitrot in modules/tools/ein is rather extreme.

This sounds like an exaggeration. Perhaps you saw Spacemacs' config, and assumed ours must be similarly complex? (Not that I've seen theirs either, haha)

Our configuration of ein amounts to these lines:

(setq ein:notebook-modes
      '(ein:notebook-multilang-mode
        ein:notebook-python-mode
        ein:notebook-plain-mode)
      ein:slice-image t)

(when (featurep! :completion company)
  (setq ein:completion-backend 'ein:use-company-backend)
  ;; sets `company-backends` buffer-locally in these modes
  (set-company-backend! '(ein:notebook-multilang-mode
                          ein:notebook-python-mode
                          ein:notebook-plain-mode)
    'ein:company-backend))

(after! ein-jupyter
  (setq ein:jupyter-server-args '("--no-browser"))
  (unless ein:jupyter-default-notebook-directory
    (setq ein:jupyter-default-notebook-directory "~/")))

I really don't think this is a non-trival or unmanageable amount of configuration. Perhaps these defaults are misguided? I'm curious to know how -- again, I don't use ein, so I rely on users to inform me of issues, but they likely won't have your level of insight. I'm very invested in the curation of these modules, so I'm more than willing to learn.

That ein:slice-image defaults true betrays a basic misunderstanding why that boolean exists.

In that case, why does it exist? I've defaulted it to on because, otherwise, next-lineing across large generated images will scroll you all the way to the other end of the image, in pursuit of the cursor, which is counter-intuitive. When sliced, next-line traverses the image incrementally.

If I misunderstood its purpose, or if this is an abuse of it, I'm interested to know how (or if other workarounds exist).

and have Doom users install EIN in the conventional way.

How would this address its volatility though? It seems like that only shifts the onus onto the end-user to keep up with changes. Which is fine, if they'd like to do that, but this doesn't necessarily seem like a better solution.

A third alternative (first is status quo, second is abolish) is to bring the Doom module upstream, test it there and ensure our tests pass. This is the most work (for me). It would help to know if any other upstream packages seamlessly insert a Doom module upon package-install when the symbol doom-version is bound.

I appreciate the offer, but I don't think this is necessary. I'll implement regression tests eventually, and I'm more than happy to put in the time to understand the packages we use.

[hlissner]

@ashkan-leo Does this help?

(after! ein
  (setq ein:slice-image nil))

[dickmao]

cf. #2635

[ashkan-leo]

2635 addresses this issue. I just upgraded and can see that plots are shown as expected.

Thanks!
No need to modify the configs in anyway.