Use-package: In some case, deferred loading means a package is never automatically loaded

[jwiegley]

Copying from https://www.reddit.com/r/emacs/comments/7hvqn1/usepackage_broken/:

So I upgraded some packages through list-packages and I am not sure of use-package was one of them, but after that I noticed that any package that defines a :bind in it won't take effect even after executing the binded commands.

This is an example:

  (use-package evil-magit
    :ensure t
    :after magit
    :commands (magit-status)
    :bind (:map magit-status-mode-map
                ("<C-tab>"   . nil)
                ("<tab>" . magit-section-cycle))
    )

The package should take effect once I execute magit-status (using C-x g) but it doesn't and I have to invoke the mode using M-x for it to take effect.

Removing the :bind lines makes it also take effect.

[jwiegley]

Your declaration says that magit-status is provided by evil-magit, which is not correct.

Can you clarify for me what exactly you're doing and when you expect what to be loading? You probably need to add :demand t to the above, to get what you're expecting. Otherwise, using :bind with a :map as you're doing tells use-package to wait until something loads evil-magit, but there's nothing in your declaration that specifically indicates when that should occur.

[kaushalmodi]

Not OP, but commenting because I have also used something like this somewhere in my config, where I load X package only once Y package is loaded.

Looks like OP wants to load evil-magit only once magit is loaded or magit-status is run, and then rebind few keys in magit-status-mode-map.

Your declaration says that magit-status is provided by evil-magit, which is not correct.

Looks like OP is simply trying to autoload X package when a function from Y package is run. I am pretty sure I have used this same technique in the past. I'll get back once I get to my computer.

[kaushalmodi]

OK, I found just one example in my config:

(use-package ag
  :commands (modi/ag-regexp-cwd
             modi/verilog-find-parent-module)
;; snip
)

The intention of that is to load the ag package when modi/ag-regexp-cwd (defined in the same form) or modi/verilog-find-parent-module (defined in a separate "config package" setup-verilog) is run. It used to work. I'll test if it still works when I get to a computer.

Well, the difference with OP's example is that I am not using :bind here to set something in the other package.

[[Source][1]]

[jwiegley]

@kaushalmodi Can you show me what that use-package form used to expand to, when it worked; and what it expands to now, when it doesn't work?

[kaushalmodi]

@jwiegley I got to my computer and just tested it out.. the example snippet I pasted still works (I did not imply that it did not work). I was more giving an example where autoloading one package based on a function outside that package worked (and still works).

It expands to:

(progn
  (defvar use-package--warning443
    #'(lambda
        (keyword err)
        (let
            ((msg
              (format "%s/%s: %s" 'ag keyword
                      (error-message-string err))))
          (ignore
           (display-warning 'use-package msg :error)))))
  (condition-case-unless-debug err
      (progn
        (unless
            (fboundp 'modi/ag-regexp-cwd)
          (autoload #'modi/ag-regexp-cwd "ag" nil t))
        (unless
            (fboundp 'modi/verilog-find-parent-module)
          (autoload #'modi/verilog-find-parent-module "ag" nil t)))
    (error
     (funcall use-package--warning443 :catch err))))

But looks like the "problem" comes up when :bind is involved too that tries to bind a key to a map outside the use-package form package.

[kaushalmodi]

Style 1

.. In order to do what the OP wants, I typically do:

(use-package ag
  :commands (modi/ag-regexp-cwd
             modi/verilog-find-parent-module)
  :init
  (with-eval-after-load 'verilog-mode
    (bind-keys :map verilog-mode-map
      ("C-c ;" . modi/verilog-find-parent-module))))

That expands to:

(progn
  (defvar use-package--warning457
    #'(lambda
        (keyword err)
        (let
            ((msg
              (format "%s/%s: %s" 'ag keyword
                      (error-message-string err))))
          (ignore
           (display-warning 'use-package msg :error)))))
  (condition-case-unless-debug err
      (progn
        (unless
            (fboundp 'modi/ag-regexp-cwd)
          (autoload #'modi/ag-regexp-cwd "ag" nil t))
        (unless
            (fboundp 'modi/verilog-find-parent-module)
          (autoload #'modi/verilog-find-parent-module "ag" nil t))
        (condition-case-unless-debug err
            (with-eval-after-load 'verilog-mode
              (bind-keys :map verilog-mode-map
                         ("C-c ;" . modi/verilog-find-parent-module)))
          (error
           (funcall use-package--warning457 :init err))))
    (error
     (funcall use-package--warning457 :catch err))))

Style 2

The form that OP wants to use is indeed more succint. Trying to use that form:

(use-package ag
  :commands (modi/ag-regexp-cwd
             modi/verilog-find-parent-module)
  :after verilog-mode
  :bind (:map verilog-mode-map
         ("C-c ;" . modi/verilog-find-parent-module)))

expands to:

(progn
  (defvar use-package--warning449
    #'(lambda
        (keyword err)
        (let
            ((msg
              (format "%s/%s: %s" 'ag keyword
                      (error-message-string err))))
          (ignore
           (display-warning 'use-package msg :error)))))
  (condition-case-unless-debug err
      (eval-after-load 'verilog-mode
        '(progn
           (unless
               (fboundp 'modi/verilog-find-parent-module)
             (autoload #'modi/verilog-find-parent-module "ag" nil t))
           (unless
               (fboundp 'modi/ag-regexp-cwd)
             (autoload #'modi/ag-regexp-cwd "ag" nil t))
           (ignore
            (bind-keys :package ag :map verilog-mode-map
                       ("C-c ;" . modi/verilog-find-parent-module)))))
    (error
     (funcall use-package--warning449 :catch err))))

~and that latter form indeed does not work.~

---

Update: Umm.. the latter form (style 2) works too! Not sure what happened last time.. Need to now get the output of the pp-macroexpand-last-sexp from the OP to compare the expansion of style 2 above with what he gets..

[jwiegley]

Your Style 2 is exactly what I would have written, fwiw.

[hisnawi]

I am still confused. My understanding for this code:

  (use-package evil-magit
    :ensure t
    :after magit
    :commands (magit-status)
    :bind (:map magit-status-mode-map
                ("<C-tab>"   . nil)
                ("<tab>" . magit-section-cycle))
    )

Is that the evil-magit package wont load until either:

1. magit-status is executed (even if this command doesnt belong to evil-magit)
2. or when while in the magit-status-mode-map I execute the keys or

John, what am I doing wrong, or what am I misunderstanding of use-package?

Is it that I can only use :command with commands local to the same package? Also with :bind, am I supposed to use it with modes local to the same package? Am I not supposed to use both :bind and :commands? Am I not supposed to use :after along with all the above combinations of :bind and :commands?

Bear with me as I am only trying to understand the issue.

[jwiegley]

Your 2nd option is the right one.

The :commands keyword says that the package being declared defines a command, so that use-package creates an autoload cookie for it in the case of deferred loading. It also implies deferred loading. It is meaningless to use :commands in a package FOO, if the command is actually provided by a package BAR.

The :after keyword prevents even the autoload cookies from being registered until after magit has loaded.

The key bindings into magit-status-mode-map will happen after magit has loaded.

What will cause evil-magit to finally load is it if you type <tab> while the magit-status-mode-map is active. This will trigger the autoload of evil-magit to define magit-section-cycle so it can be executed. If magit-section-cycle is not defined by evil-magit, it should result in a Lisp error.

In general, :commands and :bind should mention or bind keys to commands that are defined by the package being declared.

[jwiegley]

@kaushalmodi I wonder if you were being hit by #566 too...

[kaushalmodi]

No, I am not getting hit by that bug as I always use alists for my bindings. I am using the cons directly only in the above madeup examples.. trying to replicate the issue the OP mentioned.

Regarding your second-last comment above:

The :commands keyword says that the package being declared defines a command,

Surprisingly, as I mentioned in my earlier comment, that doesn't seem to have to be the case i.e. looks like any arbitrary command can autoload any arbitrary package. At least that's what it looks like in practice.

[jwiegley]

Let me clarify, (use-package foo :commands bar) expands to roughly:

(unless (fboundp 'bar)
  (autoload 'bar "foo" nil t))

[jwiegley]

It doesn't do anything more than this, except to imply deferred loading.

[kaushalmodi]

Let me clarify, (use-package foo :commands bar) expands to roughly:

Yes, I understood that portion (being a big fan of pp-macroexpand-last-sexp :) ). But then I was wondering how autoload worked.. and I emailed help-gnu-emacs list with a similar example as yours to understand this better.

If it works as I think, then we can generalize the use of :command as the keyword used to declare arbitrary function symbols that can autoload the PKG begin loaded by the use-package form.

[jwiegley]

How would we do that? What should the expanded text look like to accomplish that?

[hisnawi]

Then what should I do to have evil-magit load only when executing magit-status?

[kaushalmodi]

@hisnawi Does below work?

(use-package magit
  :defer t
  :config
  (use-package evil-magit
    :ensure t
    :bind (:map magit-status-mode-map
           ("<C-tab>" . nil)
           ("<tab>" . magit-section-cycle))))

It's basically an inside-out version of your snippet :)

You don't need :commands (magit-status).. magit-status anyways will autoload magit assuming that you are using package-initialize in your config.

[jwiegley]

@hisnawi I would recommend this:

(use-package magit
  :commands magit-status)

(use-package evil-magit
  :ensure t
  :after magit
  :demand t
  :bind (:map magit-status-mode-map
              ("<C-tab>" . nil)
              ("<tab>" . magit-section-cycle)))

It expands to (more or less):

(unless (fboundp 'magit-status)
  (autoload #'magit-status "magit" nil t))

(progn
  (use-package-ensure-elpa 'evil-magit '(t) '(:demand t))
  (eval-after-load 'magit
    '(progn
       (require 'evil-magit)
       (bind-keys :package evil-magit :map magit-status-mode-map
                  ("<C-tab>")
                  ("<tab>" . magit-section-cycle)))))

[hisnawi]

That worked, thanks a lot John.

[jwiegley]

The real trick is documenting this right so that others aren't confused, because I bet a lot of people were doing things that "just worked", without realizing it was working for pretty bogus reasons before.

[jwiegley]

For example, one thing that feels surprising is that :demand only demands loading once the :after conditions have been satisfied.