Use-package: Nesting use-package declarations vs using :after

[sondr3]

I'm a bit confused as to the difference between nesting declarations of use-package versus using the :after keyword. For example, if you were doing something like this:

(use-package flycheck
    :config
    ;; snip

    (use-package flycheck-popup 
         :config
         ;; snip ))

Would that be equivalent to having the declarations separate and using :after flycheck on flycheck-popup? Or would you have to use :after even when nesting them?

[basil-conto]

As a general tip for understanding use-package semantics and debugging your setup, I recommend inspecting the expansions of use-package forms yourself. The simplest way to do this is by setting use-package-debug to a non-nil value and then evaluating the form in question, e.g. by placing the cursor after the form and typing C-x C-e (eval-last-sexp). This will pop up a buffer displaying the expanded macro.

Would that be equivalent to having the declarations separate and using :after flycheck on flycheck-popup?

I imagine only in specific, simple cases, e.g. when the use-package form serves no purpose other than to require the corresponding package.

Here are two sample expansions to illustrate the subtle differences in semantics (note that I have simplified the expanded code for clarity):

;; 1. Using :after
(use-package flycheck-popup
  :after flycheck
  :config
  (ignore "Loaded 'flycheck-popup"))

;; Becomes:
(progn
  (with-eval-after-load 'flycheck
    (require 'flycheck-popup nil t))
  (with-eval-after-load 'flycheck-popup
    (ignore "Loaded 'flycheck-popup")))

;; 2. Nested
(use-package flycheck
  :defer
  :config
  (ignore "Loaded 'flycheck")

  (use-package flycheck-popup
    :defer
    :config
    (ignore "Loaded 'flycheck-popup")))

;; Becomes:
(progn
  (with-eval-after-load 'flycheck
    (ignore "Loaded 'flycheck")
    (with-eval-after-load 'flycheck-popup
      (ignore "Loaded 'flycheck-popup"))))

In the former case, anything that causes flycheck to be loaded (e.g. calling an autoloaded function) will also result in flycheck-popup being loaded immediately thereafter. In the latter case, loading flycheck will not directly result in flycheck-popup being loaded.

In general, I find the former approach much clearer and robust, as the :config clause of the flycheck-popup will run regardless of what caused the library to be loaded. If you really want the latter behaviour, I suggest first trying to use a flycheck hook or similar to register a callback.

I have yet to come across a sufficiently compelling reason to nest use-package forms in my own setup. Could you give an example of a situation in which you would be tempted to do this in your own setup and/or describe the intended behaviour?

As an aside, I would be interested to know how nested use-package forms behave when byte-compiled.

[basil-conto]

Here is the same example with stricter semantics (no :defer), which is closer to the code in the original question:

;; 1. Using :after
(use-package flycheck-popup
  :after flycheck
  :config
  (ignore "Loaded 'flycheck-popup"))

;; Becomes:
(progn
  (with-eval-after-load 'flycheck
    (require 'flycheck-popup nil t))
  (with-eval-after-load 'flycheck-popup
    (ignore "Loaded 'flycheck-popup")))

;; 2. Nested
(use-package flycheck
  :config
  (ignore "Loaded 'flycheck")

  (use-package flycheck-popup
    :config
    (ignore "Loaded 'flycheck-popup")))

;; Becomes:
(progn
  (when (require 'flycheck nil 't)
    (ignore "Loaded 'flycheck")
    (when (require 'flycheck-popup nil 't)
      (ignore "Loaded 'flycheck-popup"))))

Again there is a difference in semantics, but it's subtler than in the previous, non-strict example. In the former expansion, loading flycheck will pull in flycheck-popup with it, but the :config clause of flycheck-popup will always be executed after flycheck-popup is loaded, regardless of what caused it to be loaded. In the latter expansion, the :config clause of flycheck-popup will only be executed if flycheck was successfully loaded beforehand.

The only benefit of nesting here is that flycheck-popup will not be configured unless flycheck has already been successfully loaded. Again, I think this dependency could be more cleanly achieved via flycheck hooks or similar.

A related use-package keyword is :requires, which by default does not expand the use-package form at all unless certain libraries are already loaded.

[npostavs]

The simplest way to do this is by setting use-package-debug to a non-nil value and then evaluating the form in question, e.g. by placing the cursor after the form and typing C-x C-e (eval-last-sexp). This will pop up a buffer displaying the expanded macro.

I've always used M-x pp-macroexpand-last-sexp. No need to fiddle with any variables. Works with any macro.

As an aside, I would be interested to know how nested use-package forms behave when byte-compiled.

I don't think there is anything about byte-compiling that matters for that, you can try (disassemble (lambda () (use-package ...))) to see.

[basil-conto]

@npostavs Thanks for the pointers. :)

[alphapapa]

Hm, I've been using nested use-package forms for a while now. It helps keep related package configuration together in the init file. For example:

(use-package org
  :config
  (use-package org-recent-headings ...)
  ...)

It seems to make sense to me to keep the org-related packages' configuration nested inside org's own configuration.

The only benefit of nesting here is that flycheck-popup will not be configured unless flycheck has already been successfully loaded.

That seems like an important benefit to me. If the package that is depended upon fails to load, packages that depend on it won't also fail to load, preventing extra noise in the form of more errors and warnings.

Again, I think this dependency could be more cleanly achieved via flycheck hooks or similar.

Hooks are good, for packages that have the appropriate ones, but using nested use-package forms works in all cases, right?

Thanks for the discussion.

[dieggsy]

@basil-conto I'm unsure about the statement "flycheck-popup will not be configured unless flycheck has already been successfully loaded". What is telling you that that? Seems to me like :after achieves the same effect. After all, what would the point of :after be if packages were loaded even if the :after feature failed to load?

[basil-conto]

@alphapapa

It seems to make sense to me to keep the org-related packages' configuration nested inside org's own configuration.

I appreciate that. One downside that pops to mind is that nested :ensure clauses will not kick in until after their parent package is loaded, but that's just a matter of preference/circumstance. In my eyes, having separate use-package forms makes them more homogeneous, modular and declarative. :)

The only benefit of nesting here is that flycheck-popup will not be configured unless flycheck has already been successfully loaded.

That seems like an important benefit to me. If the package that is depended upon fails to load, packages that depend on it won't also fail to load, preventing extra noise in the form of more errors and warnings.

Definitely, though not all packages exhibit such a strong dependency. Error verbosity can also be a matter of preference/circumstance. For example, one might care less about silencing load errors and more about fixing the underlying cause when they arise.

using nested use-package forms works in all cases, right?

I guess it's a failproof alternative to hooks for enforcing package dependencies, but I can imagine this playing less well with other initialisation features, such as the aforementioned :ensure clause and autoloaded key bindings for nested packages.

---

@therockmandolinist

I'm unsure about the statement "flycheck-popup will not be configured unless flycheck has already been successfully loaded". What is telling you that that?

The syntax tree. :) Compare the sample expansions I listed:

;; 1. Using :after
(progn
  (with-eval-after-load 'flycheck
    (require 'flycheck-popup nil t))
  (with-eval-after-load 'flycheck-popup
    (ignore "Loaded 'flycheck-popup")))

;; 2. Nested
(progn
  (when (require 'flycheck nil 't)
    (ignore "Loaded 'flycheck")
    (when (require 'flycheck-popup nil 't)
      (ignore "Loaded 'flycheck-popup"))))

In (1), the eval-after-load forms for each package are registered one after another, and irrespective of each other. If you evaluate (1) followed by (require 'flycheck-popup), then (ignore "Loaded 'flycheck-popup") will also be evaluated (assuming the require succeeds). Whether flycheck also gets loaded as a result of this depends only on whether flycheck-popup requires it.

In (2), (ignore "Loaded 'flycheck-popup") (what I referred to as the "configuration") will only run after both flycheck and flycheck-popup have successfully been loaded, in that order. Another difference is that (2) tries to load flycheck immediately.

---

On a side note, I realise now that using message instead of ignore in my examples would have made them more illustrative in practice. :)

[sondr3]

Thanks for explaining it all so well @basil-conto, and all you other for asking questions. 😃

[basil-conto]

@sondr3 I'm glad you found this discussion helpful; I certainly did. :)

---

@therockmandolinist

what would the point of :after be if packages were loaded even if the :after feature failed to load?

Sorry I overlooked this question last time, though I believe I have at least partially answered it already. Ultimately, the point of any of these features is to expand to their respective definitions, which may or may not align with the author's intentions. A misalignment is a bug; anything beyond that is either a feature request or a discussion on naming. :)

Philosophy aside, I think we can all informally agree that :after serves to load packages of varying interdependence in tandem. In my opinion, it serves this purpose in an effective and flexible way by harnessing eval-after-load: packages can be loaded either eagerly or lazily; the loading of one package can trigger an attempt to load another package; packages using :after can still be loaded independently of one another when desired, etc. This flexibility allows the user to define whatever inter-package relations they want in their configuration. Here's an example:

(use-package dired-x
  :after dired
  :bind ("C-x C-j" . dired-jump))

dired-x will be lazy-loaded either after dired is loaded by some other means or when dired-jump is invoked. In the latter case, the onus lies on a) the maintainers of dired-x to require the correct dependencies for their code; and b) the user to either use a working package manager or install packages and their dependencies properly.

What your question seems to be suggesting, though, is a complete suppression of dependent packages; is this really your intention? Either that or you're trying to blame a botched package installation on use-package. :)

[dieggsy]

@basil-conto No need to jump to conclusions - it was simply a question based on skepticism, which you've actually addressed nicely, so thanks!

[basil-conto]

@therockmandolinist Sorry if I sounded accusing, I was just thinking of different possibilities out loud and/or rambling. :)

[vgivanovic]

(use-package muse
  :config ((use-package muse-docbook)
        (use-package muse-html)))

raises the error (invalid-function (use-package muse-docbook)). If this is not a bug, I don't understand why it should raise an error.

A single nested use-package works as expected, as does

(use-package muse)
(use-package muse-docbook :after muse)
(use-package muse-html    :after muse)
(use-package muse-latex   :after muse)
(use-package muse-texinfo :after muse)

but I don't thinkit'is as clear as multiple nested use-package's.

Could someone comment on the multiple nested use-package's?

[npostavs]

Try

(use-package muse
  :config (progn
            (use-package muse-docbook)
            (use-package muse-html)))

or

(use-package muse
  :config
  (use-package muse-docbook)
  (use-package muse-html))

[jwiegley]

I now prefer non-nested use-package declarations where possible, using :after to avoid ordering dependencies in my init file. If a package has submodules, I will even sometimes require them within the :config block, since this is closer to what's really happening, as they are not independent "packages" per se.