Straight.el: Advice on installing mu4e

[falloutphil]

Hi,

The installation of mu4e fails, I'm guessing because its build requires C build of mu as well as installation the lisp for mu4e. The build directory ends-up empty. I get the following warning:
Error (use-package): Cannot load mu4e

Reading a few other posts this might be that the recipe is wrong?

I can install this package use apt install or build it manually using make, meaning the output is placed here (or the local directory equivalent if using make):

mu binary goes here - /usr/bin/mu
lisp goes here - /usr/share/emacs/site-lisp/mu4e
etc...

My question is can I make this package install under straight given it needs autogen/configure/make, and if not what is the best practice/least disruptive way to marry this with the rest of my straight setup using an external build?

For example this hack seems to work, but is ugly:
cd /home/some_user/.emacs.d/straight/build/mu4e
ln -s /usr/share/emacs/site-lisp/mu4e/*.el .

Am I right that it's the underlying package straight is pulling in rather than a problem with straight itself which is the issue?

If it's not sensibly possible to fix the straight install of this package, should an exclusions list be maintained to prevent installation of known broken packages?

[raxod502]

This problem is because the authors of mu4e declined to publish the package to MELPA, _and_ they decided to be special and put the Lisp files in a weird subdirectory, so there does not exist any metadata which tells straight.el where to find the Lisp files.

Solution for you:

(straight-use-package
 '(mu4e :files (:defaults "mu4e/*.el")))

To solve this problem in general, mu4e must be published to a package registry. After all, it isn't reasonable to expect a package manager to "just know" how to install a weird nonstandard package that hasn't been published anywhere.

Luckily, it seems that the el-get repository has mu4e (see here), so by adding support for el-get, we'd solve this problem. This is the approach that I would recommend to contributors.

[falloutphil]

Thanks for response. Totally agree that if packages are non-standard there is little a generic package manager can reasonably do.

Agreed that https://github.com/raxod502/straight.el/issues/115 looks like it will take care of autogen/make requirements.

For now the advice on https://github.com/raxod502/straight.el/issues/386 looks like a good starting point for me producing a basic custom build, or I could simply maintain the non emacs lisp components outside of straight.

The fix you've provided above, which I believe should at least install the emacs lisp correct by appending to the default patterns matches for files - I get an error generating the repo name. I suspect I'm doing something dumb?

Debugger entered--Lisp error: (wrong-type-argument stringp nil)
  string-match("^.*/\\(.+\\)\\.git$" nil)
  straight-vc-git-local-repo-name((:files (:defaults "mu4e/*.el") :package "mu4e" :type git))
  apply(straight-vc-git-local-repo-name (:files (:defaults "mu4e/*.el") :package "mu4e" :type git))
  straight-vc(local-repo-name git (:files (:defaults "mu4e/*.el") :package "mu4e" :type git))
  straight-vc-local-repo-name((:files (:defaults "mu4e/*.el") :package "mu4e" :type git))
  straight--convert-recipe((mu4e :files (:defaults "mu4e/*.el")) nil)
  straight-use-package((mu4e :files (:defaults "mu4e/*.el")))

[raxod502]

You are expecting recipe inheritance (#116), but this feature is not in master yet. Either upgrade to develop (i.e. set straight-repository-branch to "develop" and run straight-normalize-package), or specify all the required keys:

(straight-use-package '(mu4e :host github :repo "emacsmirror/mu4e"
                             :files (:defaults "mu4e/*.el")))

[progfolio]

I believe this is possibly related to #72 as well.

mu4e has elisp files which are generated during the make process.
e.g. https://github.com/djcb/mu/blob/master/mu4e/mu4e-meta.el.in

If they are not generated when the mu server is updated, the variables in the linked file will be outdated. This can be worked around by manually updating them, but it'd be nice if one could programatically detect the change and trigger a rebuild.

[raxod502]

I really wish package maintainers wouldn't do this. There's no good technical reason to require a build step, and it makes everything so complicated.

[progfolio]

I may have time to contribute a pull toward #72 soon. Do you have any ideas for an implementation? At first glance it would seem to involve adding a :build keyword to recipes which specifies the commands to be run in the package directory before installation.

This could be an interesting feature for a number of reasons. Not only would it enable installing packages that require a build step correctly, but it could also be used to run analysis of the package before installing it. Could be useful for auditing code prior to installation.

[raxod502]

@progfolio That's great! For approach: I think it would be best if we did something that is compatible with the el-get recipe format. They already have a whole interface worked out, and if we conform to it then we get to steal all of their recipes for very little effort. Stealing other people's good ideas: the straight.el way™. (It's what we did with MELPA.) el-get uses a :build key, plus some :build/... keys for system types. I kind of don't like that, because it makes automatic recipe transformations harder if the set of keys isn't fixed. Maybe we can refactor to just a :build key that can be either the commands to run or an alist mapping the system types to commands to run.

Now luckily el-get is WTFPL'd, so we don't have to be super careful about lifting bits of code to parse the recipe format. (I still have bad memories of pbl.) Although I doubt we want to copy very much wholesale.

I think it is fine to make build commands be synchronously run and to throw an error if they fail. If we start trying to allow build commands to fail, then we get into the awkward situation of having to deal with "was this package fully built? or did the build command fail? if the latter, how do we tell when we should try re-running the build command?" This forces your Emacs configuration to have a hard dependency on the build command, but in my view that's your problem because packages shouldn't be using this stuff if they want to be portable.

You could add a step to straight--build-package, maybe after dependency resolution but before autoload generation. Then put in some new subroutines, a la straight-expand-files-directive.

See also #115. And feel free to let me know if you have further questions.

[raxod502]

This thread is being closed automatically by Tidier because it is labeled with "waiting on response" and has not seen any activity for 90 days. But don't worry—if you have any information that might advance the discussion, leave a comment and I will be happy to reopen the thread :)

[LoganBarnett]

I wound up here after chasing down a problem with mu4e after switching to nixpkg. Inspired by the workaround above, I have done this to get me onto my next Nix related error ;)

# Remove the directory first or the path will be:
# ~/.emacs.d/.local/straight/build/mu4e/mu4e
rmdir ~/.emacs.d/.local/straight/build/mu4e || true
ln -snf ~/.nix-profile/share/emacs/site-lisp/mu4e \
    ~/.emacs.d/.local/straight/build/mu4e

As a Doom Emacs user I've transitively gotten a lot of mileage out of straight.el. Thanks for the hard work!

[raxod502]

Remove the directory first or the path will be

You can pass the -T argument to ln to eliminate this behavior (ln will throw an error if the path exists already, instead of putting the target inside of it when it's a directory).

[raxod502]

Thanks for the hard work!

Oh, and thank you! ^_^

[rileyrg]

Hi, Glad I found this was going crazy. So whats the current use-package form for mu4e when using straight?

[progfolio]

Here's what I use. It builds mu in the mu repo and I use that executable. You could alter the :pre-build commands to change the installation prefix and/or run make install. In either case you need to point the mu4e-mu-binary to the mu executable.

(use-package mu4e
  :straight ( :host github 
              :repo "djcb/mu"  
              :branch "master"
              :files ("mu4e/*")   
              :pre-build (("./autogen.sh") ("make"))) 
  :custom   (mu4e-mu-binary (expand-file-name "mu/mu" (straight--repos-dir "mu")))) 

[rileyrg]

thanks. I'll give it a try. )

On Thu, 14 Jan 2021 at 12:56, Nicholas Vollmer [e-mail]
wrote:

Here's what I use. It builds mu in the mu repo and I ise that executable.
You could alter the :pre-build commands to change the prefix during the
configure script and/or run make install. In either case you need to
point the mu4e-mu-binary to the mu executable.

(use-package mu4e :straight ( :host github :files ("mu4e/*") :repo "djcb/mu" :branch "master" :pre-build (("./autogen.sh") ("make"))) :commands (mu4e mu4e-update-index) :custom (mu4e-mu-binary (expand-file-name "mu/mu" (straight--repos-dir "mu")))

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/raxod502/straight.el/issues/491#issuecomment-760150634,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACYTC52YUCEYDM7KE3TIPTSZ3LXXANCNFSM4LAHA5LQ
.

[rileyrg]

Error (use-package): mu4e/:catch: Cannot open load file: No such file or
directory, mu4e-meta
Error (use-package): mu4e-alert/:catch: Cannot open load file: No such file
or directory, mu4e-meta

Sigh. I give up. I'll just install from git into
usr/local/share/emacs/site-lisp and be done with it. Thanks anyway.

On Thu, 14 Jan 2021 at 15:42, Richard G. Riley [e-mail] wrote:

thanks. I'll give it a try. )

On Thu, 14 Jan 2021 at 12:56, Nicholas Vollmer [e-mail]
wrote:

Here's what I use. It builds mu in the mu repo and I ise that executable.
You could alter the :pre-build commands to change the prefix during the
configure script and/or run make install. In either case you need to
point the mu4e-mu-binary to the mu executable.

(use-package mu4e :straight ( :host github :files ("mu4e/*") :repo "djcb/mu" :branch "master" :pre-build (("./autogen.sh") ("make"))) :commands (mu4e mu4e-update-index) :custom (mu4e-mu-binary (expand-file-name "mu/mu" (straight--repos-dir "mu")))

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/raxod502/straight.el/issues/491#issuecomment-760150634,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACYTC52YUCEYDM7KE3TIPTSZ3LXXANCNFSM4LAHA5LQ
.

[progfolio]

mu4e-meta is built during the make step. Likely that your :pre-build commands were not run.

Sorry, I should have checked: What version of straight.el are you using?
M-x straight-version?

:pre-build was :build up until recently.

[rileyrg]

Nicholas Vollmer writes:

Sorry, I should have checked: What version of straight.el are you using?
M-x straight-version?

,----
| prerelease (HEAD -> master, origin/master, origin/HEAD) 2d407bc 2020-12-20
| You can run the command ‘straight-version’ with M-x st-ve RET
`----

"self" bootstrapped from scratch.

I'd really like to be reliant just on straight rather than needing to
install mu4e from git I must admit.

[rileyrg]

I should add I can also install from Debian repos but, again, I prefer not
to.

On Thu, 14 Jan 2021 at 16:22, Richard G. Riley [e-mail] wrote:

>

Nicholas Vollmer writes:

Sorry, I should have checked: What version of straight.el are you using?
M-x straight-version?

,----
| prerelease (HEAD -> master, origin/master, origin/HEAD) 2d407bc
2020-12-20
| You can run the command ‘straight-version’ with M-x st-ve RET
`----

"self" bootstrapped from scratch.

I'd really like to be reliant just on straight rather than needing to
install mu4e from git I must admit.

[progfolio]

Ah, I see the issue. You're on the "master" branch. The features required to run the :pre-build commands are on the "develop" branch. There are a few options:

1. Update straight.el to track the "develop" branch. You should be able to use the recipe I provided earlier in that case. Bear in mind this is where development takes place, so you may encounter some bugs. You can always freeze the version of straight to help mitigate this.

2. Use straight.el to clone/install mu4e. Manually run the :pre-build commands in a shell in the mu repo. I did this while implementing the feature which allows running :pre-build commands from a recipe. The disadvantage being you'll have to repeat this process any time you update mu/mu4e.

3. Rely on your OS package manager. I did this before I used straight and, as you've expressed, I wasn't too keen on it- but it works so long as your satisfied with the version of mu the package manager provides. This may be the most painless if you do not have time for any of the previous options at the moment.

[rileyrg]

Changed my mind and put the full recipe back in and manually did the autogen inside straight repos. All good now. And it seems s long as mu4e has a matching mu it doesn't matter if that clashes with the system one that my mbsync systemd service launches. So... many thanks and all good.

[mroll]

@progfolio I was able to build mu4e using your recipe, I just had to figure out how to switch to the develop branch. Thanks for posting that solution!

For anyone who wants to try this build recipe but does not know how to switch straight.el to use the a different branch:

• Before loading the snippet for bootstrapping straight.el, set the variable straight-repository-branch to the name of the branch you'd like to use. The value of the branch name must be a string. For example, to use the develop branch:
  
  
  
  • (setq straight-repository-branch "develop")
  
  
• Delete the contents of ~/.emacs.d/straight/ and restart Emacs to cause straight to download the new branch.