Straight.el: Cannot install Org on windows

[Whil-]

What's wrong

Cannot install Org on Windows 10.

Directions to reproduce

• (temporarily) reset your Emacs env. by moving ~/.emacs.d somewhere.
• Create ~/.emacs.d/init.el with the following content

(defvar bootstrap-version)
(let ((bootstrap-file
       (expand-file-name "straight/repos/straight.el/bootstrap.el" user-emacs-directory))
      (bootstrap-version 5))
  (unless (file-exists-p bootstrap-file)
    (with-current-buffer
        (url-retrieve-synchronously
         "https://raw.githubusercontent.com/raxod502/straight.el/develop/install.el"
         'silent 'inhibit-cookies)
      (goto-char (point-max))
      (eval-print-last-sexp)))
  (load bootstrap-file nil 'nomessage))

(straight-use-package 'org)

• start emacs
• observe *Warnings* buffer popping up with the following content:

Warning (straight): Failed to run "make"; see buffer *straight-process*

Additional comments

It seems a dependency on make has been inserted somewhere. This might be okay on Linux but on Windows that will be a blocker for many. If the dependency is chosen to still be kept, then at least update the readme with a statement about that?

Version information

• Emacs version: 27.1
• Operating system: Windows 10

[Whil-]

To add to this, even when installing make the default installation of Emacs creates whitespace-issues it seems...

From the *straight-process* buffer:

$ cd "c:/Users/Whil-/.emacs.d/straight/repos/org/"
$ "make" "autoloads" "EMACS=c:/Program Files/Emacs/x86_64/bin/emacs"

======================================================
= Invoke "make help" for a synopsis of make targets. =
= Created a default local.mk template.               =
= Setting "oldorg" as the default target.            =
= Please adapt local.mk to your local setup!         =
======================================================
/usr/bin/sh: c:/Program: No such file or directory
make: [mk/targets.mk:98: local.mk] Error 127 (ignored)
c:/ProgramData/chocolatey/lib/make/tools/install/bin/make.exe -C lisp autoloads
make[1]: Entering directory 'c:/Users/Whil-/.emacs.d/straight/repos/org/lisp'
rm -f org-version.el org-loaddefs.el org-version.elc org-loaddefs.elc org-install.elc
org-version: 9.4.4 (release_9.4.4-253-g4d463e)
/usr/bin/sh: c:/Program: No such file or directory
make[1]: *** [Makefile:72: org-version.el] Error 127
make[1]: Leaving directory 'c:/Users/Whil-/.emacs.d/straight/repos/org/lisp'
make: *** [mk/targets.mk:141: autoloads] Error 2

[Whil-]

(To not leave any threads hanging, and for those wondering - Moving the installation path of Emacs to a location without spaces does solve the is issue. Ofc. still dependent on make.)

[progfolio]

Thank you for the bug report.

It seems a dependency on make has been inserted somewhere. This might be okay on Linux but on Windows that will be a blocker for many. If the dependency is chosen to still be kept, then at least update the readme with a statement about that?

The dependency for make has always existed for the development version of Org mode. What has changed on straight's end is that we are properly building Org mode now, instead of relying on a hack which lead to a mixed installation and other subtle bugs. I don't think the README is the appropriate place for such a notice, as we would need to add notices for any package that has similar dependencies.
It would make more sense to add a check for make and a better error message to the org recipes to make that dependency clearer.

If you are aware of another method to install the development version of Org on a Windows system (I do not use Windows, so I am not familiar with any myself), please let me know and I will amend the recipe to use that method.

I see you are using chocolatey's version of make. Your shell is splitting the Emacs executable's path at the first space. We're using call-process to directly invoke make from Emacs, so I was under the impression that shell-quoting that argument was not necessary. That may be an incorrect assumption as far as Windows is concerned. I will look into that and patch it if necessary.

[Whil-]

The dependency for make has always existed for the development version of Org mode. What has changed on straight's end is that we are properly building Org mode now, instead of relying on a hack which lead to a mixed installation and other subtle bugs. I don't think the README is the appropriate place for such a notice, as we would need to add notices for any package that has similar dependencies.
It would make more sense to add a check for make and a better error message to the org recipes to make that dependency clearer.

I've used Org mode and Straight on Windows for quite some time now and it's only now, when I tried a clean install, this issue came up. Maybe there is a "halfway happy" version?

If multiple packages rely on external software I do believe all those dependencies should be explicit in the documentation. It could be under one common heading in the readme, just specifying that "This list of software are used for some packages and not having them installed will likely lead to breakages". And then maybe a general Windows instruction on how to go about that. Since Linux utilities on Windows can be a bit obscure to configure.

This is just one persons opinion of course.

The shell-quote fix is welcome, thanks!

[progfolio]

I've used Org mode and Straight on Windows for quite some time now and it's only now, when I tried a clean install, this issue came up. Maybe there is a "halfway happy" version?

As I mentioned, at one point straight worked around properly building Org. It may have worked for you, but the way it was set up lead to mixed installations and subtle bugs for other users. The correct way to handle it is to properly build Org (via make).

If multiple packages rely on external software I do believe all those dependencies should be explicit in the documentation

The dependency is Org's and it is explicit in their documentation:

Note that in this case, ‘make autoloads’ is mandatory: it defines Org’s version in ‘org-version.el’ and Org’s autoloads in ‘org-loaddefs.el’.

https://orgmode.org/manual/Installation.html

If multiple packages rely on external software I do believe all those dependencies should be explicit in the documentation. It could be under one common heading in the readme, just specifying that "This list of software are used for some packages and not having them installed will likely lead to breakages". And then maybe a general Windows instruction on how to go about that. Since Linux utilities on Windows can be a bit obscure to configure.

IMO, this should be the responsibility of the recipe author. Now in this case, we have authored the Org recipe. I can add a check for the make executable to the recipe and issue an error when it is absent. However, it would be a maintenance burden to try to amass a list of software that might be used by some packages and how to install/configure them for various operating systems. This is better suited for something like a tutorial article or wiki rather than a README.

The shell-quote fix is welcome, thanks!

Would you mind testing to see if this patch works on your system?
You can evaluate the following test case in your *scratch* buffer.
It should install straight.el and Org in a clean, temporary, testing environment.
Once it is done running, the *straight-bug-report-process* buffer should pop up with the results, which can be copied into a comment here.

Test Case

(straight-bug-report
  :user-dir "shell-quote-emacs.test"
  :pre-bootstrap 
  (setq straight-repository-user "progfolio" straight-repository-branch "fix/org-recipe-windows-shell-quote")
  :post-bootstrap 
  (straight-use-package 'org-plus-contrib)
  (org-version nil 'full 'message))

• Test run at: 2021-03-05 17:50:08
• system-type: gnu/linux
• straight-version: prerelease (HEAD -> fix/org-recipe-windows-shell-quote, fork/fix/org-recipe-windows-shell-quote) b46ce36 2021-03-05
• emacs-version: GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo version 1.17.4, Xaw3d scroll bars) of 2021-03-04

Output

Bootstrapping straight.el...
Bootstrapping straight.el...done
Rebuilding all packages due to build cache schema change
Looking for gnu-elpa-mirror recipe → Cloning melpa...
Looking for gnu-elpa-mirror recipe → Cloning melpa...done
Looking for emacsmirror-mirror recipe → Cloning gnu-elpa-mirror...
Looking for emacsmirror-mirror recipe → Cloning gnu-elpa-mirror...done
Looking for emacsmirror-mirror recipe → Cloning el-get...
Looking for emacsmirror-mirror recipe → Cloning el-get...done
Looking for straight recipe → Cloning emacsmirror-mirror...
Looking for straight recipe → Cloning emacsmirror-mirror...done
Building straight...
Building straight...done

Test run with version: prerelease (HEAD -> fix/org-recipe-windows-shell-quote, origin/fix/org-recipe-windows-shell-quote) b46ce36 2021-03-05
Cloning org (for org-plus-contrib)...
Cloning org (for org-plus-contrib)...done
Building org-plus-contrib...
Building org-plus-contrib...done

Org mode version 9.4.4 (release_9.4.4-253-g4d463e @ /tmp/shell-quote-emacs.test/straight/build/org-plus-contrib/)

[progfolio]

Scratch that test case, I've tested it on a win 10 VM and the problem persists.
I'll see if I can think of a way around this.

[progfolio]

OK. I've implemented a way to build org-version.el without relying on make. I've tested on a Win 10 VM, and everything seems to be working fine. If you don't mind, please run the following test case in your Emacs (as described above) and paste the results in a comment here.

Test Case

(straight-bug-report
  :user-dir "org-no-make"
  :pre-bootstrap 
  (setq straight-repository-user "progfolio" straight-repository-branch "fix/org-no-make")
  :post-bootstrap 
  (straight-use-package 'org-plus-contrib)
  (org-version nil 'full 'message))

• Test run at: 2021-03-05 20:46:12
• system-type: windows-nt
• straight-version: prerelease (HEAD -> master, origin/master, origin/HEAD) d21e829 %cs
• emacs-version: GNU Emacs 27.1 (build 1, x86_64-w64-mingw32) of 2020-08-21

Output

Rebuilding all packages due to build cache schema change
Looking for gnu-elpa-mirror recipe -> Cloning melpa...
Looking for gnu-elpa-mirror recipe -> Cloning melpa...done
Looking for emacsmirror-mirror recipe -> Cloning gnu-elpa-mirror...
Looking for emacsmirror-mirror recipe -> Cloning gnu-elpa-mirror...done
Looking for emacsmirror-mirror recipe -> Cloning el-get...
Looking for emacsmirror-mirror recipe -> Cloning el-get...done
Looking for straight recipe -> Cloning emacsmirror-mirror...
Looking for straight recipe -> Cloning emacsmirror-mirror...done
Building straight...
Building straight...done
Cloning org (for org-plus-contrib)...
Cloning org (for org-plus-contrib)...done
Building org-plus-contrib...
Building org-plus-contrib...done
Bootstrapping straight.el...
Bootstrapping straight.el...done

Test run with version: prerelease
 nil

Org mode version release_9.4.4 (release_9.4.4-g4d463e @ c:/Users/win10/AppData/Local/Temp/org-no-make/straight/build/org-plus-contrib/)

[Whil-]

On a different PC than when the bug was reported. But I've reproduced it with the same instructions as the bug report, except a different emacs version.

Test Case

(straight-bug-report
  :user-dir "org-no-make"
  :pre-bootstrap 
  (setq straight-repository-user "progfolio" straight-repository-branch "fix/org-no-make")
  :post-bootstrap 
  (straight-use-package 'org-plus-contrib)
  (org-version nil 'full 'message))

• Test run at: 2021-03-06 10:51:11
• system-type: windows-nt
• straight-version: prerelease nil
• emacs-version: GNU Emacs 28.0.50 (build 1, x86_64-w64-mingw32) of 2021-01-15

Output

Rebuilding all packages due to build cache schema change
Looking for gnu-elpa-mirror recipe -> Cloning melpa...
Looking for gnu-elpa-mirror recipe -> Cloning melpa...done
Looking for emacsmirror-mirror recipe -> Cloning gnu-elpa-mirror...
Looking for emacsmirror-mirror recipe -> Cloning gnu-elpa-mirror...done
Looking for emacsmirror-mirror recipe -> Cloning el-get...
Looking for emacsmirror-mirror recipe -> Cloning el-get...done
Looking for straight recipe -> Cloning emacsmirror-mirror...
Looking for straight recipe -> Cloning emacsmirror-mirror...done
Building straight...
Building straight...done
Cloning org (for org-plus-contrib)...
Cloning org (for org-plus-contrib)...done
Building org-plus-contrib...
Building org-plus-contrib...done
Bootstrapping straight.el...
Bootstrapping straight.el...done

Test run with version: prerelease
 nil

Org mode version release_9.4.4 (release_9.4.4-g4d463e @ c:/Users/Gustav/AppData/Local/Temp/org-no-make/straight/build/org-plus-contrib/)

[progfolio]

I've merged the recipe change into the "develop" branch. Please try rebuilding Org using this branch of straight.el and let me know the results.

[progfolio]

Not quite there yet. You should be able to build Org now, but there is a slight bug with org-version. I will fix this tonight.

[progfolio]

This should be good now. I merged a patch yesterday to "develop". Testing appreciated if you have time.

[Whil-]

Tested according to the bug-report above., Only difference is the usage of develop branch instead of master. So with a clean environment, Emacs in the default Program Files folder, no make, and the configuration below, it all worked!

(setq straight-repository-branch "develop")
(defvar bootstrap-version)
(let ((bootstrap-file
       (expand-file-name "straight/repos/straight.el/bootstrap.el" user-emacs-directory))
      (bootstrap-version 5))
  (unless (file-exists-p bootstrap-file)
    (with-current-buffer
        (url-retrieve-synchronously
         "https://raw.githubusercontent.com/raxod502/straight.el/develop/install.el"
         'silent 'inhibit-cookies)
      (goto-char (point-max))
      (eval-print-last-sexp)))
  (load bootstrap-file nil 'nomessage))

(straight-use-package 'org)

Great!

[cwhatley]

FYI, the change in #708 has broken my ability to build org on emacs on a mac with ZSH as my shell because of how ZSH handles globbing.

Perhaps straight-recipes-org-elpa--build could be a windows-only operation?

Core issue:

(let ((shell-file-name "/bin/zsh")
      (default-directory "~/.emacs.d/straight/repos/org"))
  (prin1 (shell-command-to-string "git describe --match release\* --abbrev=0 HEAD"))
  (prin1 (shell-command-to-string "noglob git describe --match release\* --abbrev=0 HEAD"))
  (setq shell-file-name "/bin/bash")
  (prin1 (shell-command-to-string "git describe --match release\* --abbrev=0 HEAD"))
  nil
  )
"zsh:1: no matches found: release*
""release_9.4.4
""release_9.4.4

(edit: I pasted the wrong output before)

[progfolio]

We should be good with the recipe that's on "develop" now. Closing this issue, but please feel free to comment if the issue persists.

[nivpgir]

I just hit the same problem, solved it with this workaround:
``emacs lisp (defconst org-plus-contrib-fixed-recipe (org-plus-contrib
:type git
:repo "https://code.orgmode.org/bzg/org-mode.git"
:local-repo "org"
:depth full
:pre-build ,(list
(concat
(when (eq system-type 'berkeley-unix) "g")
"make")
"autoloads"
;; changes are here (extra "'" before directory and at the end
(concat "EMACS=" "'" invocation-directory invocation-name "'"))
:build
(:not autoloads)
:files
(:defaults "lisp/.el"
("etc/styles/" "etc/styles/")
"contrib/lisp/*.el")
:includes org))

(straight-override-recipe org-plus-contrib-fixed-recipe)
```
I think I'll stick with my solution because I don't want to use development just for this fix.
I hope this will be useful to others that run into this issue.