Straight.el: Emacs freeze when using `watchexec` to improve startup

[babygau]

What's wrong

emacs freeze when scrolling down after use this snippet

Directions to reproduce

Add the following snippet in init.el before straight.el bootstrap

 (if (and (executable-find "watchexec")
          (executable-find "python3"))
     (setq straight-check-for-modifications '(watch-files find-when-checking))
   (setq straight-check-for-modifications
         '(find-at-startup find-when-checking)))

Open Emacs, scrolling down until screen refresh, Emacs will freeze and have to kill it.

Version information

• Emacs version: 28.0.5
• Operating system: macOS 10.15.6

[progfolio]

Which case is the freeze occurring for? It would be helpful to know what the value of straight-check-for-modifications is when you experience the freeze.

[babygau]

@progfolio It's my bad. I couldn't reproduce the problem with new updated gccemacs. So I think that I used a "bad-git-commit" compiled gccemacs at the time.

[babygau]

@progfolio, now it happened again. Same snippet:

 (if (and (executable-find "watchexec")
          (executable-find "python3"))
     (setq straight-check-for-modifications '(watch-files find-when-checking))
   (setq straight-check-for-modifications
         '(find-at-startup find-when-checking)))

FYI, I'm using evil-mode, so moving around, make some random edit then Emacs freeze. I opened new terminal, accessed freezing emacs with kill -USR2 pid, and it showed this bug

If I comment out the above snippet, the issue's gone.

Please let me know if you need any further information

[progfolio]

Could you try evaluating this in your *scratch* buffer?:

(straight-bug-report
  :pre-bootstrap
  (if (and (executable-find "watchexec")
           (executable-find "python3"))
      (setq straight-check-for-modifications '(watch-files find-when-checking))
    (setq straight-check-for-modifications
          '(find-at-startup find-when-checking)))
  :post-bootstrap
  (message "straight-check-for-modifications: %S" straight-check-for-modifications))

It will bootstrap straight in a clean environment and give us some more information.
After the test is run, a window should open with the output. You can paste that entire window's content here.

[babygau]

@progfolio Pls have a look

[babygau]

In shell, I link the compiled gccemacs with

# Need this if I use `gccemacs`
if [[ "$OSTYPE" == "darwin"* ]]; then
  if [ -f "/Applications/Emacs.app/Contents/MacOS/Emacs" ]; then
    alias edaemon="/Applications/Emacs.app/Contents/MacOS/Emacs --daemon"
    alias emacs="/Applications/Emacs.app/Contents/MacOS/Emacs -nw"
  fi

  if [ -f "/Applications/Emacs.app/Contents/MacOS/bin/emacsclient" ]; then
    alias ekill="/Applications/Emacs.app/Contents/MacOS/bin/emacsclient -nw -e '(kill-emacs)'"
    alias e="/Applications/Emacs.app/Contents/MacOS/bin/emacsclient -nw -a=''"
  fi
fi

[progfolio]

I see. You can add a key val pair to that bug-report to specify the executable path:

(straight-bug-report
  :executable "/Applications/Emacs.app/Contents/MacOS/Emacs"
  :pre-bootstrap
  (if (and (executable-find "watchexec")
           (executable-find "python3"))
      (setq straight-check-for-modifications '(watch-files find-when-checking))
    (setq straight-check-for-modifications
          '(find-at-startup find-when-checking)))
  :post-bootstrap
  (message "straight-check-for-modifications: %S" straight-check-for-modifications))

[babygau]

Here you are

Test Case

(straight-bug-report
  :executable "/Applications/Emacs.app/Contents/MacOS/Emacs"
  :pre-bootstrap 
  (if
      (and
       (executable-find "watchexec")
       (executable-find "python3"))
      (setq straight-check-for-modifications
            '(watch-files find-when-checking))
    (setq straight-check-for-modifications
          '(find-at-startup find-when-checking)))
  :post-bootstrap 
  (message "straight-check-for-modifications: %S" straight-check-for-modifications))

• Test run at: 2020-10-05 16:09:54
• system-type: darwin
• straight-version: prerelease (HEAD -> develop, origin/develop) 64e5361 2020-10-01
• emacs-version: GNU Emacs 28.0.50 (build 1, x86_64-apple-darwin19.6.0, NS appkit-1894.60 Version 10.15.7 (Build 19H2)) of 2020-10-03

Output

Bootstrapping straight.el...
Bootstrapping straight.el...done
Rebuilding all packages due to build cache schema change
Looking for gnu-elpa-mirror recipe → Cloning melpa...
Looking for gnu-elpa-mirror recipe → Cloning melpa...done
Looking for emacsmirror-mirror recipe → Cloning gnu-elpa-mirror...
Looking for emacsmirror-mirror recipe → Cloning gnu-elpa-mirror...done
Looking for straight recipe → Cloning emacsmirror-mirror...
Looking for straight recipe → Cloning emacsmirror-mirror...done
Building straight...
Building straight...done
Setting up filesystem watcher...
Setting up filesystem watcher...done

Test run with version: prerelease (HEAD -> develop, origin/develop) 64e5361 2020-10-01
straight-check-for-modifications: (watch-files find-when-checking)

[progfolio]

Thank you. Another thing to try:

(straight-bug-report
  :interactive t
  :executable "/Applications/Emacs.app/Contents/MacOS/Emacs"
  :pre-bootstrap 
  (if
      (and
       (executable-find "watchexec")
       (executable-find "python3"))
      (setq straight-check-for-modifications
            '(watch-files find-when-checking))
    (setq straight-check-for-modifications
          '(find-at-startup find-when-checking)))
  :post-bootstrap 
  (message "straight-check-for-modifications: %S" straight-check-for-modifications))

This will start an interactive Emacs session in a clean environment.
Your init won't be loaded, (and so no Evil), but can you see if you can reproduce the problem by moving around in that environment?

If you are able to get it to freeze, send the same USR2 signal and copy the entire output of the backtrace buffer here.

[babygau]

@progfolio I coulnd't reproduce the error. So it's not straight.el bug, right?

[progfolio]

I'm not 100% sure yet. If you go back to your set-up and trigger the freeze, does pressing C-g unfreeze Emacs? (You may have to press it several times)

If it does, you can M-x toggle-debug-on-quit before the freeze and you should get a similar backtrace to what you posted in the image earlier. I'd be interested to see the full output of that backtrace buffer. You can also step through that using the d key to see what's being executed. This may give a clue as to what is causing the freeze. (you can still step through the backtrace with the SIG USR2 method as well if C-g does not work)

[babygau]

This is what happened when I pressed C-g after emacs frozen

[progfolio]

According to: https://www.gnu.org/software/emacs/manual/html_node

Saying y to ‘Auto-save?’ causes immediate auto-saving of all modified buffers in which auto-saving is enabled. Saying n skips this. This question is omitted if Emacs is in a state where auto-saving cannot be done safely.

Saying y to ‘Abort (and dump core)?’ causes Emacs to crash, dumping core. This is to enable a wizard to figure out why Emacs was failing to quit in the first place. Execution does not continue after a core dump.

If you answer this question n, Emacs execution resumes. With luck, Emacs will ultimately do the requested quit. If not, each subsequent C-g invokes emergency escape again.

If Emacs is not really hung, just slow, you may invoke the double C-g feature without really meaning to. Then just resume and answer n to both questions, and you will get back to the former state. The quit you requested will happen by and by.

So I'd skip the auto-save unless necessary and skip the core-dump to see if Emacs resumes or is still frozen.

[babygau]

I did, and I couldn't resume Emacs because it gave me a frozen blank screen. Here is what I could do with debugger. Pls let me know if you need anything I could help.

[progfolio]

Thank you. It's late for me, so I'll have to look at this tomorrow.

[babygau]

@progfolio you're welcom. Thank you for your time ❤️

[raxod502]

Sorry, that looks like you've found a pretty low-level bug which might be difficult to get to the bottom of. It might well be something we could address in straight.el, but the problem is definitely happening way deeper in Emacs internals than any straight.el-specific code.

You could have a look at the straight.el code related to the filesystem watcher. Basically, what enabling that option does is spawn a watchexec subprocess. Apparently, having that subprocess spawned causes the freeze that you are observing. Not sure where to go from there... :/

[babygau]

@raxod502 thanks for your clarification. If that is the case, I believe it is because I'm using gccemacs which is supposed to be error prone. I could not decide to close this issue or not. So please help me do it if you think so. Have a great day!

[raxod502]

I guess I'll close it, and if anybody has ideas about how the problem could be something addressable in straight.el then leave a comment and I can re-open.