Helm: `helm-buffer--details` too slow on TRAMP buffers

[Fuco1]

In my day job I work mostly over TRAMP and I often have 100+ TRAMP buffers opened. helm-buffer--details calls a lot of file-system hitting functions (file-exists-p, verify-visited-file-modtime, etc...). It takes about 200ms to process one buffer. This makes helm time out (or something similar) and helm-buffers-list is completely unusable.

The simple solution is disable these checks on tramp files (tramp-tramp-file-p) or file-remote-p (not sure if this checks the same thing). As a temporary fix, I have simply commented out the slow branches of the cond in the above mentioned function.

The above could maybe be controlled with some user-customizable flag---I'd rather have usable buffer switching without all the extra details than none at all :)

[thierryvolpiatto]

Matus Goljer [email protected] writes:

In my day job I work mostly over TRAMP and I often have 100+ TRAMP
buffers opened. helm-buffer--details calls a lot of file-system
hitting functions (file-exists-p, verify-visited-file-modtime,
etc...). It takes about 200ms to process one buffer. This makes helm
time out (or something similar) and helm-buffers-list is completely
unusable.

The simple solution is disable these checks on tramp files
(tramp-tramp-file-p) or file-remote-p (not sure if this checks the
same thing). As a temporary fix, I have simply commented out the slow
branches of the cond in the above mentioned function.

The above could maybe be controlled with some user-customizable
flag---I'd rather have usable buffer switching without all the extra
details than none at all :)

Did you try to customize helm-buffer-details-flag'. (you can also toggle it withC-]')

Thierry
Get my Gnupg key:
gpg --keyserver pgp.mit.edu --recv-keys 59F29997

[tuhdo]

I experienced this as well. The details should be enabled when working local, but disabled by default on local. Only when user explicitly enables it through a customize variable, the details will then be processed and be visible every time the command is used. Can this behavior be the default?

[thierryvolpiatto]

Tu Do [email protected] writes:

I experienced this as well. The details should be enabled when working
local, but disabled by default on local. Only when user explicitly
enables it through a customize variable, the details will then be
processed and be visible every time the command is used. Can this
behavior be the default?

What's the problem with toggling rhe var with `C-]' ?

Thierry
Get my Gnupg key:
gpg --keyserver pgp.mit.edu --recv-keys 59F29997

[tuhdo]

It would affect other commands as well, while we only want to disable the feature in this specific Tramp case.

[thierryvolpiatto]

Tu Do [email protected] writes:

It would affect other commands as well,

Which commands ?
Only buffers list is affected by this.

Thierry
Get my Gnupg key:
gpg --keyserver pgp.mit.edu --recv-keys 59F29997

[Fuco1]

That variable does not affect it. The value only changes the behaviour of helm-buffer--show-details which is called _after_ the expensive file-system checks are done. So while the visuals change, the logic behind it does not. This is probably a bug (or rather, badly designed).

However, I still want the details on local buffers, what I don't want is to test remote (tramp) files.

[thierryvolpiatto]

Matus Goljer [email protected] writes:

That variable does not affect it. The value only changes the behaviour
of helm-buffer--show-details which is called after the expensive
file-system checks are done.

Right.

So while the visuals change, the logic behind it does not. This is
probably a bug (or rather, badly designed).

It is not, the costly thing is to show different colors depending on the
state of buffer, not the details.

However, I still want the details on local buffers, what I don't want
is to test remote (tramp) files.

So you will not have the state of the remote buffers.
Fine to me.

Thierry
Get my Gnupg key:
gpg --keyserver pgp.mit.edu --recv-keys 59F29997

[tuhdo]

Turning on/off all buffers is undesirable as well. What I meant is that by default, local buffers still have details but at the same time, the remote buffers (the ones prefixed with @) have their details removed.

And you are right, C-] only turns on/off buffers. Probably I was mistaken with some file commands that use the same helm-ff-show-only-basename variable.

[tuhdo]

And by removing the details, it means that Helm should ignore completely the check for details, as suggested by @Fuco1 , rather than simple cosmetic change.

[tuhdo]

For some reason, when I start having many Tramp buffers, helm-mini or helm-buffers-lists returns an emtpy Helm buffer for me frequently. Then I turned off the detail listing and I did not encounter such situation anymore. I guess disable detail listing helped, you could give it a try to see if it is more responsive.

@Fuco1 I suggest if you currently have to work with many remote buffers, then just use stock switch-to-buffer with helm-mode enabled. It actually does what you want: buffer switching without all the extra details. Of course, you won't have fuzzy matching but stock Helm matching. However, you still have buffer history source at the bottom, so it's still better than stock switch-to-buffer.

[tuhdo]

@Fuco1 By default, Helm replaces C-x C-b with helm-mini, so whenever you want to use helm-buffers-list features, you can use helm-mini through C-x C-b.

[tuhdo]

helm-mini is fast again with Tramp now. Thanks for fixing. But we should give it some time to test.

[tuhdo]

Buffer listing is fine for me now. It's fast with many tramp buffers. @Fuco1 is it ok with you now?

[pclouds]

I'm using 20150504 from melpa and helm-mini is still slow for me when tramp buffers are opened. It takes a few seconds (even with C-] to turn off detail mode). That's too long for fast switching between buffers. I have recentf in helm-mini but I don't think it matters here. Is there any switch to avoid tramp in helm-mini?

[tuhdo]

@pclouds You can set helm-buffer-skip-remote-checking to t and no remote checking for buffer is performed and there will be no more lag.

[pclouds]

Perfect! Thank you!

[pclouds]

Actually I it's still slow once in a while, then the following helm-mini is fast again. If I kill all those "_tramp/ssh _ buffers then it's always fast. Is it possible that helm-mini look into these buffers or something, that makes tramp contact the remote side?

[thierryvolpiatto]

Set tramp-verbose to 6.
Set helm-buffer-skip-remote-checking to t.
Now switch to one of the tramp debug buffers ("_debug tramp/ssh user@host_") in a full window.
Move the point at end of buffer.
Now run helm-mini.
If tramp is involved, you will see the buffer growing, and you will know what is going on.

[pclouds]

There are tramp activities even when helm-buffer-ski.. is t..

11:45:30.513828 tramp-do-file-attributes-with-stat (5) # file attributes with stat: /path/to/file
11:45:30.514346 tramp-send-command (6) # ( (test -e /path/to/file || test -h /path/to/file) && \stat -c '(("%N") %h "%U" "%G" %Xe0 %Ye0 %Ze0 %se0 "%A" t %ie0 -1)' /path/to/file || echo nil) 2>/dev/null; echo tramp_exit_status $?
11:45:31.350298 tramp-wait-for-regexp (6) # 
(("`/path/to/file'") 1 "username" "groupname" 1431483914e0 1274184003e0 1424840897e0 1955e0 "-r--r--r--" t 257064e0 -1)
tramp_exit_status 0
///942db2a41325592f6326284b3aed29a5#$
11:45:31.355274 tramp-do-file-attributes-with-stat (5) # file attributes with stat: /path/to/file2
11:45:31.356051 tramp-send-command (6) # ( (test -e /path/to/file2 || test -h /path/to/file2) && \stat -c '(("%N") %h "%U" "%G" %Xe0 %Ye0 %Ze0 %se0 "%A" t %ie0 -1)' /path/to/file2 || echo nil) 2>/dev/null; echo tramp_exit_status $?
11:45:31.784243 tramp-wait-for-regexp (6) # 
(("`/path/to/file2'") 1 "username" "groupname" 1431483904e0 1274184003e0 1424840897e0 2164e0 "-r--r--r--" t 257065e0 -1)
tramp_exit_status 0
///942db2a41325592f6326284b3aed29a5#$
11:45:31.788479 tramp-do-file-attributes-with-stat (5) # file attributes with stat: /path/to/file3
11:45:31.789159 tramp-send-command (6) # ( (test -e /path/to/file3 || test -h /path/to/file3) && \stat -c '(("%N") %h "%U" "%G" %Xe0 %Ye0 %Ze0 %se0 "%A" t %ie0 -1)' /path/to/file3 || echo nil) 2>/dev/null; echo tramp_exit_status $?
11:45:32.198584 tramp-wait-for-regexp (6) # 
(("`/path/to/file3'") 1 "username" "groupname" 1431479518e0 1429153950e0 1429167552e0 86226e0 "-rw-r--r--" t 243202e0 -1)
tramp_exit_status 0
///942db2a41325592f6326284b3aed29a5#$
11:45:32.201716 tramp-do-file-attributes-with-stat (5) # file attributes with stat: /path/to/file4
11:45:32.202272 tramp-send-command (6) # ( (test -e /path/to/file4 || test -h /path/to/file4) && \stat -c '(("%N") %h "%U" "%G" %Xe0 %Ye0 %Ze0 %se0 "%A" t %ie0 -1)' /path/to/file4 || echo nil) 2>/dev/null; echo tramp_exit_status $?
11:45:32.608360 tramp-wait-for-regexp (6) # 
(("`/path/to/file4'") 1 "username" "groupname" 1431478567e0 1426119664e0 1429167552e0 6615e0 "-rw-r--r--" t 243091e0 -1)
tramp_exit_status 0
///942db2a41325592f6326284b3aed29a5#$

[thierryvolpiatto]

Nguyễn Thái Ngọc Duy [email protected] writes:

There are tramp activities even when helm-buffer-ski.. is t..

11:45:30.513828 tramp-do-file-attributes-with-stat (5) # file attributes with stat: /path/to/file

Ok it is a call to file-attributes somewhere, which sources are you
using with helm-mini ?

Thierry
Get my Gnupg key:
gpg --keyserver pgp.mit.edu --recv-keys 59F29997

[pclouds]

Not sure how to lists sources. In helm-mini I see three sections: buffers, recentf and create buffer. This is a grep from my .emacs

 13: '(helm-buffer-skip-remote-checking t)
 14: '(helm-ff-file-name-history-use-recentf t)
 15: '(helm-ff-tramp-not-fancy t)
 16: '(helm-mode-reverse-history nil)
117:(require 'helm-config)
118:(helm-mode 1)
119:(eval-after-load 'helm-mode
120:  '(add-to-list 'helm-completing-read-handlers-alist '(find-file)))
122:(define-key helm-map (kbd "<tab>") 'helm-execute-persistent-action)
124:(global-set-key '[C-tab] 'helm-mini)
366:(global-set-key (kbd "M-i") 'helm-imenu)

[thierryvolpiatto]

Nguyễn Thái Ngọc Duy [email protected] writes:

Not sure how to lists sources. In helm-mini I see three sections:
buffers, recentf and create buffer.

Hmm, with this same sources in helm-mini, I have no tramp activities
here, so I wonder if something in your packages installed is not calling
file-attributes somewhere (i.e something else than helm).
Can you try to reproduce with ./emacs-helm.sh and
helm-buffer-skip-remote-checking set ?
Don't forget to set tramp-verbose to 6 once in ./emacs-helm.sh.

Thierry
Get my Gnupg key:
gpg --keyserver pgp.mit.edu --recv-keys 59F29997

[pclouds]

./emacs-helm.sh runs cleanly. It turns out I have another recentf.el in my load-path and it must have conflicted with the one from emacs distribution. Thank you for the support and sorry for all this noise.