Selectrum: Some enhancement proposals for narrowing incremental search framework

[gnusupport]

;; 0. Function (YOU-NAME-THE-MODE-completing-read PROMPT COLLECTION &OPTIONAL ...)
;;    and possibility to automatically replace the built-in function named
;;    `completing-read' when `YOU-NAME-THE-MODE-mode' is turned on
;;
;; 1. It should get selection by using words in reverse order, for
;;    example to select "AMERICAN SAMOA" when one writes "SAMOA
;;    AMERICAN". Could be customizable like ivy. Default in helm.
;;
;; 2. Highlighting like in Ivy or Helm or Dmenu
;;
;; 3. Capability to choose other actions, not only the default action,
;;    like M-o in Ivy or TAB in Helm.
;;
;; 4. Capability to choose multiple items, like marking of the item
;;    with C-c SPC in Helm and conducting actions on multiple items
;;
;; 5. Capability to complete at various places, reference to ivy here,
;;    completing in a pop-up, completing inside of
;;    minibuffer,completing in a window like helm, although Ivy
;;    supports the same.
;;
;; 6. Customizable option for full window with mini buffer, not only
;;    half window or not only minibuffer.
;;
;; Some of those features may already be in selectrum. When those
;; features are within narrowing incremental completion framework,
;; such allow for many other applications to be easier developed.

[clemera]

Thanks for the suggestions. Here are my thoughts, I think the following points are already solved:

1. For filtering you can set selectrum-refine-candidates-function and there are packages with nice filtering capabilities you can use like orderless and prescient. See the README and wiki for setup instructions.
2. For highlighting you can set selectrum-highlight-candidates-function this is typically also handled by the package which provides the filtering.
3. For this you can use the embark package, see the wiki for setup instructions to integrate it with selectrum.
4. This can be done using completing-read-multiple anything else wouldn't be compatible with the completion API.
5. You can use the mini-frame package for this.

The following points weren't clear to me or may need more discussion:

1. What would be the purpose of this? You can use completing-read and selectrum will be used when selectrum-mode is active
2. You can control the height by setting selectrum-num-candidates-displayed and maybe also adjusting max-mini-window-height if you run into problems please open a separate issue describing your setup. Using another buffer would be a separate issue I think but I would be open to this idea, Selectrum could have an option to show completions in some other buffer.

[gnusupport]

• Clemens Radermacher [e-mail] [2020-11-07 10:49]:
  > Thanks for the suggestions. Here are my thoughts, I think the following points are already solved:
  >
  > 1. For filtering you can set selectrum-refine-candidates-function
  > and there are packages with nice filtering capabilities you can use
  > like orderless and prescient. See the README and wiki for setup
  > instructions.

Thank you. Yesterday I found out about orderless.el

In my opinion, feature to search words without order should be bundled
in completion package, basically in all of them.

For helm I do not know what it uses, but I know it works.

For ivy I just set this below:

(setq ivy-re-builders-alist
'((t . ivy--regex-ignore-order)))

then I can use reverse order of words.

1. For highlighting you can set
   selectrum-highlight-candidates-function this is typically also
   handled by the package which provides the filtering.

I understand I could expand use of any package but that does not make
package itself more useful, it makes external programs useful.

1. For this you can use the embark package, see the wiki for setup
   instructions to integrate it with selectrum.

I hope you understand how my proposal is directed to enhance
selectrum, not really to use selectrum with other packages or that I
use selectrum with other packages.

1. This can be done using completing-read-multiple anything else
   wouldn't be compatible with the completion API.

You are right, I have not expressed myself well. For example
`ivy-completing-read' does not support multiple choices, I don't think
so.

Byt `ivy-read' have something similar:

(ivy-read PROMPT COLLECTION &key PREDICATE REQUIRE-MATCH INITIAL-INPUT
HISTORY PRESELECT DEF KEYMAP UPDATE-FN SORT ACTION MULTI-ACTION UNWIND
RE-BUILDER MATCHER DYNAMIC-COLLECTION EXTRA-PROPS CALLER)

Then I can at least define multi-action to act on multiple choices.

1. You can use the mini-frame package for this.

I am already using something else. Enhancement proposals are directed
to selectrum, they need not be adopted of course.

The following points weren't clear to me or may need more discussion:

1. What would be the purpose of this? You can use completing-read
   and selectrum will be used when selectrum-mode is active

You are right that it does not make much of difference. It is just for
choice making for those uses that I have personally, so it is not
useful for public. I can turn on (selectrum-mode 1) when I need it.

1. You can control the height by setting
   selectrum-num-candidates-displayed and maybe also adjusting

That increases minibuffer height. It confuses users who are supposed
to enter their strings into mini buffer. Users of application are not
experienced Emacs users or developers.

Of course it is matter of choice. I can say helm does it right, it is
my taste, as helm does not remove the mode line up-down when searching
for something. In general almost all Emacs functions never remove
modeline up-down neither enlarge mini buffer. Then ivy, icomplete,
selectrum, they enlarge mini buffer. Imagine that as surprise. It is
also contrary to what it is "mini" buffer.

max-mini-window-height if you run into problems please open a
separate issue describing your setup. Using another buffer would be
a separate issue I think but I would be open to this idea, Selectrum
could have an option to show completions in some other buffer.

Feature is not built-in into Ivy but yesterday on emacs-devel somebody
pointed it to me and I just need to configure it into
ivy-display-functions-alist but that does not solve the problem that
package itself does not have that enhancement built-in

(defun ivy-display-function-window (text)
  (let ((buffer (get-buffer-create "*ivy-candidate-window*"))
        (str (with-current-buffer (get-buffer-create " *Minibuf-1*")
               (let ((point (point))
                     (string (concat (buffer-string) "  " text)))
                 (add-face-text-property
                  (- point 1) point 'ivy-cursor t string)
                 string))))
    (with-current-buffer buffer
      (let ((inhibit-read-only t))
        (erase-buffer)
        (insert str)))
    (with-ivy-window
      (display-buffer
       buffer
       `((display-buffer-reuse-window
          display-buffer-below-selected)
         (window-height . ,(1+ (ivy--height (ivy-state-caller ivy-last)))))))))

Our eyes are on minibuffer. Focus is on minibuffer. Why jump up down
with focus of the user? Helm is doing it right in that sense, it does
not disturb minibuffer by default.

[clemera]

I hope you understand how my proposal is directed to enhance
selectrum, not really to use selectrum with other packages or that I
use selectrum with other packages.

Selectrum follows a different approach, instead of incorporating all those features into Selectrum itself one can use third party packages to get certain features. The idea is to have composable extensions instead of a huge monolith which handles everything out of the box. See #174 for a proposal to make configuration (and out of the box experience) for users more convenient.

Our eyes are on minibuffer. Focus is on minibuffer. Why jump up down
with focus of the user? Helm is doing it right in that sense, it does
not disturb minibuffer by default.

Having the option to use a different buffer is doable and I think is a good suggestion.

[raxod502]

Yes, as @clemera said it is a feature, not a bug, that Selectrum does not attempt to be a replacement for every other completion-related Emacs package at the same time. Any modern computer system is the result of many different pieces of software running in concert, and there are advantages to modularity.

It is also a feature, not a bug, that Selectrum displays candidates in the minibuffer. I understand that not everyone likes this behavior; traditionally, such people have used Helm. It seems like it could be reasonably simple to make this behavior customizable, so I suppose I would have no objection (although I am at least mildly wary of Selectrum attempting to become all things to everyone).

[leungbk]

It is also a feature, not a bug, that Selectrum displays candidates in the minibuffer. I understand that not everyone likes this behavior; traditionally, such people have used Helm. It seems like it could be reasonably simple to make this behavior customizable, so I suppose I would have no objection (although I am at least mildly wary of Selectrum attempting to become all things to everyone).

An alternative is to wait for someone to make a selectrum-posframe package, allowing the candidates to be displayed at a location of the user's choosing.

[clemera]

@leungbk Just in case you missed it in the discussion above you can already use mini-frame.

[clemera]

With #230 you should be able to display candidates in any window you want. I also added a selectrum-display-full-frame action that I liked to use myself in helm from time to time. Give it a try if you like and let me know if you run into problems.

[gnusupport]

• Clemens Radermacher [e-mail] [2020-11-10 22:53]:
  > With #230 you should be able to display candidates in any window you
  > want. I also added a selectrum-display-full-frame action that I
  > liked to use myself in helm from time to time. Give it a try and let
  > me know if you run into problems.

I appreciate that. I will look into that right now.

You may see here comparison of ivy and helm within video (more on the
end), it is 17M video:

https://gnu.support/images/2020/11/2020-11-10/2020-11-10-20:10:49.ogv

• ivy is more useful as it is directly in GNU ELPA. I have many
  computers, many users and I like that when program is invoked that
  packages may be downloaded from built-in GNU ELPA. I prefer having
  such feature inside of Emacs.

  It abuses mini buffer, as minibuffer is not mini anymore. Same with
  Selectrum unless this feature you implemented remedies it.

  It removes modline. That is side effect that nobody mentions. Manual
  explains what is modeline:

  1.3 The Mode Line

  At the bottom of each window is a “mode line”, which describes what
  is going on in the current buffer. When there is only one window,
  the mode

  If it is on the bottom, then it should stay on the buttom. Mode line
  jumping up and down... for heaven's sake no. We focus our eyes on
  the mode line. I am reading information. Sometimes I have just few
  minutes work on a laptop with low battery, we need line numbers and
  so on, time, date. Anything. We need `keycast-mode' and not that it
  jumps up-down. Helm does it right. But it is overkill and does not
  have nice display.

So let me test it.

Another issue is speed. I will test selectrum again, I think there are
issues with speed. It does matter with 20000 items.

I will report back after test.

[gnusupport]

If I am not mistaken it is not in main branch, so I will wait.

[gnusupport]

• Clemens Radermacher [e-mail] [2020-11-10 22:53]:
  > With #230 you should be able to display candidates in any window you
  > want. I also added a selectrum-display-full-frame action that I
  > liked to use myself in helm from time to time. Give it a try and let
  > me know if you run into problems.

Help me how to get that piece of code. git pull alone does not
work. Do I need to merge something and how?

[clemera]

Here is how I do it: Add the following to your gitconfig

[remote "upstream"]
    url = https://github.com/raxod502/selectrum.git
    fetch = +refs/heads/*:refs/remotes/upstream/*
    fetch = +refs/pull/*/head:refs/pull/upstream/*

Then in magit use f a to fetch all and afterwards checkout the pr branch (refs/pull/upstream/230).

[gnusupport]

@clemera thank you, I could use few simple commands and get the branch. No need for magit. Video is here:
https://open.tube/videos/watch/6202254a-bd5f-4323-980a-73d633e4b8ad

At my side using that full frame function is not working as I expected. I do not get full frame if I am in split window. Completion comes on top over the split window, things are happening in some unpredicted or unfamiliar manner. Not that I am complaining, just giving you insights as maybe you discover how to enhance it.

[clemera]

Completion comes on top over the split window, things are happening in some unpredicted or unfamiliar manner.

Do you use the latest version? Maybe it is related to your other configuration? The full frame action function works correctly in emacs -Q using the following minimal setup:

(selectrum-mode 1)
(setq selectrum-display-action '(selectrum-display-full-frame))

[clemera]

@raxod502

I am at least mildly wary of Selectrum attempting to become all things to everyone

I agree, it would be a bad idea to try to handle all use cases out of the box but I think it would be nice to allow implementing different UIs using selectrums API. If someone wants to display the candidates in a different way that should be fine as long selectrum doesn't have to deal with the details. For example I think it would be useful to also have a format buffer function to let clients implement custom things like presenting candidates in a grid or simply like icomplete in a single line. The commands to control those UIs could live in the client code so we wouldn't have to deal with it on selectrums side.

[gnusupport]

• Clemens Radermacher [e-mail] [2020-11-13 16:39]:
  > @raxod502
  > > I am at least mildly wary of Selectrum attempting to become all things to everyone
  >
  > I agree, I would be a bad idea to handle all use cases out of the box but I think it would be nice to allow implementing different UIs using selectrums API. If someone wants to display the candidates in a different way that should be fine as long selectrum doesn't have to deal with the details. For example I think it would be useful to also have a format buffer function to let clients implement custom things like presenting candidates in a grid or simply like icomplete in a single line. The commands to control those UIs would also live in the client code so we wouldn't have to deal with it on selectrums side.

Let me add that I have tested the implementation where mode line does
not jump up and down and selectrum behaves well similar to Helm with
that function. But as function is outside of Selectrum it is not my
choice to use the branch as it becomes difficult to install or
maintain it.

There is comparison with other completion packages in attached
pictures on the following email:
https://lists.gnu.org/archive/html/emacs-devel/2020-11/msg00482.html

The split window before completing is initial condition:
https://lists.gnu.org/archive/html/emacs-devel/2020-11/pngB_djvgE1PW.png

Helm replaces bottom window without disturbing user
interface:
https://lists.gnu.org/archive/html/emacs-devel/2020-11/pngMnhrkGos0V.png

ivy disturbs user interface:
https://lists.gnu.org/archive/html/emacs-devel/2020-11/pngHefqhgDN5k.png

when using outside function, ivy behaves better, but disturbs sizes of windwos:
https://lists.gnu.org/archive/html/emacs-devel/2020-11/pngvw1uDbUQIV.png

Selectrum does not disturb user interface only with the function
offered here in the branch 230:
https://lists.gnu.org/archive/html/emacs-devel/2020-11/pngsK43Zw0TK7.png

When I say here "disturbs" that means that windows that user uses are
unnecessarily enlarged or made smaller.

Selectrum by default disturbs user interface.

As user, I have windows because I wish to use them. Maybe I wish to
insert something into the window so I do not want that window made
smaller.

If completion read is enlarging minibuffer it is practically and
visually already splitting the window. But aesthetically and
practically it disturbs the common user interface as mode line jumps
up.

That one wish to expand minibuffer is one wish. It collides with the
default that modeline should be on bottom. Not above the center of
Emacs which can happen with ivy, icomplete, selectrum.

Advice is to keep mode line where it is. I understand that people use
completion personally. If you wish your code be reused then please
think on hundreds of people who will be using completion, they are not
you. They will read Emacs Manual and find out that modeline is on
bottom and will get used to mode line being at bottom.

And I see it is hard to communicate by words, including by
pictures. Some people do not see what is happening as they are
familiar to interface.

In my Emacs M-x ruler-mode is showing like 96 characters. And
linum-mode tells me I have just about 26 lines or rows. Visibility is
great and does not impart me in any manner.

Now I am splitting a window with: C-x 2

Then I have one mode line between windows, horizontally. Let us say
now I wish to insert something in my buffer and choose selectrum.

As soon as I open minibuffer for selectrum completion I am getting 3
windows. You may not call it window, but for me is practically
window. One is top, one that was bottom window and minibuffer
enlarging.

In general I get 3 windows about equal in its size.

Reference to usability guidelines:
https://www.usability.gov/what-and-why/user-interface-design.html

[clemera]

@gnusupport

But as function is outside of Selectrum it is not my
choice to use the branch as it becomes difficult to install or
maintain it.

This sounds like your are not familiar with the process of a PR on github: If you make a PR the plan is usually to merge it eventually. I want to wait until it is reviewed and @raxod502 approves the changes because it is a significant new feature which I don't want to decide to include on my own and without testing it for at least a few days and maybe more feedback from others. You can help by testing the branch and report any issues you find.

[gnusupport]

• Clemens Radermacher [e-mail] [2020-11-13 17:41]:
  > @gnusupport
  > > But as function is outside of Selectrum it is not my
  > choice to use the branch as it becomes difficult to install or
  > maintain it.
  >
  > This sounds like your are not familiar with the process of a PR on
  > github: If you make a PR the plan is usually to merge it
  > eventually. I want to wait until it is reviewed and @raxod502
  > approves the changes because it is a significant new feature which I
  > don't want to decide to include on my own and without testing it for
  > at least a few days.

I am not familiar, sorry. I have not get any reference on these pages
to understand that separate feature is to be merged eventually.

[clemera]

No problem, as I said feel free to comment on the PR if you have any suggestions about that new feature.

[gnusupport]

• Clemens Radermacher [e-mail] [2020-11-12 02:56]:
  > > Completion comes on top over the split window, things are happening in some unpredicted or unfamiliar manner.
  >
  > Do you use the latest version? Maybe it is related to your other configuration? The full frame action function works correctly in emacs -Q using the following minimal setup:
  >
  > ```elisp
  > (selectrum-mode 1)
  > (setq selectrum-display-action '(selectrum-display-full-frame))

That is useful feature. Exactly something that I need, only that
selectrum does not provide by default reverse order and spaces.

[clemera]

Exactly something that I need, only that
selectrum does not provide by default reverse order and spaces

For more space between lines you can set line-spacing and with #305 you will be able to set a custom insertion function which could insert the candidates in reverse order. With that I think we have discussed and solved the points you mentioned? For future issues please try to keep the issue focused to a single context this makes it easier for us to keep track of things.

[raxod502]

This thread is being closed automatically by Tidier because it is labeled with "waiting on response" and has not seen any activity for 90 days. But don't worry—if you have any information that might advance the discussion, leave a comment and I will be happy to reopen the thread :)