Selectrum: is it possible to rebind TAB to equivalent of ivy-partial?

[aspiers]

Hi there, I'm excited to be trying out selectrum, consult, prescient, marginalia after using ivy/counsel/swiper for a while. One of the first things which puzzled me is that TAB completes the currently selected item rather than the longest common substring across all matching candidates. I see that the README says:

(What this actually does is insert the currently selected candidate into the minibuffer, which for find-file has the effect of navigating into a directory.)

and I do appreciate the value of that. However, the other behaviour of inserting the longest common substring is standard in pretty much every other completion interface I've used, and I'd prefer to keep that if possible, otherwise I have to go against about 25 years of muscle memory for tab completion. Is this possible to configure somehow?

For comparison, in ivy, it defaults to ivy-partial-or-done, and can be overridden to do ivy-partial.

[clemera]

I would prefer if we keep the UI simple and opinionated but this might be a good thing for the wiki? Here is my quick attempt to port the command from ivy without guarantees because I don't know how it is supposed to work exactly:

(defun selectrum-partial ()
  "Complete the minibuffer text as much as possible."
  (interactive)
  (let* ((matchstr (if minibuffer-completing-file-name
                       (or (file-name-nondirectory
                            (minibuffer-contents)) "")
                  (minibuffer-contents)))
         (parts (or (split-string matchstr " " t) (list "")))
         (tail (last parts))
         (postfix (car tail))
         (new (try-completion postfix
                              selectrum--refined-candidates)))
    (cond ((or (eq new t) (null new)) nil)
          ((string= new matchstr) nil)
          ((string= (car tail) (car (split-string new " " t))) nil)
          (new
           (delete-region (save-excursion
                            (search-backward matchstr nil t)
                            (point))
                          (point-max))
           (setcar tail new)
           (insert (mapconcat #'identity parts " "))
           t))))

(define-key selectrum-minibuffer-map
  (kbd "TAB") 'selectrum-partial)

[aspiers]

Thanks a lot for this! I've just tried it with consult-buffer and it didn't insert anything. I stepped through with edebug and it showed that while postfix is assigned correctly, (try-completion ...) returns nil.

On the design side, FWIW here is my $0.02 which of course you are welcome to ignore! While I agree that keeping the UI simple and opinionated is a good approach, I suspect that deviating from the standard completion behaviour of TAB as seen in "raw" emacs, bash, zsh, etc. is likely to cause confusion for many people used to that behaviour, not just me. So even if you don't want to change the default behaviour, IMHO it would be worth offering this behaviour "out of the box" as an option, rather requiring copying and pasting code from a wiki.

[clemera]

I've just tried it with consult-buffer and it didn't insert anything. I stepped through with edebug and it showed that while postfix is assigned correctly, (try-completion ...) returns nil.

When I input *scra it completes to *scratch*, can you provide more detail on the case that does not work like you expect?

On the design side, FWIW here is my $0.02 which of course you are welcome to ignore! While I agree that keeping the UI simple and opinionated is a good approach, I suspect that deviating from the standard completion behaviour of TAB as seen in "raw" emacs, bash, zsh, etc. is likely to cause confusion for many people used to that behaviour, not just me. So even if you don't want to change the default behaviour, IMHO it would be worth offering this behaviour "out of the box" as an option, rather requiring copying and pasting code from a wiki.

Thanks, for your input, if there aren't to many edge cases we need to consider we could also think about adding it. This was just my first reaction when I looked at the ivy version which seemed to handle quite a few cases in a special way. We can wait a bit if there are more people interested in this functionality and until we have a version that works well enough.

[clemera]

Sorry I haven't read your response closely enough, consult-buffer also does not work for me.

[clemera]

That is because the candidates of it contain other chars (hidden from view) which is another indicator for me that such a partial complete command could cause us to much trouble down the road.

[aspiers]

Ahh. I see them too, e.g.

#("b​ *Customize Option: Prescient Filter Method*" 0 3 (display "") 3 46 (face consult-buffer))

Where does that initial "b " come from, and what hides it?

[clemera]

This is the hack that makes it special filtering possible. You can type m SPC to filter to bookmarks, b SPC to filter to buffers and so on. If you are really unhappy with the way how TAB works in Selectrum you could also give icomplete-vertical a try.

[aspiers]

Ah OK! That's nice, I missed that feature - is it undocumented or am I just not reading the README carefully enough? If those hidden characters are added by Selectrum, presumably it would not be hard to write an inverse function which strips them out again, and that could be applied to the candidate list via mapcar.

I definitely want to stick with Selectrum, for now at least ;-)

[clemera]

Those chars are added by consult, you can even write your own commands using it, too. See the narrow prefix of consult--read. It's still early in development so @minad is probably busy coding ;)

[aspiers]

I see this comment before consult--line-prefix:

;; HACK: Disambiguate the line by prepending it with unicode
;; characters in the supplementary private use plane b.
;; This will certainly have many ugly consequences.
(defsubst consult--line-prefix (width line)

Embedding metadata as text within each candidate does indeed feel like the wrong way of going about it. A cleaner way might be to embed it as text properties, and then teaching the matching code how to match properties as well as the text.

[clemera]

The purpose of this is to provide this functionality in a framework independent way so it works with any framework which is compliant to completing-read API. You can get the same functionality with Selectrum without this hack but then it works only for Selectrum :(

[aspiers]

I see. So maybe one quick win would be to make it optional so that Selectrum users can turn off the hack and stop it interfering with other stuff like our attempts above :-)

[minad]

@aspiers The hack is actually pretty solid and there is no reason to change it. It should be possible to get the partial completion to work - I just have to modify the example by @clemera slightly as a poc. The only thing which must be done is to add the narrowing prefix in front. The narrowing prefix could be taken from the currently selected candidate. The following works for consult-buffer if you have multiple buffers with the same prefix, enter that prefix and then press TAB.

~~~ elisp
(defun selectrum-partial ()
"Complete the minibuffer text as much as possible."
(interactive)
(let* ((matchstr (if minibuffer-completing-file-name
(or (file-name-nondirectory
(minibuffer-contents)) "")
(minibuffer-contents)))
(parts (or (split-string matchstr " " t) (list "")))
(tail (last parts))
(postfix (car tail))
(new (try-completion (concat "b" consult--narrow-separator postfix) ;; XXX: Note the difference
selectrum--refined-candidates)))
(cond ((or (eq new t) (null new)) nil)
((string= new matchstr) nil)
((string= (car tail) (car (split-string new " " t))) nil)
(new
(delete-region (save-excursion
(search-backward matchstr nil t)
(point))
(point-max))
(setcar tail new)
(insert (mapconcat #'identity parts " "))
t))))

(define-key selectrum-minibuffer-map
(kbd "TAB") 'selectrum-partial)
~~~

Another approach one could try is to use a narrowing/unique suffix instead of a prefix. But then one has to check how well this plays together with the completion styles.

EDIT: I did a quick test and it seems that using a suffix would also work out, maybe that would be more benign since it would allow use-cases like selectrum-partial. Opinions? @clemera do you see any implications of this? I think for narrowing it may not work as well for primitive completion styles.

[aspiers]

Got it, thanks. So prefix vs. suffix aside, the main thing that would need to be changed with your example is replacing the hardcoded "b" with whatever prefix the currently selected candidate has?

[minad]

Got it, thanks. So prefix vs. suffix aside, the main thing that would need to be changed with your example is replacing the hardcoded "b" with whatever prefix the currently selected candidate has?

Got it, thanks. So prefix vs. suffix aside, the main thing that would need to be changed with your example is replacing the hardcoded "b" with whatever prefix the currently selected candidate has?

Yes, for example.

But this would not work for consult-line, since there every line really has a unique prefix since we need the disambiguation.

Regarding suffix/prefix: For consult-line the unique suffix might work better than the prefix we have right now and selectrum-partial would just work. I think it would not have any bad implications. But for the narrowing use case I am not so sure, there I think using the prefix is more appropriate and selectrum-partial would have to work around this as we just discussed.

If you are interested in having something like selectrum-partial/consult-partial supported, feel free to propose a PR to consult!

[aspiers]

Thanks, and good point about consult-line. What issues do you see with changing the narrowing prefix to a suffix?

[minad]

I think it may not work well with completion styles and I think it does not make sense.

The narrowing prefix is no different than simply completing strings with a prefix.

'("b buffer1" "b buffer2" "f file1" ...)

The only difference is that there is a special char used as separator in order to avoid false matches. Furthermore in order to reduce noise, the prefix is not displayed, but this is just a rendering issue.

To say it a bit more bluntly - fo consult-line the prefix is a hack, for the narrowing it is not. If you dislike the special character, you can also use an ascii character as separator e.g. #. Then you can simply type out the prefix "b # SPC".

EDIT: I think I should make it even more clear - the "hack" is only a hack because I am using unicode characters - not because I am using a prefix. I could as well use the line number as string as prefix. Using unicode chars has just one effect - it prevents accidental matching, such that the user can search for numbers in consult-line and does not match the line numbers.

EDIT2: I did some experiment with unique suffixes for consult-line. See here https://github.com/minad/consult/tree/unique-suffix. The downside is that if the user presses TAB, the whole string completes (with the added suffix). In order to undo, the suffix has to be deleted first before editing the actual candidate. So I am not sure if this is better than using the prefix.

[aspiers]

Thanks for the updates. I agree that having to delete the suffix would mostly negate the UX benefits of changing TAB's behaviour in the first place.

I'm still inclined to suspect that embedding the metadata as text properties on the existing matches rather than as additional hidden text would facilitate a cleaner solution due to the clean separation of data from metadata. But of course I'm not nearly as close to the code as you are so I could be missing something.

[minad]

I'm still inclined to suspect that embedding the metadata as text properties on the existing matches rather than as additional hidden text would facilitate a cleaner solution due to the clean separation of data from metadata. But of course I'm not nearly as close to the code as you are so I could be missing something.

No, it is not - you are indeed missing something. Please try to implement it and then come back with the solution - I cannot say something else. It is not that we did not try properties. In fact it was the first thing I tried.

[aspiers]

Albert Einstein said "The definition of insanity is doing the same thing over and over and expecting different results" so I think I will kindly decline your invite to try that again ;-)

[minad]

:sweat_smile:

[clemera]

@minad What do you think about adding the selectrum-candidate-full property to the actual candidate strings on consults side? I know you would like to see it removed from Selectrum but for such purposes it could still be useful.

[minad]

@clemera Right now my stance is that any specifics should be removed - and we should push it as far as we can. We can still add selectrum-specific tweaks back later if it turns out that things are not possible. But right now I am not seeing this - at least not for this particular issue. Dynamic candidates are another story - there I am sure we need selectrum specific tweaks in order to avoid these recomputations all the time.

[clemera]

It also occurred to me that this would probably also mess up the return value because you are probably expecting to get back the full version (with hidden display text) so this already indicates this wouldn't be a good idea.

[clemera]

@aspiers I'm wondering in which use case the behaviour you are aiming for is more useful compared to the way it works by default in Selectrum? I think the behaviour of bash completion and such was designed for a different interaction model (where you don't have continuous completion), but I may also miss something here.

[aspiers]

You're right that it's a different interaction model, and this is actually a very good question which could significantly help clarifying how a solution could be implemented, so thanks for asking.

Other than the motivation to behave similarly to the way users coming from most other completion systems would expect, the main use case which I personally feel the need for is to refine the list of matches based on exact contiguous substring matches. For example, if I have fuzzy matching enabled and type M-x consu, I will see not only a load of commands beginning with consult-, but also commands such as:

• counsel-imenu, counsel-ibuffer etc.
• comint-send-input, comint-stop-subjob etc.
• comment-set-column

However if I type M-s f to disable fuzzy matching, I can see that the only commands which contain the contiguous substring consu all begin with consult-.

In the scenario with fuzzy matching enabled, pressing TAB can mean "please complete consu to consult-", or more generally speaking, "please complete the rest of the contiguous substring which is common to all candidates which exactly match the substring I've typed so far".

Note that in this example, if there existed one or more commands beginning with consume, then the desired behaviour of pressing TAB after M-x consu would be to do nothing, or perhaps emit a warning message saying that no more completion can be done, because there is no common contiguous substring longer than consu.

In summary, this use case allows further refinement of a fuzzy search by use of non-fuzzy criteria. There may be other use cases but this is the one which immediately strikes me as useful. Or maybe this is the only useful use case, in which case it would only apply when fuzzy search is enabled.

[clemera]

I don't use fuzzy search myself but thanks to your example I see how it could be useful in that case. I'm starting to think if it would make sense to have such a command on a different key. Does the code we have so far work well enough for you (except the case with consult-buffer)?

[minad]

@aspiers Is fuzzy the style from prescient similar to the flex style in orderless?

[minad]

@clemera If you are considering to add such a command, my suggestion would be to add the command to consult instead of to selectrum, since it could be useful for icomplete too. What do you think?

I am also not using flex/fuzzy matching. It matches too much and I think it slows down things.

[clemera]

@minad I would prefer that, though I believe with icomplete it is sort of built-in? I'm not sure because I never used icomplete seriously.

[minad]

Ok, we should check icomplete again. If we add it to consult we could at least guarantee that it works nicely with consult commands too :)

[clemera]

That would be nice, bu I just realized there are Selectrum specific things we need to handle (the command above already uses some internal Selectrum variables), so it isn't really a good candidate for consult.

[minad]

@clemera Ok, these refined candidate list? Maybe these can be obtained via other mechanisms. Generally it would be cool if we can push generic things out of selectrum, maybe it does not work in that case but we should keep it in mind for other things.

[clemera]

Yes, I agree, I think UI commands will probably often be hard to generalize but we should always think about it.

[aspiers]

@minad commented on December 18, 2020 12:13 PM:

@aspiers Is fuzzy the style from prescient similar to the flex style in orderless?

Yes I believe so, based on the description of orderless-flex at https://github.com/oantolin/orderless#component-matching-styles.

[aspiers]

This is just my ignorance showing, but if it was implemented in consult, how would it work with the normal binding of M-x to execute-extended-command?

[aspiers]

Just realised that the use case I described above of disambiguating literal substring matches from fuzzy matches could equally apply to disambiguating from initialism matches. For example, if I set prescient to match by (initialism literal), then M-x roo matches:

• commands which include the word root
• commands like ffap-read-only-other-{frame,window}, find-file-read-only-other-{frame,window}

In this case TAB could complete to root. Admittedly it's not a good example since it doesn't save any keystrokes, but you get the point.

[aspiers]

@clemera commented on December 18, 2020 12:13 PM:

I'm starting to think if it would make sense to have such a command on a different key.

Could do. That doesn't address my point about how most users will intuitively expect TAB to behave based on their experience of shells and other completion contexts, but I suppose they could always rebind TAB to that command if it bothers them.

Does the code we have so far work well enough for you (except the case with consult-buffer)?

Not quite. For example, C-h f promo finds a bunch of functions which contain either promote or promotion, but TAB does not add the t common to both.

[clemera]

Okay, hopefully someone can come up with a more reliable implementation, I have to admit I strongly dislike the behaviour you want, so I'm a bit unmotivated to put more work into this ;)

[aspiers]

No problem at all, and I really appreciate your help so far. BTW I'm very open to the possibility that there are better approaches to matching, and so I'd be very interested to hear the way you do it :-)

[clemera]

I understand that being able to reuse your habits would be nice, so I would be okay with adding such a command if we have something that you think is robust and works well. As I use no fuzzy matching I type space and add more input if I need to narrow things further down. I use orderless which is similar to prescient but allows for more flexible matching styles. I still use prescient for sorting, though.

[DivineDominion]

I never understood why I couldn't hit TAB as often as I wanted with ivy because sometimes it would pick the best match. Making this configurable/extensible does make sense to appeal to ivy users, but replacing selectrum's behavior? -- That's remove one differentiating factor of the whole selectrum/consult/prescient combo.

[lsth]

I have the same problem, but the code snippet above does not work for me. Says "void variable consult--narrow-separator".

The intended behavior for me would be as follows:

• tab inserts the longest common extension of all remaining candidates that
  start with the first query term,
  e.g., say for find-file the query is "ab" and there are 3 remaining candidates "abc1", "abc2" and "xyabd1", then
• ab<TAB> should yield "abc", but
• ab 1<TAB> will work on further narrowed down candidates "abc1" and
  "xyabd1" and thus should yield "abc1".

The following hack works so far (using the s library):

(defun my-selectrum--insert (s)
  "Insert string S as the user input."
  (with-selected-window (active-minibuffer-window)
    (cond ((not selectrum--crm-p)
           (delete-region (minibuffer-prompt-end)
                          (point-max))
           (insert s))
          (t
           (goto-char
            (if (re-search-backward crm-separator
                                    (minibuffer-prompt-end) t)
                (match-end 0)
              (minibuffer-prompt-end)))
           (delete-region (point) (point-max))
           (insert s)
           (when-let ((match
                       (assoc crm-separator
                              selectrum--crm-separator-alist)))
             (insert (cdr match))))))
  ;; Ensure refresh of UI. The input input string might be the
  ;; same when the prompt was reinserted. When the prompt was
  ;; selected this will switch selection to first candidate.
  (setq selectrum--previous-input-string nil))

(defun my-selectrum-insert-longest-common-extension-initial-match ()
  "Insert the longest common extension of all candidates who's beginning matches the first query term."
  (interactive)
  (with-selected-window (active-minibuffer-window)
    (if-let ((first-queryterm (nth 0 (s-split " " (selectrum-get-current-input))))
         (candidates (seq-filter (lambda (x) (s-starts-with? first-queryterm x)) 
                     (selectrum-get-current-candidates)))
         (prefix (seq-reduce #'s-shared-start candidates (nth 0 candidates))))
    (progn
      (my-selectrum--insert prefix)
      )
      (unless completion-fail-discreetly
        (ding)
        (minibuffer-message "No match")))))

(define-key selectrum-minibuffer-map
  (kbd "TAB") #'my-selectrum-insert-longest-common-extension-initial-match)

Only the three lines in if-let are new and there is no reference to consult. If such a tab completion could be supported out of the box, likely with a more professional implementation than my simple hack, that would be great !

[clemera]

Maybe after you use Selectrum for a while these needs will go away? @aspiers Do you still use Selectrum? Are you more comfortable with the behaviour now or are you still missing ivy-partial?

[lsth]

As OP wrote already: tab completion is in all the shells. So either people using both learn two different paradigms or customize one to work like the other. -- By the way, the argument "these needs will go away" is what the other major emacs completion framework w/o tab completion, helm, also uses. ;)

[clemera]

I think the desired behaviour make more sense for shells (outside of Emacs) as there is no continuous feedback and different matching. But clearly there are people which feel differently and there is nothing wrong with that, as I mentioned above I would be okay adding an option for it if we have a better version.

[aspiers]

@clemera commented on January 7, 2021 5:49 PM:

Maybe after you use Selectrum for a while these needs will go away? @aspiers Do you still use Selectrum?

Yes.

Are you more comfortable with the behaviour now or are you still missing ivy-partial?

I'm still missing it; to be fair it's still early days, but I suspect that won't change.

@lsth commented on January 7, 2021 5:19 PM:

I have the same problem, but the code snippet above does not work for me. Says "void variable consult--narrow-separator".

The intended behavior for me would be as follows:

• tab inserts the longest common extension of all remaining candidates that
  start with the _first_ query term,

I like this thinking a lot. The only minor tweak I would suggest is that it could potentially modify the term your cursor is currently over, rather than the first one. But that's just an initial gut reaction to your suggestion; I would probably have to try both for a while to see which I preferred. Maybe my preference would depend on the use case. I'll see if I can get your hack working. Thanks a lot for this!

[oantolin]

I think this feature should be responsibility of the completion style, which is actually how Emacs does it (try-completion). For some completion styles it doesn't actually make much sense to insert the longest common substring. For example, if you are using orderless with (only) the initialism style, inserting the longest common substring usually means you don't get any matches any more! We discussed this a bit over at oantolin/orderless#32.

[clemera]

@oantolin That sounds ideal, we could have an option to configure the styles to be used for such a command.

[minad]

I must admit I also need this TAB completing to the longest match behavior, in particular in the eshell. This is what is preventing me from using eshell, since it is just too different from other shells. I agree that it is the responsibility of the completion style. In particular if one sets selectrum-complete-in-buffer, Selectrum is not even used for the completion at point. If I switch to basic completion style it should probably work but I have to figure out how to set this only for completion at point (Any hints?). In contrast, in the minibuffer I never missed the TAB completion after starting to use Selectrum. There the orderless behavior is perfectly fine for me. Weird.

EDIT: Using something like this now:

~ elisp
(setq-default selectrum-complete-in-buffer nil)
(let ((orig completion-in-region-function))
(setq completion-in-region-function
(lambda (&rest args)
(let ((completion-styles '(basic partial-completion)))
(apply orig args)))))
~

[oantolin]

It might be easier to use (setq completion-styles '(substring orderless)). That feels like a slightly enhanced orderless to me: it's normal orderless except that if your query has only one component and that single component matches literally, then TAB will insert the longest common substring.

[minad]

@oantolin Right, I missed this in suggestion in https://github.com/oantolin/orderless/issues/32. Does this have any downside with respect to orderless since I don't want to lose the power of orderless? I guess usually the substring will not match and orderless will just take over. But if substring matches, did I understand correctly that orderless matches are not shown then? It seems to me that the Emacs completion style is just badly designed in its current form and we should all be using Icicles matching. I wonder if some Iceberg mode could be added to orderless which combines substring and orderless matches and then allows tab completion as long as the substring part of orderless suceeded. The only difference would be that orderless would show more matches. Would that make sense or did I misunderstand how these completion styles are applied after each other?

[minad]

Or maybe it really does not make a difference at all since substring can already match everywhere? I think I had basic in my mind just now.

EDIT: I will give this a try and maybe this should be the standard setting to promote everywhere, e.g., in the orderless README since it allows tab completion without downsides?

EDIT2: Seems to work for me. I am glad I asked here again. Thank you, @oantolin!

[oantolin]

I should mention substring and partial-completion's path completion in the orderless README.

[minad]

Is partial-completion necessary? If I type something like C-x C-f /a/b/c TAB it just seems to work in Selectrum with (substring orderless). But I believe this should definitely be the recommendation if there are no downsides.

[oantolin]

That's something special @clemera added to selectrum (_instead_ of fully supporting completion-styles 😛).

[oantolin]

I'm just kidding, @clemera, I think it is a very reasonable compromise.

[minad]

Oh okay, Selectrum is doing its own partial completion somehow. I like that it works... It is probably special cased since otherwise TRAMP and all the magic won't work properly?

This means the recommended styles are as follows?

• With selectrum (substring orderless)
• Without selectrum (substring partial-completion orderless)

[oantolin]

I wouldn't go as far as recommending either one. I will mention the settings and what they accomplish.

[oantolin]

By the way, the reason I personally don't use partial-completion for paths is that by typing TAB (with orderless TAB expands _if_ there is a unique match) instead of /, I get the same mostly the behavior but with the following advantages (from my point of view): I get visual confirmation along the way that I'm in the right directories, and I learn right away if some directory component does not have a unique match. Of course I loose an advantage of partial-completion: while you may be flying blind with it, it does have the attractive feature that a future component may help disambiguate a previous component that matches too much.

EDIT: Obviously I meant, "when not using Selectrum", which uses TAB for something else entirely.

[clemera]

(instead of fully supporting completion-styles stuck_out_tongue).

To give some more detail on that, the file completion function uses partial style input as another way to get candidates but the filtering afterwards doesn't include it automatically so it can still make sense to add partial-completion if you want to use that on the results.

[minad]

@oantolin I think partial completion is mostly useful if you know that some sequence will match for some deep directory structure.

[clemera]

I did not use partial-completion in the past, if I need fuzzy completion I add a ~ which I have stolen from an orderless config somewhere :sunglasses: but recently I also started to enjoy the built-in partial completions as for some cases it can be really convenient when you know the path at some deeper level.

[oantolin]

@oantolin I think partial completion is mostly useful if you know that some sequence will match for some deep directory structure.

Yes @minad, but for my directory structures, pressing TAB instead of each /, usually gives exactly the same result as partial-completion, with the advantage that I see in the minibuffer the full path. Same number of keypresses, more information on the screen.

[minad]

I don't understand, what about /d/e/e/p/d/i/r TAB?

[oantolin]

I don't understand, what about /d/e/e/p/d/i/r TAB?

I type /d TAB e TAB e TAB p TAB d TAB i TAB r, instead. Sometimes a single letter is not enough, so I type ^d or a second letter. I like seeing the full path develop in the minibuffer, so I can check I'm in the right place.

[clemera]

I like seeing the full path develop in the minibuffer, so I can check I'm in the right place.

Note that with Selectrum (which you should definitely enable by default ;)) you will also see how the path develops by looking at the resulting path completions as you type.

[oantolin]

I just tried it. That does look very nice, @clemera!

[minad]

@oantolin But your way only works if you have unique elements all the way. Partial-completion should work in more scenarios.

[oantolin]

That's exactly what I meant when I said, partial-completion "does have the attractive feature that a future component may help disambiguate a previous component that matches too much", @minad. In practice I think this advantage is not very important. I often do get unique matches typing just two letters at each step. (Maybe my directory hierachy is particularly simple?)

Also I enable tab cycling, so despite using orderless, TAB doesn't only complete when there is a single match, but whenever there at most 3 matches (or 5, I can't remember what I set completion-cycle-threshold to) it cycles among them.

I'm not saying partial-completion for paths isn't useful, I'm just saying I find it less useful than being able to see the whole path as I go along. Maybe Selectrum's nice UI makes partial-completion better than completing each step for me, I'd have to use it more to see. But with default completion I definitely prefer completing each step to praying and completing at the end.

[minad]

Selectrum does not seem to support completion-cycle-threshold (#419)?

[oantolin]

Nope, it doesn't. In Selectrum I do the exact same thing I mentioned of using l TAB instead of / but using <down> instead.

[oantolin]

I mean that's what I did in Selectrum before, now I'll use / because the partial completion UI is so nice.

[Ergus]

Hi:

I have been trying to replace ivy with selectrum. But without tab partial completion it becomes very uncomfortable. Emacs function names are becoming longer and longer to have a sort of namespace; that makes the partial completion very handy to reduce typing and speedup working. I know that selectrum is trying to stay simple; but this only needs an extra function... so please...

[oantolin]

@Ergus With selectrum you do not need to have the full completion candidate in the minibuffer, it's enough to bring it to the top of the list, then you press RET. From that point of view, TAB completion doesn't actually help you at all: it inserts more text into the minibuffer in such a way that the candidate list does not change at all! So TAB actually makes 0 progress towards narrowing the candidate list.

I recommend using Selectrum with either Prescient or Orderless. With those you can keep pressing space and adding new queries to narrow down the candidate list.

[oantolin]

The same goes for Ivy, actually, in Ivy TAB also does not make progress (in the sense of reducing the candidate list) either.

[minad]

I agree with the argument but progress is not only measured by the number of candidates but also by the remaining work. If you use TAB you can quickly complete the namespaces that @Ergus mentioned. The next character you type after TAB is then adjacent to the TAB completed prefix and as such can match less than hitting space plus this character.

[Ergus]

Exactly:

ico\<TAB\> saves to type mplete.
pack\<TAB\> saves from typing age-
disp\<TAB\> saves typing play-

The idea is not to reduce the candidates list, but complete the common part.

[oantolin]

Fair enough.

[oantolin]

Exactly:

ico\<TAB\> saves to type mplete.
pack\<TAB\> saves from typing age-
disp\<TAB\> saves typing play-

The idea is not to reduce the candidates list, but complete the common part.

But why though? What advantage do you get from having that part completed, @Ergus? Is it what @minad said or do you derive a different benefit?

[Ergus]

But why though? What advantage do you get from having that part completed, @Ergus? Is it what @minad said or do you derive a different benefit?

In general it is that we have the tab recorded in our muscular memory (from bash and zsh), plus it saves from typing a lot after some time (or reading in the list of candidates), reduce the list faster like @minad suggested.

[Ergus]

Using tab to insert the full first candidate is somehow a bit redundant IMO when we have RET that executes it directly.

[minad]

@Ergus Did you try the snippet pasted by @clemera above? This works for me.

~~~ elisp
(defun selectrum-partial ()
"Complete the minibuffer text as much as possible."
(interactive)
(let* ((matchstr (if minibuffer-completing-file-name
(or (file-name-nondirectory
(minibuffer-contents)) "")
(minibuffer-contents)))
(parts (or (split-string matchstr " " t) (list "")))
(tail (last parts))
(postfix (car tail))
(new (try-completion postfix
selectrum--refined-candidates)))
(cond ((or (eq new t) (null new)) nil)
((string= new matchstr) nil)
((string= (car tail) (car (split-string new " " t))) nil)
(new
(delete-region (save-excursion
(search-backward matchstr nil t)
(point))
(point-max))
(setcar tail new)
(insert (mapconcat #'identity parts " "))
t))))

(define-key selectrum-minibuffer-map
(kbd "TAB") 'selectrum-partial)
~~~

With M-x it does not seem very useful to me. At least it does not speed up my completions. The argument I made above holds theoretically but in practice I am much faster using the space completion style which @oantolin advocates. In the minibuffer orderless completion is the fastest way to select a candidate imo. TAB completion does not work well with many distinct candidates which share a similar prefix. For example if you press M-x fi TAB there is really zero progress being made since after the TAB press the minibuffer still shows fi. This happens way too often to make this useful. Furthermore TAB completion does not fit well with the more powerful Emacs completion styles.

Nevertheless I am sympathetic to TAB completion since I am also pretty much accustomed to it in shells. And I made the argument before that TAB completion has additional value when doing completion in region, since you initiate the completion by pressing TAB (see my comment https://github.com/raxod502/selectrum/issues/82#issuecomment-778851974). Basically in the shell you are doing completion in region all the time with the basic completion style. My experience with shell completion is that it works well under very specific circumstances, for example when the candidates to complete are sufficiently distinct. This holds for example if you complete directory names, when pressing C-x f something TAB. Often I deliberately named directories such that they have distinct prefixes. But I would actually consider this to be an ugly workaround.

@clemera How would you like to proceed with this? Either add such a command to Selectrum which can be optionally configured for TAB or keep it only in the wiki?

[clemera]

@minad The UI is tailored to the current behavior so I changed my mind and would prefer to have it on the wiki because I fear there are cases and inconsistencies when using another default TAB behavior and I wouldn't like to maintain or workaround such things.

[minad]

I would not change the default, but one could offer it as an alternative command for TAB. I am unsure about the best approach, either add it to the repository with the risk that it will not be well maintained since the maintainers themselves don't use it. And it is understandable that you don't want to spend time on it. Personally I would probably also not use it, but it depends a bit on my current experiments with completion-in-region. Alternatively add it to the wiki with the same bit-rotting problem and less convenience for the users. Either way, potential users of selectrum-partial command should try the command, check that everything works, enhance/improve it. Then one can see from there.

[clemera]

When using it as a replacement for TAB the command would also need to make use of selectrum--get-full which is relied upon in some cases (most importantly for file completions). And then there are commands like selectrum-select-from-history which also assume the "insert full candidate" behavior. When including it, surely some users would start to bind it to TAB, I think it is better to avoid this and by putting the command on the wiki it is more clear that this isn't an official offer that you can use as an alternative without downsides.

[aspiers]

Hi again all, OP here. Lots of fantastic discussion since I last checked in - thanks all for your contributions. It is getting quite complicated though, and I do slightly worry that we might be in danger of derailing from the original intention I had when filing this issue. For example:

@Ergus commented on February 19, 2021 4:09 AM:

Exactly:

ico\<TAB\> saves to type mplete.
pack\<TAB\> saves from typing age-
disp\<TAB\> saves typing play-

The idea is not to reduce the candidates list, but complete the common part.

Now, I realise I am quoting without the immediate surrounding context in which this probably a perfectly valid idea, but I want to make it clear that it's actually the direct opposite of my original idea in which TAB does reduce the candidates list. If that behaviour (of not reducing the candidates list) is desired, then maybe it would be helpful to propose it in a separate issue.

As @oantolin noted, pressing TAB is not very useful if it doesn't reduce the candidates list. OTOH @minad correctly noted that it can potentially still reduce the amount of remaining work since you now have a fuller query string in the minibuffer which can improve how the candidate list will be further refined in the future. But I want to re-emphasise the value of the original idea of using TAB to immediately reduce the candidates list.

If it is not 100% clear what I mean, please (re-)read this comment and then this follow-up comment. Hopefully this clarifies why I only partially agree with the following comment:

TAB completion does not work well with many distinct candidates which share a similar prefix. For example if you press M-x fi TAB there is really zero progress being made since after the TAB press the minibuffer still shows fi. This happens way too often to make this useful. Furthermore TAB completion does not fit well with the more powerful Emacs completion styles.

The first two sentences are of course correct, but I'm not sure agree that "this happens way too often to make this useful". That sounds like it might be throwing the baby out with the bathwater. Even if it's useful in 10% of cases, it's still useful. As you can see from my original examples linked above, this issue was primarily concerned with pressing TAB after longer prefixes than just two characters. You're right that nothing useful can be done by TAB after M-x fi, but as I said, something useful can be done by TAB after M-x consu (at least, in the fuzzy/flex matching case).

@lsth made a proposal for how this could work which I liked a lot, and he even included a working implementation. @oantolin then noted that this feature should be the responsibility of the completion style, which makes a lot of sense to me.

However, I'm not clear what should be the next step towards an "official" solution.

I should add a disclaimer that I have struggled to fully understand some of the later comments in this issue, since I haven't switched from prescient to orderless yet (although I do intend to try orderless). Consequently I'm not familiar with all the subtle nuances of how it differs from prescient, although I do have a rough idea.

Thanks again everyone for your valuable thoughts on this!

[Ergus]

@Ergus Did you try the snippet pasted by @clemera above? This works for me.

Hi @minad:

Yes I did and that's what I am using... the problem with this kind of snippets is the long time support and the bad initial step for newcommers. I don't advocate to set this a default for sure... just to add the function to the package and a comment in the readme how to enable it.

[clemera]

@Ergus I described the reasons I see against adding above, I'm also worried that as soon as it is included people will start to open issues for the cases where it doesn't integrates nicely.

[minad]

@Ergus

I don't advocate to set this a default for sure... just to add the function to the package and a comment in the readme how to enable it.

It is not a good idea to add something half-baked to the package. In Consult I also rejected some PRs before when they didn't reach the necessary quality. Otherwise one ends up with another (even bigger) long term support problem. As I see it, if this feature is really strongly desired, but the maintainers themselves do not need and use it, the minimum requirement is that someone steps up doing the necessary work. This means creating a PR, ensuring that the command works under all circumstances and does not interact badly with other existing functionality. Then the changes of getting such a feature merged would be increased and it would allow for a more informed decision.

[Ergus]

@Ergus I described the reasons I see against adding above, I'm also worried that as soon as it is included people will start to open issues for the cases where it doesn't integrates nicely.

Hi @clemera:

This TAB completions is the most basic completion around and has been there for decades in shells and many other programs (also in the emacs completions, icomplete, icycles). I can't actually understand why selectrum decided (like helm) to go against that and add the redundant current behavior of inserting a candidate that is easier executed with RET. So IMHO the current tab behavior is either unextected and useless (having a RET).

Tab first option inserion makes sense when the program accepts composed lines at once like: command options arguments(example: vim, zsh)

Because after completing command the list exposes the options and after that the some possible arguments and so on to "construct" the full command at once. But AFAIK emacs does not allow to do that.

The command is a single one and very basic. Emacs already provides all the infrastructure ready to use and the functions to complete common parts (as in minibuffer-complete; just look at completion--do-completion).

So I can't imagine that such a simple feature will create so many new issues that make it impossible to maintain.

[clemera]

@Ergus
No one said it is impossible to maintain, just note that I'm not very motivated to do it, same goes for implementing it in a way that it works nicely for all corner cases which assume the current interaction model (I mentioned a few above). Personally I don't see any benefit as I really like the current behavior for continuous out of order completion for which the UI is primarily designed.

[oantolin]

@ergus Have you tried simply (define-key selectrum-minibuffer-map (kbd "TAB") #'minibuffer-complete)? It seems to work well in the default configuration of Selectrum, which relies on completion-styles, if you use completion styles that offer tab completion. I tried with (setq completion-styles '(basic substring orderless)).

[clemera]

@oantolin Nice that this works, so when accepting the downsides this seems like an easy way to get the wanted behavior!

[oantolin]

Unfortunately I cannot think of any similar cheap trick to get something like the feature that @aspiers wants, but at least for the feature that @ergus wants that seems pretty close. (I confess I didn't read the start of this discussion very carefully and didn't notice until recently that @aspiers actually wants something pretty different.)

[minad]

Is it possible in Selectrum to bind the usual completion commands, like minibuffer-complete? See what I wrote here https://github.com/minad/vertico/#tab-completion. If you use an appropriate completion style, TAB completion should work.

~ elisp
(setq completion-styles '(substring orderless)
completion-auto-help nil
completion-show-inline-help nil)
(define-key selectrum-minibuffer-map (kbd "M-TAB") #'minibuffer-complete)
~

I tried quickly and it seems to work. Therefore I believe that the issue can be closed - no need to invent our own wheels here.

[minad]

Since this has also been discussed here - Consult used prefixes to disambiguates duplicate candidates.

I just changed Consult such that it disambiguates candidates by adding a suffix (https://github.com/minad/consult/commit/4d0a8b9b5a7f76e7055726fb79ed73d01a49aa01). This is much more benign with respect to the basic completion style and TAB completion. The line number prefixes are generated lazily now which will allow faster consult-line startup.

[clemera]

I just changed Consult such that it disambiguates candidates by adding a suffix (minad/consult@4d0a8b9).

Nice! Changing to suffixes also avoids the cursor problem in #373