fzf needs a -u/--unique option

[stephentalley]

It would be nice if fzf had a way to omit repeated lines.

I typically run fzf with this pattern:

(cmd1; cmd2) | fzf

where:

• cmd1 and cmd2 return the same results, mostly.
• cmd1 is fast but unreliable (results may be out of date).
• cmd2 is slow but reliable (results are always current).

This allows me to get fast matches that are probably good, while slower ones that are definitely good trickle in.

Unfortunately this leaves me with a lot of duplicated lines in fzf.

For example, I'd like for:

(echo 'hello'; sleep 3; echo 'hello') | fzf

...to only display "hello" once.

A -u/--unique option would be nice.

Apologies if there's already a way to do this and I've overlooked it.

Thanks!

[junegunn]

Thanks for the suggestion, but it's not the responsibility of fzf and should be done by a separate filter. Trying to implement such behavior in fzf is non-trivial and it's not in accordance to the unix philosophy.

We both know uniq doesn't work here as it is for sorted input, and I don't know if there's a pre-existing unix filter for the purpose, but it's trivial to write one using any scripting language. For example, with Ruby,

#!/usr/bin/env ruby

require 'set'
found = Set.new
$stdout.sync = true
ARGF.each do |line|
  unless found.include? line
    found << line
    puts line
  end
end

(echo 'hello'; sleep 3; echo 'hello') | nodup | fzf

[stephentalley]

Thanks, that's a fair assessment.

[edi9999]

Great to see that you don't accept too many features so that the thing doesn't get too complicated to maintain (I guess it's already quite big)

[rafaeln]

I understand why you can't use uniq to solve the original problem, but I solved a similar problem using uniq and thought I could share my solution with you people. I use fzf's ctrl-r command a lot to, for instance, obsessively issue the command to update my vim plugins, as you can see in the image below.

I didn't want to see repeated commands anymore, and changing the definition of ctrl-r in key-bindings.bash solves this issue. The only change is on the line HISTTIMEFORMAT= history |, which becomes HISTTIMEFORMAT= history | sort -k 2 | uniq -f 1 |. I copy the new version below:

__fzf_history__() (
  local line
  shopt -u nocaseglob nocasematch
  line=$(
    HISTTIMEFORMAT= history | sort -k 2 | uniq -f 1 |
    FZF_DEFAULT_OPTS="--height ${FZF_TMUX_HEIGHT:-40%} $FZF_DEFAULT_OPTS --tac -n2..,.. --tiebreak=index --bind=ctrl-r:toggle-sort $FZF_CTRL_R_OPTS +m" $(__fzfcmd) |
    command grep '^ *[0-9]') &&
    if [[ $- =~ H ]]; then
      sed 's/^ *\([0-9]*\)\** .*/!\1/' <<< "$line"
    else
      sed 's/^ *\([0-9]*\)\** *//' <<< "$line"
    fi
)

[edi9999]

There also is nauniq (it is non adjacent uniq, eg it can uniq without the input being sorted).

I use it like this : https://github.com/junegunn/fzf/issues/600#issuecomment-227135066