Fzf: Avoid duplicate entry when fzf-ing shell history

[benoittgt]

• Category
  
  
  
  • [X] fzf binary
  
  
  • [ ] fzf-tmux script
  
  
  • [ ] Key bindings
  
  
  • [ ] Completion
  
  
  • [ ] Vim
  
  
  • [ ] Neovim
  
  
  • [ ] Etc.
  
  
• Shell
  
  
  
  • [ ] bash
  
  
  • [X] zsh
  
  
  • [ ] fish
  
  

I would like to know if it's possible to limit the duplication in fzf specially in the shell history command? Or maybe I'm do it it wrong?

[junegunn]

Hi, this is one of the recurring questions. Please see #49, #88, #492. Thanks.

[benoittgt]

Thanks @junegunn and sorry for the duplicate

[edi9999]

For other people encoutering this issue, here is the changes I've made to remove duplicates :

``` __fzf_history__() { local line countskip="$(history | tail -n 1 | grep -E '^[0-9]+' -o | wc -c)" countskip="$(( countskip + 1 ))" line=$( HISTTIMEFORMAT= history | grep '.\{1,79\}' | sed 's/ *$//g' | tac | nauniq --skip-chars="$countskip" | tac | $(__fzfcmd) +s --tac +m -n2..,.. --tiebreak=index --toggle-sort=ctrl-r | \grep '^ *[0-9]') && sed 's/ *\([0-9]*\)\** .*/!\1/' <<< "$line" }

It requires nanuniq https://metacpan.org/pod/distribution/App-nauniq/bin/nauniq
``````

[chew-z]

perl -ne 'print if !$seen{$_}++' should do.

[GCorbel]

Using setopt HIST_IGNORE_ALL_DUPS on zsh did the trick in my case.

[benjaminbauer]

There are rare, desperate, cases, where I appreciate to browse through the history, in order to reconstruct something.

On ZSH one might use HIST_FIND_NO_DUPS to suppress duplicates in the search. Dups still get written to history, as opposed to HIST_IGNORE_ALL_DUPS.

[giladsh1]

if you're using ZSH, put this in your ~/.zsh file -

export HISTFILE=~/.zsh_history
export HISTFILESIZE=1000000000
export HISTSIZE=1000000000
setopt INC_APPEND_HISTORY
export HISTTIMEFORMAT="[%F %T] "
setopt EXTENDED_HISTORY
setopt HIST_FIND_NO_DUPS
setopt HIST_IGNORE_ALL_DUPS

Look here for a better explanation

[kljensen]

I wanted the same feature (removing repeated lines after fzf filtering) and wrote this small Rust program to satisfy my need: semiuniq. The program removes repeated lines by scanning a window of user-specified fixed size over the input in a single pass without sorting. I'd appreciate comments and pull requests. cc/ #49, #88, #492.