Spyder: Add fuzzy search functionality to the Variable Explorer

Created on 28 Jan 2017  路  21Comments  路  Source: spyder-ide/spyder

It is hard to explain so let me present a scenario:
You are researching and have many dataframes with various names in your current session. you want to look at some dataframe you just created. printing it is rarely helpful.
Right now, I search for my dataframe in the variable explorer and open it there to have a nicer view of it.
But if I created many variables searching for the one I want becomes tedious.
What I want is the ability to have faster access to the variable by providing the variable's name.
maybe writing:
view(mydataframe)

Or having a shortcut to search all variables in variable explorer (hopefully in a fuzzy manner) and hitting Enter to view it.

Any suggestions to the tired researcher?

Variable Explorer Enhancement

Most helpful comment

I have had a first look at it and here are my first questions & findings.

Assumptions

  1. We are only interested in searching through the variable names (not the types or sizes)
  2. We want a fuzzy finder so unlike the search/find widget for the editor we do not limit to exact matches and we do not need to add the complexity of having options like case sensitive, etc.
  3. We want to avoid having external dependancies

First thoughts

Here is what I first came up with:

  • Add a search button which, when clicked, shows an input text box
  • The user just enters some text as usual
  • We search through the list of shown variable names (only the one that are shown not to introduce confusion but it can be discussed) using difflib.get_close_matches
  • The result is a variable explorer table sorted by relevance based on get_close_matches score

Example

fuzzyfinder

Questions

  1. Sorting the table was what I came up with because it avoids filtering/hidding some variables. However this may not be users would expect. I am open to suggestions here :smile: It could be filtering or simply highlighting the most relevant variable (without sorting) and the up/down icons would help going through the variables by order of relevance.
  2. Should I use something more advanced than difflib?
  3. Are the assumptions correct?

All 21 comments

Cool suggestion!! We'll try to work on it for Spyder 4 :-)

Or you could give it a try for Spyder 3. We already have a fuzzy search library, so it'd be a matter of using that library with the variable explorer.

@goanpeca, what do you think? Should we try to add this in Spyder 3.2?

@ccordoba12 yep sounds like a good idea.

This would be a good moment to make the fuzzy search dialog more generic. The current one use for files makes a lot of file assumptions

I was just thinking on adding a search text box to the Variable Explorer toolbar and connecting it to the variable names.

Or add Ctrl F ?

Lets think this through a bit more. I dont want to clutter the interface.

Good idea. We could add a search icon that when pressed adds a search widget, like the one present in the Editor and consoles :-)

Hi. I was going to post an issue very similar to this one. My suggestion is the following:

1. Since the Variable Explorer do not accept typing when focused, my suggestion is that a key press could highlight the first variable that starts with that letter. If the same key is pressed again it would highlight the second occurrence and so on..

2. If "Ctrl + F" is pressed when Variable Explorer is focused, it could open a search box (maybe the same search box in the editor or a new one at the bottom of the Variable Explorer window) and filter variables that match the search text.

@jnsebgosselin, this could be a fun project for you ;-)

@ccordoba12 Alright, this definitely sounds interesting :). I will take a look into this soon.

@Prikers I said I would work on this one, but I think this might be more suitable for you since you've worked a lot on the variable explorer recently. Besides, I don't know when I'll be able to work on this one. So if you are interested, go for it :+1:

@jnsebgosselin thanks! I'll have a look at it 馃憤

I have had a first look at it and here are my first questions & findings.

Assumptions

  1. We are only interested in searching through the variable names (not the types or sizes)
  2. We want a fuzzy finder so unlike the search/find widget for the editor we do not limit to exact matches and we do not need to add the complexity of having options like case sensitive, etc.
  3. We want to avoid having external dependancies

First thoughts

Here is what I first came up with:

  • Add a search button which, when clicked, shows an input text box
  • The user just enters some text as usual
  • We search through the list of shown variable names (only the one that are shown not to introduce confusion but it can be discussed) using difflib.get_close_matches
  • The result is a variable explorer table sorted by relevance based on get_close_matches score

Example

fuzzyfinder

Questions

  1. Sorting the table was what I came up with because it avoids filtering/hidding some variables. However this may not be users would expect. I am open to suggestions here :smile: It could be filtering or simply highlighting the most relevant variable (without sorting) and the up/down icons would help going through the variables by order of relevance.
  2. Should I use something more advanced than difflib?
  3. Are the assumptions correct?

What you've done looks really good so far!

My thoughts:

Sorting the table was what I came up with because it avoids filtering/hidding some variables.

I'd prefer filtering/hiding. I think it feels more intuitive than sorting. @goanpeca?

Should I use something more advanced than difflib?

We have a filtering library of our own that we use to filter the keyboard shortcuts table. So we'd prefer to use that one instead:

https://github.com/spyder-ide/spyder/blob/master/spyder/utils/stringmatching.py

Are the assumptions correct?

Yep, I think they are correct. I only don't understand why you added up/down arrow buttons to the search field. It makes little sense to me.

I'd prefer filtering/hiding. I think it feels more intuitive than sorting.

I agree, it is more intuitive to hide the stuff that does not match. We could just have the textbox and the close button. Escape key should either apply the same as the closing button or clearing the textbox.

Since this is a find like feature, we need to think of how the UI works. On the editors and console, find-like features appear on the bottom of the widget. So although this is useful I feel we are now moving in 3 separate directions.

Thanks @ccordoba12 and @goanpeca for your feedbacks.

  • After testing for a while, I agree, I prefer to filter the table!
  • I will use get_search_scores. It is slightly less fuzzy but it fits our needs here :+1:
  • I had a look to shortcuts plugin and it seems to be doing the same thing as we are trying to achieve here. It uses a QSortFilterProxyModel. Should I try to move in that direction? I should note that it adds a (hidden) column to the NamespaceBrowserfor the search score but as the NamespaceBrowser comes from the ReadOnlyCollectionsModel, it may add a hidden column to all collections table (if we want to keep the code modification relatively simple). Is that ok?
  • @goanpeca should I make the search box appear on the bottom? It may feel weird since the buttons are on top... On the top it is similar to the find in files widget

I only don't understand why you added up/down arrow buttons to the search field. It makes little sense to me.

  • These are garbage stuff from prior trials, they are not usefull :smile:

@Prikers I added that hidden column as a hack in order to achieve what we wanted. We could probably do the same for the variable explorer, but we would need to create it in the highest parent possible and yes, try to go with the proxy model. Thoughts @ccordoba12 ?

The highest parent possible would be ReadOnlyCollectionsModel. It is a QAbstractTableModel.

Very good job.

Please also consider my previous suggestion to add a quick navigation through the variables:

  1. Since the Variable Explorer do not accept typing when focused, my suggestion is that a key press could highlight the first variable that starts with that letter. If the same key is pressed again it would highlight the second occurrence and so on.

@yuribr84, your suggestion is hard to implement and it provides essentially the same functionality of the search bar, so I don't agree with it it too much.

@yuribr84 what you are suggesting is already kind of possible: if you sort the variable explorer table by variable name and then hit any letter key, it will highlight the first variable that starts with this key. The other variables starting with this key are then following as the table has been sorted.
I know it is not obvious but as @ccordoba12 mentioned it will be complicated to implement something more user-friendly for this use-case.

hello,

if i am not wrong you are talking to add some search/filtering options in the variable explorer. I would like to propose something more: search and filtering inside these variables. this would be very useful when using database as pandas where a single variable could contain thousands, or more, things

maybe i wasn't very clear so here there is an example from R Studio under filtering and searching, also the "advanced topics" is interesting

Was this page helpful?
0 / 5 - 0 ratings

Related issues

danieltomasz picture danieltomasz  路  3Comments

JesterEE picture JesterEE  路  3Comments

Khalilsqu picture Khalilsqu  路  3Comments

cchu08 picture cchu08  路  3Comments

yogu220172 picture yogu220172  路  3Comments