Fuse: Return whole list instead of empty array if no search term is set

[christianmalek]

I'm using fuse.js with the includeMatches option to easily highlight matches. The problem is that the usage of includeMatches changes the structure of the list because it has to includes the matches in a fitting manner:

// with includeMatches (the structure changed to include the matches)
[
  {
    "item": {
      "title": "Old Man's War",
      "author": {
        "firstName": "John",
        "lastName": "Scalzi"
      }
    },
    "matches": [
      {
        "indices": [
          [
            0,
            12
          ]
        ],
        "value": "Old Man's War",
        "key": "title",
        "arrayIndex": 0
      }
    ]
  }
]

// without includeMatches
[
  {
    "title": "Old Man's War",
    "author": {
      "firstName": "John",
      "lastName": "Scalzi"
    }
  }
]

My problem is: I want to output the whole list if no search term is set. But fuse.js returns (for logical reasons) an empty array. Instead of this I need the whole list with the matches structure. Otherwise I need to treat it different.

Is it possible to return the whole list with empty matches if no search term is set?

[krisk]

This seems like quite an odd request 😅 If not items are found, then the search result should be empty. Returning the complete list in the same output structure as if there were any results would be contrary to the purpose of this library.

[d3vr]

This can be useful. Here is an example: suppose you're showing a list of contacts and you want to use a search bar to filter it. If the search term is empty display all the list, otherwise display only the filtered results.

Is there any way to achieve this?

[devgg]

I would also be interested in this!

[ketysek]

@d3vr definitely mentioned the correct use case

[ljm42]

Can we reopen this @krisk ? Getting back the structured array (with "item") when no search term is set would be helpful.

[Brandoncapecci]

You can achieve this by just mapping over the input array yourself. The resulting output structure is hardly very different...

const correctlyShapedArray = inputArray.map(val => ({
    item: Object.assign(val, {}),
    matches: [],
    score: 1
}));

[w-etc]

Also interested in this.

[chrisrowe]

+1 on this

[gjunming]

Also interested in this.

[klandell]

+1

[RizwanAslam]

+1

[acnebs]

https://github.com/krisk/Fuse/issues/229#issuecomment-441907010 seems to be the best solution for the time being.

As an aside @Brandoncapecci, I'm wondering why you used item: Object.assign(val, {}), instead of just item: val,?

[Griefchief]

+1 as well, this is about to make a mess out of my stateless React component since now I have to send a ton more stuff into it through the stateful wrapper component instead of just setting this as an option in Fuse.

Or to hack it in the stateless react component by accessing Fuse.list array and hope Fuse is meant to be used this way as per Brandoncapecci's comment.

[fekstr]

+1

[reinvanimschoot]

+1

[indexcoder]

+1

[krisk]

I wonder if this could be handled simply outside of Fuse. Personally, I don’t think you should change the format of the results (when there aren’t any, and thus default to the original list). I think that it is something that should be handled separately. If you’re using a state container (like redux), perhaps you can update a state property to true whenever Fuse returned values, and you render those, otherwise set it to false, and render the original list.

One such way perhaps:

const setup = (items, options) => {
  // Pre-precess the items first...
  let fuse = new Fuse(items, options)

  const search = patten => {
    let results = fuse.search(pattern)
    if (!results.length) {
      // use the search results
      setState({ foundMatches: true, results })
    } else {
      // use the default items
      setState({ foundMatches: false, results: items })
    }
  }

  return search
}

And you could use it like this:

const search = setup(items, options)

// And search..
search(/* query */)
search(/* another query */)

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days

[acnebs]

@krisk I think the use-case here for most people is that they are using Fuse for "filtering" behaviour rather than just searching, in that you have some list of items which are displayed to the user, the user enters a search term, and now you have a smaller list of items. When it comes to rendering these results to your UI, you don't want to have two different rendering patterns for the different cases – you just want to have a single for loop that renders either every item or some subset of the items. This is much easier if Fuse has an option to spit out a "formatted" version of the data when no search term is specified, as it allows better separation of your data logic from your view logic.

[Winangel]

I agree with @acnebs, we are using fuse for mention purpose, we need to display results as soon as the user hit @ or # key, the search string is empty at this moment and we need to handle this edge case in a very odd manner when it could just be handled in the same manner as the general case...

[danielfdickinson]

I disagree. Of course I'm biased in that I'm using this for a website search and displaying every page with no search pattern would be exactly the wrong behaviour. If this is going to done it should be as an option not as the only, or even default, behaviour.

[acnebs]

@cshoredaniel I don't think anyone wanted this to be the default behavior (esp. because that would be a breaking change which is unnecessary in this case).

So yes, my own conception of this is that it would be an option, and I assume that applies to everyone else here as well.

[danielfdickinson]

@acnebs It wasn't clear from the comments that making it optional was the ask (not explicitly mentioned by anyone, and it looked to me like some were arguing that it was the main use case). I think as an option there is no problem with this from a user perspective. @krisk might have concerns from maintainability / appropriateness for the library.

[Segfaultd]

Any chance this is getting attention? We need that feature! :D

[krisk]

Thinking about this, I suppose if there’s an option, then yes, we could make it such that when the search string is empty, then the fuse instance would return the entire fuse-formatted list.

Clearly there’s appetite for this feature. I will thus include this as a minor update for next release.

Now someone please tell me what should that option actually be called? 😅

[Griefchief]

returnAllOnEmptyString?

On Fri, 29 May 2020 at 07:52, Kiro Risk [e-mail] wrote:

Reopened #229 https://github.com/krisk/Fuse/issues/229.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/krisk/Fuse/issues/229#event-3385725478, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ABL5WCH4DJVA3BI4OE243JTRT5ESHANCNFSM4EWAYD3Q
.

[tommedema]

matchIfEmpty

[krisk]

_I anticipate a few reactions of disappointment. Emotionally bracing myself._

So, I've been thinking more about this, and have a working implementation. However, I don't think this should be built-in behavior for Fuse, anymore.

Folks have mentioned the use-case of Fuse being used as filtering, as opposed to just search. If so, this could very well be done in userland, by simply applying a custom filtering function directly:

const search = (query) => {
  let results = fuse.search(query)
  // If there are results, great.
  if (results.length) {
    return results
  }

  // If not, create a list of docs conforming to the Fuse format.
  // (But probably cache it for optimization)
  return myList.map((doc, idx) => ({
    item: doc,
    score: 1,
    refIndex: idx
  }))
}

// Usage
const fuse = new Fuse(myList, { ... })

search('hello')

I don't think this is a hack at all. The Fuse document format is after all already a public API (i.e, this is how docs are parsed by clients anyway).

Here's an alternative with caching:

const searchFilter = (fuse) => {
  const list = myList.map((doc, idx) => ({
    item: doc,
    score: 1,
    refIndex: idx
  }))

  return (query) => {
    let results = fuse.search(query)
    return results.length ? results : list
  }
}

// Usage
const fuse = new Fuse(myList, { ... })
const search = searchFilter(fuse)

search('hello')

[acnebs]

What is the argument against including this behavior in the lib?

Seems like it is a common enough use-case where it makes sense to just include this fairly simple behavior (as your code shows) so that everyone doesn't need to re-implement the same pattern in userland.

[krisk]

I mean, I added the feature. You can play around it with it in [email protected], by setting matchEmptyQuery to true.

My rationale for not wanting this:

Unexpected behavior: Is this an appropriate behavior for a search library? I mentioned that I don't think Fuse.js should include things that can be trivially done in userland, while also striving to keep the library purely for searching purposes. For example, would you expect Array.filter to return the whole array when no item is matched? Even if it were optional, would that be intuitive? Would you expect the same behavior from other search libraries, or would you create a helper function to enable such?

Feature creep: All options and features of Fuse.js are to maximize its fuzzy searching potential. The "return all when empty" option is something that is not necessary for its proper functioning.

Binary size: This also contributes to the size of the library, for little value, because, again, it can be done in userland, and the vast majority of use cases don't actually need this. In isolation, this may be totally minor, but we should be careful in setting the precedent that just because something is a common-enough use case, it should be added to the library.

Performance and maintainability implications: Generating the formatted list every time isn't ideal, so it would be better to have an internal cached copy. This then requires syncing between copies whenever an item is removed/added. This will start to get complicated and will require re-engineering to accommodate the extra functionality.

Alternative

Aside from what I suggested in previous comments, maybe what we'd need is more of a plugin system, where people can create their own Fuse.js plugins, and publish them for public use. We could even have a list of official plugins.

Conclusion

Ultimately, I'll keep this in beta release, but it will not be published in latest.

(cc: @cshoredaniel for thoughts on this)

[danielfdickinson]

@krisk I agree with your rationale for not wanting in Fuse proper.

I don't think a plugin system is quite the right answer though, but rather a second 'wrapper' library (possibly) with plugins (different repo):

• plugins imply even more of a promised API which makes maintenance etc more difficult.
• a plugin API tends to want access to (and guarantees from) more of the internals
• plugins for enhancing search functionality would fit a search library; plugins for cosmetics wouldn't IMO.
• since, so far, most or all of the 'cosmetics'/'extras' can be implemented in userland they could just as well be implemented in a wrapper library for those who need a library because of their framework or whatever.
• basically the 'fuse-tools' library is a convenience library and keeping it separate from fuse proper encourages better library design, keeps the main library smaller, and avoids scope creep in the fuse library.

Just my 2¢

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days

[9mm]

I definitely really think there is a good use case for this to be added, for all the above mentioned reasons, and in addition, I have ~45 views in my app, all which cache data. This means that I need to cache TWICE the amount of data, simply because I can't extract the rows out of fuse without applying searching, so I need to store the original data set as well and keep it in memory just so I can show that. This to me is a pretty huge impact for performance, especially when some tables have 15,000 rows. Storing 30K rows simply because I can't retrieve all the data without applying a search feels like pure madness for a library that is used for filtering probably JUST as much as searching.

I do however think that most of this thread might be thinking about it from a bit of a backwards way... I wouldn't think of this feature as return all results when search term is empty, that would be crazy for a search library and is very mind warping to think about given the context of "searching". I would instead think of it as show all results if no search term is specified. Honestly, I would even settle for a method like getCurrentIndex or something, and solve it by checking if search query is blank or not.

What about passing something like false into search() instead of a string?

[james-dillon]

@krisk +1 for adding a filtering option. (Yes, I'm using fuse for fuzzy filtering). The fact that fuse.js inserts "item" at the top of the tree for every item in the original array makes it quite difficult to manage in React. The restructured data means I need data.id for the original data and data.item.id for the filtered data. From a developer perspective this is very frustrating. (For search it's fine; for filtering, you expect to get your data back in the structure it started in, just filtered.)

If you ultimately decide not to make this part of fuse.js, could you please add to the documentation the workaround you devised above? It seems there are many devs who really need this to work, and some chatter online from people trying to make it work. So a note in the docs would help a lot.

[9mm]

yes ^ it also already has half the filtering features built in anyway, such as findAllMatches, so feature creep seems weird when you're halfway supporting a feature that isn't even complete

[danielfdickinson]

I think this is a perfect opportunity for folks who care about functionality that (mostly) doesn't actually require changes to fuse to set up a repo for an 'fusejs-utils' or 'fusejs-tools' library that implements the functionality outside of fusejs, but gives interested developers a library to use (since that seems to be a wish of many).

Also, I was under the impression that something like getCurrentIndex() already existed in order to enable the reuse of the index. @krisk Am I mistaken on that?

[9mm]

@cshoredaniel so to be clear, you think a searching/filtering library which can deal with 50,000 rows should need a whole extra separate library to simply show the results without searching? Otherwise, we should store literally twice the amount of data simply to show the results when no search is applied...? I can't wrap my head around that logic... the way you use fuse is by initializing it with an array (rather than manipulating the status of an existing array passed in as an argument), therefore it seems fairly logical to think that now that fuse.js is the store of the data, there should be ways to access the data outside of a search term.

[danielfdickinson]

@9mm I may expand this reply later, but for the moment here are a few key points:

1. From What is Fuse.js?, the very first piece of information in the documentation:
   "Fuse.js is a powerful, lightweight fuzzy-search library, with zero dependencies". There no mention of filtering as a library goal. The fact that many have used it that way simply shows the power of a general solution.
2. Despite your reinterpretation of the problem, a null query is not the same as an all query and I believe you are conflating the two.
3. I do think it could be argued that it could be beneficial to alter the API so that a 'wrapper' object is used in order to effectively achieve pass by reference instead of pass by value (which is what gives rise to the duplication problem), although I'd ague for backwards compatiblity this should be implemented via a separate entry point. Also I'd have to recheck the code; I don't recall that the code actually modifies the original array data (which would make pass by reference a problem): @krisk is that a correct recollection? EDIT: On further review of the code there isn't an actual duplication of the original array.
4. This is not the only feature request which I would anticipate being in an additional library. That would indeed be absurd. I believe I have mentioned in previous comments that there are other feature requests that fall outside the scope of the fuse.js library itself, but are of interest. Those I would anticipate being included in a library such as I have discussed.
5. The incredulous 'tone' of your comment doesn't help your case. I would suggest you find other ways to express your disagreement.

[james-dillon]

@cshoredaniel In reply to your pondering, “I don't recall that the code actually modifies the original array data”: yes, indeed, it does. Fuse returns an array of objects, each of which has a property “item” whose value is a row from the original array. So Fuse basically inserts “item” at the base of the tree. For example, data[0].id in the original would become data[0].item.id after Fuse does its magic. If that were not the case, filtering would be fairly trivial to achieve.

The obvious reason for this is that Fuse adds additional properties alongside “item,” like a search ranking score. But again it just makes an otherwise trivial filtering usage quite cumbersome.

[danielfdickinson]

Sorry it’s been a while since I looked at the code. To clarify, does the Fuse alter the original array or does it create a new one?

It does indeed complicate things if the original array is modified.

[danielfdickinson]

So I took a _brief_ look at the code and from what I could see, without ‘includeAllMatches’ the returned dataset is quite small and is returned via a new array (with the index being used to return a reference to the original data). Again with just a quick skim it looks like ‘includeAllMatches’ adds data during the search, but again not the whole original ‘object’ (used loosely to mean anything that is not basic type like ‘int’ or ‘string’). If the origin is an array of strings, then the original object is copied because it is a ‘simple’ object.
refIndex AIUI is used to reference the index in the original array for complex objects so that one can retrieve the full original record if one so desires.

Now this was just a skim so I could be misreading, but I don’t think the array is duplicated or modified in Fuse, but that rather if one does not keep one’s own copy of the array it will be garbage collected as it will no longer be referenced.

[james-dillon]

@cshoredaniel Sorry about the confusion. Fuse appears to create a new array with a different object structure from what is seen in the original array. New array (no problem) with new structure (annoying).

[danielfdickinson]

Apologies for my slightly over-reactive post to 9mm. I've updated the post to be more neutral in hopes this doesn't get out of hand.

[9mm]

It's all good Daniel, a lot of times I am unintentionally harsh, and it just doesn't seem that way "in my head at all", but it definitely can from the outside.

[aleclarson]

@krisk What about when minMatchCharLength equals zero? Could all items be returned then?

[MegaMech]

The dev's could consider making the userland solution easier to find (Edit original comment perhaps?). So that those of us going above and beyond to implement fuse in an incorrect manner can find this valuable addition to our pseudo-searches that in-reality are just trivial filters.

Also, plugin/library that disables filtering functionality and only allows fuse to be implemented properly; as a search, when?

[9mm]

@MegaMech what is the actual userland solution?

[MegaMech]

@9mm This by Krisk:

const searchFilter = (fuse) => {
  const list = myList.map((doc, idx) => ({
    item: doc,
    score: 1,
    refIndex: idx
  }))

  return (query) => {
    let results = fuse.search(query)
    return results.length ? results : list
  }
}

I don't like this style though. It's kinda messy to comprehend what's going on.
This is what I did:

        if (pattern === "") {
            search_sort_table(d.map((doc, idx) => ({
                item: doc,
                score: 1,
                refIndex: idx
            })));
        } else {
            search_sort_table(fuse.search(pattern));
        }

I wonder if the fuse website isn't written very well if people are unknowingly using fuse as a filter. Like how does their example differ from my implementation. I basically just copy/pasted the code. But what I did isn't a search it's a filter? Like what?