Redisearch: Fuzzy Search Number

[xevix]

When searching with a number I get a syntax error

FT.SEARCH idx "%3%"
(error) Syntax error at offset 1 near '3'

Is this not supported?

[xevix]

Ah, this appears to be caused by MIN PREFIX set to 2 by default. Setting to 1 works. Still odd that this is a parse error, but now I know.

[xevix]

Ah, actually that setting only applies to prefix search not fuzzy matching. So this issue is still valid.

[xevix]

Example

FT.CREATE myIdx SCHEMA title TEXT WEIGHT 5.0
FT.ADD myIdx doc1 1.0 FIELDS title 'thing 23'
FT.SEARCH myIdx "%2%"
FT.SEARCH myIdx "%d2%"

Output

OK
OK
(error) Syntax error at offset 1 near '2'
1) (integer) 1
2) "doc1"
3) 1) "title"
    2) "thing 23"

[mnunberg]

This does not only happen with numbers, but also with small letter strings as well.

[foadyousefi]

I see this is an old issue, but the issue still exists. It also happens with stop words for example

FT.SEARCH myIdx "%with%"
FT.SEARCH myIdx "%at%"

This is a problematic. Imagine you have a search field on your website, if you want to fuzzy search all search queries inserted by the users, you must wrap the query between % like "%search% %for% %the% %best% %database%".

There is some workarounds, but all have their disadvantages.

• You can disable stop words totally, but this is not ideal.
• Before converting search query into fuzzy query, you can check if this is a stop word, then do wrap it in between %. Something like "%Search% for the %best% %database%". This also has its own issues. First of all, you can't be always 100% sure about stop words, unless you specify them by yourself. On the other hands, although looping through a list of strings might not seem to much a performance issue, but still, it takes some milliseconds.

Or maybe there is some other ways I'm not aware of.

[foadyousefi]

@mnunberg What is the status on this issue, is there any developments done?

I've created a Wordpress plugin and need to implement fuzzy search.

[mnunberg]

Hi, we're going to be redoing the parsing engine soon and will address that issue then. I haven't forgotten!

[foadyousefi]

@mnunberg What is the status of this issue? Have you re-written the parsing engine?

[MeirShpilraien]

@foadyousefi if its still relevant then this should fix it https://github.com/RediSearch/RediSearch/pull/744.
Let me know if it helps ..

[mnunberg]

we should probably close this

[foadyousefi]

@MeirShpilraien Unfortunately not fixed yet.

It seems the issue with fuzzy searching numbers and small strings fixed, but not stop-words.

[mnunberg]

It seems our parser swallows stopwords.. let me take a stab at it..

[foadyousefi]

The company I'm working with, have created Wordpress plugin and Drupal 8 module. For CMS integration, it will be extremely useful if we could pass in fuzzy with Levenshtein distance like an argument to FT.SEARCH instead of wrapping all individual words in the query. Something like FT.SEARCH myIndex "lorem ipsum" FUZZY 2. ElasticSearch does this.

This way, we just pass in search query directly from the search input into RediSearch query (of course after some validations). Currently, we need to loop through and wrap all individual words in between %.

[K-Jo]

fixed by https://github.com/RediSearch/RediSearch/pull/747 https://github.com/RediSearch/RediSearch/pull/744

[K-Jo]

@xevix should be fixed, please validate but closing this issue

[foadyousefi]

@K-Jo Although the issue with the number only terms and very short strings fixed, the issue with stop-words still exists and not fixed.

Please can you re-open this issue or do I need to open a new one?

[mnunberg]

Open a new issue, please