Elasticpress: Fuzziness broken in v3.5

[scooterlord]

Fuzziness has stopped working in latest version. From what I saw the main post query has changed, but can't seem to be able to add fuzziness back and get it to work.

Sample function:

function change_fuz( $formatted_args, $args ) {
  $formatted_args['query']['bool']['should'][0]['multi_match']['fuzziness'] = 2;
  $formatted_args['query']['bool']['should'][1]['multi_match']['fuzziness'] = 2;
  return $formatted_args;
}
add_filter( 'ep_formatted_args', 'change_fuz', 10, 2 );

The query then is printed as expected, but is not working.

    [query] => Array
        (
            [bool] => Array
                (
                    [should] => Array
                        (
                            [0] => Array
                                (
                                    [multi_match] => Array
                                        (
                                            [query] => peuget
                                            [type] => phrase
                                            [fields] => Array
                                                (
                                                    [0] => post_title
                                                    [1] => post_excerpt
                                                    [2] => post_content
                                                )
                                            [boost] => 3
                                            [fuzziness] => 2
                                        )
                                )
                            [1] => Array
                                (
                                    [multi_match] => Array
                                        (
                                            [query] => peuget
                                            [fields] => Array
                                                (
                                                    [0] => post_title
                                                    [1] => post_excerpt
                                                    [2] => post_content
                                                )
                                            [type] => phrase
                                            [slop] => 5
                                            [fuzziness] => 2
                                        )
                                )
                        )
                )
        )

[tlovett1]

Hey, check out the search algorithm upgrade notice in the readme. You can revert back to the old algorithm with the following code:

add_filter( 'ep_search_algorithm_version', function() { return '3.4'; } );

[theodejager]

Is there a way to still hook into the fuzziness args or something that would have a similar effect? Right now search queries with 1 spelling mistake return no results by default.

[scooterlord]

Also, how often will updates to the algorithm will happen? I just finished my app and the update broke the whole search mechanism.

[brandwaffle]

@scooterlord very, very rarely. In fact, this is the first time I can remember that we've updated the algorithm. The change came about because a lot of folks see very irrelevant results near the end of the search listings due to the previous fuzzy search algorithm. That said, we also knew folks might need things to continue working "as-is", hence the new filter to set the algorithm version.

If you do want to leverage the new algorithm but add back some fuzzy matching, you'll want to copy the particular multi_match array you want to bring over from the old algorithm, and then append that entire array to the current search. The reason is that phrase (the type of search we perform in the new algorithm) actually doesn't support fuzziness as an argument, so adding it as a parameter to the existing (new) multi_match arrays won't do anything.

[scooterlord]

@brandwaffle So, I just replicate the 'old' query through ep_formatted_args into the new one?

I wish there was some new documentation with pointers to elasticsearch where required; currently whenever I try to fix one thing, it breaks another, eg fuzziness not working with phrase, fuzziness not working with cross_fields, wildcards not working, edgeNgram breaks my synonyms. A simple guide with a few examples would promote the plugin - tbh, I can't really find my way through the current documentation without some examples and I feel that some things can be changed within many functions, so whenever I am trying to test something I am not really sure if I added it in the 'right' function so that I don't break anything else.

[scooterlord]

@brandwaffle Hello again, I have a local version of my online website that uses the Elasticpress version 3.4.3 but links to the same EP Server. All the previous functions are broken, (inc fuzziness) which leads me to believe that the data is also sent differently on the EP Server? I am only looking for validation for this, so I know how I can handle it.

Query monitor gives the following error:

400 Bad Request

    HTTP API Transport: curl
    DNS Resolution Time: 0.1153
    Connection Time: 1.1815
    Transfer Start Time (TTFB): 1.5449
    Response Size: 458 B
    Response Content Type: application/json; charset=UTF-8
    IP Address: <ip>

Same also applies if I use the 'old' query on the new version, or even if I change something on the 'new' query, like add fuzziness, etc

This essentially means, that I can't copy the previous query and append it, unless I am missing something.

[brandwaffle]

@scooterlord if the query throws a 400 then ElasticPress falls back to WP search which is probably why you're seeing no fuzziness. My guess is something about the copied query is off and causing a faulty query. If you're running 3.4.3 there is nothing there that could be using the 3.5 algorithm and fuzziness is enabled if the query does not fail.

To get back to what you're trying to accomplish: if you just want the older query algorithm in 3.5 you need to apply the single one line filter mentioned previously. You do not want to copy the old query and filter it in or anything like that. Just add the one line filter to specify the previous algorithm.

[theodejager]

@brandwaffle the new search algorithm is certainly better at reducing irrelevant results, but is there any way to configure it so that queries with minor spelling mistakes still return results?

[brandwaffle]

@theodejager you can add the final of the 3 search queries back into the new algorithm: https://github.com/10up/ElasticPress/blob/0ef93c49043c35712b558e62fa82c8699467fea2/includes/classes/Indexable/Post/Post.php#L1148-L1162

Note that re-adding fuzzy matching will cause a lot of false matches with shorter search terms. For example, "dogs" can become "does" "logs" "bogs" "hogs" etc, etc. It can get messy fast 😄

[felipeelia]

At this point, it seems we've answered all the questions about the new search algorithm. That said, I'm closing the issue. Thank you all for contributing!