Amplify-cli: Support ElasticSearch fuzzy query

[starpebble]

Is your feature request related to a problem? Please describe.

Users often misspell search terms. Websites offer search features where the user types in a string, guessing the spelling and whitespacing. It should be easy to provide entity search with fuzzy matching.

Describe the solution you'd like

For @searchable entities, please support ElasticSearch's fuzzy match.

Example - searching a 'Company' entity with an attribute 'name.'

const searchCriteria = {
            filter: {
                name: {
                    fuzzyMatch: 'Amazin'
                }
            }
        };

const allMatches = await API.graphql(
  graphqlOperation(
     queries.searchCompanys, 
     searchCriteria
   )
);

When the Company collection has a record for a company with the 'name' attribute, the response contains elasticsearch's fuzzy matches. For example, the record 'name':'Amazon' will fuzzy match with the default fuzziness of 2.

Documentation on the fuzzy match query:
https://www.elastic.co/guide/en/elasticsearch/reference/6.2/query-dsl-fuzzy-query.html

Amplify's JS should permit a request to specify the elasticsearch fuzzy match attributes: 1. fuzziness (int) and 2. transposition (boolean)

const searchCriteria = {
            filter: {
                name: {
                    fuzzyMatch: 'Amazin',
                    fuzziness: 3,
                    transpositions: true
                }
            }
        };

Describe alternatives you've considered

1. Write a custom js react component that searches the elastic search database.

This is difficult to create because it complicates the web application security model. Amplify's security model, where the web app is authorized to call aws appsync hosted APIs, is easier.

1. Attaching an AWS CloudSearch domain to the DynamoDB table is also an alternative.

The dynamodb stream would push updates to a second service, aws cloudsearch. Lot's of custom code sucks.

Additional context

AWS Amplify supports ElasticSearch's 'match' and 'matchPhrase' however these are not fuzzy matches.

Elastic Search is really awesome with faceted search. AWS Amplify should also provide better search capabilities with user entered strings.

[YikSanChan]

I believe this can be done by adding a custom resolver (in VTL unfortunately) that takes the fuzziness and query as parameters and queries ES with a complicated query. But this is not ideal. This should be better done by the framework.

[mapkbalaji]

Are fuzziness and transpositions gonna be added? If not, please guide me to achieve the same.

[starpebble]

I agree, more framework support may be best.

New option: The framework has a new feature - lambda resolvers. For custom resolvers.

[stdmn]

+1 on this. Having 'fuzzy' available in @searchable out of the box would be ideal

[thenderson55]

Any comment or updates on this?
Will fuzziness be added?
What is the best current way to do it?

[starpebble]

@thenderson55 I simply don't have the teeniest idea.

[starpebble]

I guess this will just be possible with a custom resolver alone.

[VincentDevelopment]

Hi! can anybody give ma any hint about how to implement fuzzy search?!
Would be great!

[starpebble]

@VincentDevelopment It's possibly always going to be a custom AppSync resolver with one Lambda function.

Two hints:

• Title: Combining GraphQL Resolvers

Attach a custom Lambda resolver to searchPosts. Everybody's on their own, there's no fuzzy search code in the tutorial to put in the custom resolver Lambda.

• Title: Fuzzy Query

See the JSON example with "fuzzy".

I simply haven't heard anything from the maintainers. That means the amplify-cli can't exactly do it.

[VincentDevelopment]

@starpebble Thanks a lot! I will give it a try! :) Hopefully i will manage to do this since i am quite new to amplify and appsync and some times ive got the feeling that documentations are a bit slim and hard to find...

[starpebble]

@VincentDevelopment Keep going! Possibly search for examples on github with AppSync, Amplify, and Bonsai. Bonsai is a less expensive ElasticSearch host. Consider asking question on StackOverflow when something is hard to find.

[PabloLMartinez]

Hey guys, you should try to use "matchPhrasePrefix" filter and you will get a similar result like "fuzzyMatch" filter. I've just tryed in my project and it works like a charm.

     filter: {
            name: {
                matchPhrasePrefix: 'Amazin'
            }
        }

[starpebble]

@PabloLMartinez Neat customization! That's some amazing autocomplete magic, that little match query.

[asmiki]

I found it useful to use resolver so you get full control over Elastic Search query execution. Here is a simple solution that takes FUZZINESS parameter

UPDATE - schema.graphql

type UnitConnection {
  items: [Unit]
  nextToken: String
}

type Query {
  unitSearch(keyword: String!): UnitConnection
}

UPDATE - stacks/CustomResources.json

"Resources": {
                "QueryUnitSearch": {
            "Type": "AWS::AppSync::Resolver",
            "Properties": {
                "ApiId": {
                    "Ref": "AppSyncApiId"
                },
                "DataSourceName": "ElasticSearchDomain",
                "TypeName": "Query",
                "FieldName": "unitSearch",
                "RequestMappingTemplateS3Location": {
                    "Fn::Sub": [
                        "s3://${S3DeploymentBucket}/${S3DeploymentRootKey}/resolvers/Query.unitSearch.req.vtl",
                        {
                            "S3DeploymentBucket": {
                                "Ref": "S3DeploymentBucket"
                            },
                            "S3DeploymentRootKey": {
                                "Ref": "S3DeploymentRootKey"
                            }
                        }
                    ]
                },
                "ResponseMappingTemplateS3Location": {
                    "Fn::Sub": [
                        "s3://${S3DeploymentBucket}/${S3DeploymentRootKey}/resolvers/Query.unitSearch.res.vtl",
                        {
                            "S3DeploymentBucket": {
                                "Ref": "S3DeploymentBucket"
                            },
                            "S3DeploymentRootKey": {
                                "Ref": "S3DeploymentRootKey"
                            }
                        }
                    ]
                }
            }
        }
},

NEW - resolvers/Query.unitSearch.req.vtl

## Query.unitSearch.req.vtl
## Objects of type Unit will be stored in the /unit index

#set( $indexPath = "/unit/doc/_search" )
{
    "version": "2017-02-28",
    "operation": "GET",
    "path": "$indexPath.toLowerCase()",
    "params": {
        "body": {
            "query": {
                "match" : {
                    "title" : {
                        "query" : $util.toJson($ctx.args.keyword),
                        "fuzziness": "AUTO"
                    }
                }
            }
        }
    }
}

NEW - resolvers/Query.unitSearch.res.vtl

## Query.unitSearch.res.vtl

#set( $items = [] )
#foreach( $entry in $context.result.hits.hits )
  #if( !$foreach.hasNext )
    #set( $nextToken = "$entry.sort.get(0)" )
  #end
  $util.qr($items.add($entry.get("_source")))
#end
$util.toJson({
  "items": $items,
  "total": $ctx.result.hits.total,
  "nextToken": $nextToken
})

HTH!