Cms: Couple of Cyrillic chars break search sql queries

[mint-jar]

Description

Hi, We encountered a strange issue while using the search functionality in GraphQL.

A query exception occurs when we try to search for terms with one of the following character: ы,э

After some digging I found an issue with craft\helpers\StringHelper::asciiCharMap. It seems this one converts ы to y'. The ' at the end breaks the query later on. I was able to fix this locally by overriding the function craft\services\Search::_normalizeTerm with the following:

    private function _normalizeTerm(string $term, $siteId = null): string
    {
        static $terms = [];

        if (!array_key_exists($term, $terms)) {
            if ($siteId && !is_array($siteId)) {
                $site = Craft::$app->getSites()->getSiteById($siteId);
            }
            $terms[$term] = SearchHelper::normalizeKeywords($term, [], true, $site->language ?? null);
        }

        // trim the quote 
        foreach ($terms as $key => $value) {
            $terms[$key] = trim($value, "'");
        }


        return $terms[$term];
    }

Steps to reproduce

1.Do the following GraphQL request:

query searchEntries {
    entries(
        search: "*ы",
        orderBy: "score DESC"
    ) {
        id
        typeHandle
        title
        slug
        postDate
        score: searchScore
    }
}

Expected: response

Result: An exception is thrown and the following is returned

{"errors":[{"message":"SQLSTATE[42601]: Syntax error: 7 ERROR: unterminated quoted string at or near \"') AND \"siteId\" = 3 AND \"elementId\" IN (3,24,26,28,30,32,34,42,50,58,66,74,82,90,98,106,114,122,130,138,146,154,162,170,178,186,376,382,388,394,5,461,501,517,541,549,559,571,573)\"\nLINE 1: ... * FROM \"searchindex\" WHERE (\"keywords\" LIKE '%y'%') AND \"si...\n ^\nThe SQL being executed was: SELECT * FROM \"searchindex\" WHERE (\"keywords\" LIKE '%y'%') AND \"siteId\" = 3 AND \"elementId\" IN (3,24,26,28,30,32,34,42,50,58,66,74,82,90,98,106,114,122,130,138,146,154,162,170,178,186,376,382,388,394,5,461,501,517,541,549,559,571,573)"}]}

Additional info

• Craft version: 3.6.2
• PHP version: 7.4.14
• Database driver & version: PostgreSQL 12.2
• Plugins & versions:
  
  
  
  • Azure Blob Remote Volume 1.1.1
  
  
  • Embedded Assets 2.4.5
  
  
  • Redactor 2.8.5
  
  
  • Sentry 1.5.1
  
  
  • Sprout Sitemaps 1.3.0
  
  

[brandonkelly]

Strange. I just submitted an issue on the voku/portable-ascii repo about that (https://github.com/voku/portable-ascii/issues/58), and added a little patch to Craft for the next release to work around it in the meantime.

To get the fix early, change your craftcms/cms requirement in composer.json to "dev-develop as 3.6.3"and run composer update.

[mint-jar]

Whoa! You are freakin fast!

Thanks. I'll try the fix.

[brandonkelly]

Craft 3.6.4 is out now with that fix ✨

[mint-jar]

:star_struck: Even better! Thank you!