Elasticpress: Enabling Search Result Highlighting/Context
Have you searched for similar issues before submitting this one?
Yes
Is this a bug, question or feature request?
Question (possible feature request?)
Describe the issue you encountered:
I'm trying to enable search highlighting/context in my search results without success. I tried the suggestion in #684, but it is not working for me (the highlight fields are not returned).
Essentially, I'd like it to return the same highlight fields as I get when I do a DSL query. Example:
DSL Query Portion -
"highlight" : {
"fragment_size" : 100,
"fields" : {
"post_title" : { "number_of_fragments" : 0, "pre_tags" : ["<span class='keyword-highlight'>"], "post_tags" : ["</span>"] },
"post_content" : { "number_of_fragments" : 3, "pre_tags" : ["<span class='keyword-highlight'>"], "post_tags" : ["</span>"] }
}
}
Sample Result -
"highlight": {
"post_title": [
"This is the <span class='keyword-highlight'>Title</span> of my Really <span class='keyword-highlight'>Neat</span> Post"
],
"post_content": [
"Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod <span class='keyword-highlight'>title</span> incididunt ut labore et dolore magna aliqua",
"ut enim ad minim veniam, <span class='keyword-highlight'>title</span> nostrud exercitation ullamco laboris nisi ut aliquip ex ea <span class='keyword-highlight'>neat</span> consequat",
"sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit <span class='keyword-highlight'>neat</span> id est laborum"
]
}
Has anyone gotten this to work?
Current WordPress version: 4.9.8
Current ElasticPress version: 2.8.1
Current Elasticsearch version: 6.3.0
Where do you host your Elasticsearch server: Private Ubuntu instance, modified docker-elk
Other plugins installed (WooCommerce, Simple Redirect Manager, etc..): Many, however, I do not believe that any are interfering as the function scoring is working... just not highlighting.
Steps to reproduce:
Here is my code...
add_filter( 'ep_formatted_args', 'my_search_request_args', 10, 2 );
function my_search_request_args( array $es_query, array $wp_query_args = [] ) {
// Source: https://github.com/10up/ElasticPress/issues/226#issue-63436693
$gauss = new \stdClass();
$gauss->post_date = [ 'scale' => '180d', 'offset' => '90d', 'decay' => 0.75 ];
$function_score = array(
'function_score' => array(
'functions' => [ [ 'gauss' => $gauss ] ],
'score_mode' => 'multiply',
'query' => $es_query[ 'query' ]
)
);
// Reference: https://github.com/10up/ElasticPress/issues/684#issuecomment-275834011
$highlight_settings = array(
'fragment_size' => 100,
'fields' => [
'post_title' => [ 'number_of_fragments' => 0, 'pre_tags' => [ '<span class="keyword-highlight">' ], 'post_tags' => [ '</span>' ] ]
'post_content' => [ 'number_of_fragments' => 3, 'pre_tags' => [ '<span class="keyword-highlight">' ], 'post_tags' => [ '</span>' ] ]
]
);
$es_query['query'] = $function_score; // This is working
$es_query['highlight'] = $highlight_settings; // This is not
return $es_query;
}
(Obviously, I'll need to do some additional processing on the fragment results, such as removing unwanted HTML tags, etc. I'm not worried about that. I would just like the highlight fields returned.)
Thank you.
dmhendricks
All 15 comments
Hey @dmhendricks,
I had the same issue, i added two hooks in order to get the highlight result on the final post object.
There is a basic example:
add_filter('ep_retrieve_the_post', function ($post, $hit) {
if (!empty($hit['highlight'])) {
$post['highlight'] = $hit['highlight'];
}
return $post;
}, 10, 2);
add_filter('ep_search_post_return_args', function (array $post_return_args) {
if (!in_array('highlight', $post_return_args)) {
$post_return_args[] = 'highlight';
}
return $post_return_args;
});
juxR
on 1 Mar 2019
@julienroland This didn't work for me.
speedwheel
on 2 Mar 2019
@speedwheel when using the elasticpress debug bar, do you see the highlight in the elastic response ?
juxR
on 2 Mar 2019
@julienroland I don't see it, I also did a print_r on both $post_return_args and $post and there is no key/property with the name 'highlight'.
speedwheel
on 2 Mar 2019
@speedwheel i'm guessing there something wrong with your query.
Could you display your ep_formatted_args filter ?
juxR
on 2 Mar 2019
@julienroland Hello, here is the output, (I am using the default wp search):
Array
(
[from] => 0
[size] => 12
[sort] => Array
(
[0] => Array
(
[_score] => Array
(
[order] => desc
)
)
)
[post_filter] => Array
(
[bool] => Array
(
[must] => Array
(
[0] => Array
(
[terms] => Array
(
[post_mime_type] => Array
(
[0] => application/pdf
[1] => application/vnd.ms-powerpoint
[2] => application/vnd.openxmlformats-officedocument.presentationml.presentation
[3] => application/vnd.ms-excel
[4] => application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
[5] => application/msword
[6] => application/vnd.openxmlformats-officedocument.wordprocessingml.document
[7] =>
)
)
)
[1] => Array
(
[terms] => Array
(
[post_type.raw] => Array
(
[0] => post
[1] => page
)
)
)
[2] => Array
(
[terms] => Array
(
[post_status] => Array
(
[0] => inherit
[1] => publish
)
)
)
)
)
)
[aggs] => Array
(
[terms] => Array
(
[filter] => Array
(
[bool] => Array
(
[must] => Array
(
[0] => Array
(
[terms] => Array
(
[post_mime_type] => Array
(
[0] => application/pdf
[1] => application/vnd.ms-powerpoint
[2] => application/vnd.openxmlformats-officedocument.presentationml.presentation
[3] => application/vnd.ms-excel
[4] => application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
[5] => application/msword
[6] => application/vnd.openxmlformats-officedocument.wordprocessingml.document
[7] =>
)
)
)
[1] => Array
(
[terms] => Array
(
[post_type.raw] => Array
(
[0] => post
[1] => page
)
)
)
[2] => Array
(
[terms] => Array
(
[post_status] => Array
(
[0] => inherit
[1] => publish
)
)
)
)
)
)
[aggs] => Array
(
[category] => Array
(
[terms] => Array
(
[size] => 10000
[field] => terms.category.slug
)
)
[post_tag] => Array
(
[terms] => Array
(
[size] => 10000
[field] => terms.post_tag.slug
)
)
[post_format] => Array
(
[terms] => Array
(
[size] => 10000
[field] => terms.post_format.slug
)
)
[testimonial_tag] => Array
(
[terms] => Array
(
[size] => 10000
[field] => terms.testimonial_tag.slug
)
)
[resource_category] => Array
(
[terms] => Array
(
[size] => 10000
[field] => terms.resource_category.slug
)
)
[faq_tag] => Array
(
[terms] => Array
(
[size] => 10000
[field] => terms.faq_tag.slug
)
)
[program_category] => Array
(
[terms] => Array
(
[size] => 10000
[field] => terms.program_category.slug
)
)
)
)
)
[query] => Array
(
[function_score] => Array
(
[query] => Array
(
[function_score] => Array
(
[query] => Array
(
[bool] => Array
(
[should] => Array
(
[0] => Array
(
[multi_match] => Array
(
[query] => school
…
@speedwheel There is no highlighting in your query.
You have to use a filter for that: https://github.com/10up/ElasticPress/issues/684#issuecomment-275834011
Example:
$es_query['highlight'] = [
'pre_tags' => ['<span class="highlight">'],
'post_tags' => ['</span>'],
'fields' => [
'post_title' => [
'no_match_size' => 100
],
'post_content' => [
'matched_fields' => ['post_content', 'post_excerpt'],
'no_match_size' => 100
],
]
];
@julienroland Thanks! it's working now. The only issue is that it's also displaying the Gutenberg comment tags in the excerpt: -- wp:heading -->.
speedwheel
on 6 Mar 2019
@speedwheel You could clean the post_content field like that:
/**
* @param array $post
* @return array
*/
public function cleanPostBeforeIndexation(array $post = [])
{
$post['post_title'] = wp_strip_all_tags($post['post_title']);
$post['post_content'] = wp_strip_all_tags($post['post_content']);
$post['post_excerpt'] = wp_strip_all_tags($post['post_excerpt']);
return $post;
}
add_filter('ep_pre_index_post', [$this, 'cleanPostBeforeIndexation']);
add_filter('ep_post_sync_args_post_prepare_meta', [$this, 'cleanPostBeforeIndexation'], 10, 2);
@julienroland Thanks I will try it.
Do you know and why sometimes words that are not related to my search are being highlighted? In this case I searched for "bully", but the word "fully" also got highlighted:
speedwheel
on 6 Mar 2019
@speedwheel I have updated my last answer by the way (added a new filter)
Yes it's because de post_content has fuziness in the query. You can change the way the highlight works by updating the highlight query:
$es_query['highlight'] = [
'pre_tags' => ['<span class="highlight">'],
'post_tags' => ['</span>'],
'fields' => [
'post_content' => [
'highlight_query' => [
'bool' => [
'must' => [
'match' => [
'post_content' => [
'query' => get_query_var('s', ''),
]
]
],
],
],
],
]
];
@julienroland thanks that worked, is there any way to adjust the fuziness? because I am getting a lot of irrelevant search results after 2-3 pages.
speedwheel
on 6 Mar 2019
@speedwheel
You can change the fuzziness using this filter:
```
add_filter('ep_fuzziness_arg', function ($fuzziness, array $search_fields = [], array $args = []) {
return 0;
}, 10, 2);
```
Here is a part of the documentation about fuzziness:
Of course, the impact that a single edit has on a string depends on the length of the string. Two edits to the word hat can produce mad, so allowing two edits on a string of length 3 is overkill. The fuzziness parameter can be set to AUTO, which results in the following maximum edit distances:
0 for strings of one or two characters
1 for strings of three, four, or five characters
2 for strings of more than five characters
Of course, you may find that an edit distance of 2 is still overkill, and returns results that don’t appear to be related. You may get better results, and better performance, with a maximum fuzziness of 1.
juxR
on 6 Mar 2019
@julienroland Your are awesome! thank you, this is what I was looking for.
speedwheel
on 6 Mar 2019
Thank you so much, @julienroland! You rock!
dmhendricks
on 7 Mar 2019
Related issues
blindpet
·
7Comments
brunocrosier
·
3Comments
dnikola
·
6Comments
keg
·
4Comments
vnmedeiros
·
3Comments
Most helpful comment
Hey @dmhendricks,
I had the same issue, i added two hooks in order to get the highlight result on the final post object.
There is a basic example: