Orchardcore: Lucene Queries documentation

[dodyg]

Lucene Queries documentation needs more work:

• It needs to document on how to pass parameters in the query on the admin UI and via code.
• It is not clear whether it is possible to select what fields to return.
• It is not clear on how to include Content Type parts/fields in the index. The facility is available in the UI but unless you know it, it's not obvious.
• It is not clear how to find the name of the stored field once you enable indexing a Content Type parts/field.
• The UI links directly to the Elastic Search API documentation but it's unclear how many of the Elastic Search syntax is actually supported. _source does not seem to be supported for example.
• There is no general discussion on what Lucene module can be actually used for in Orchard Core. The Blog Theme for example has a Lucene query but it does not have paging.
• In the Admin UI, there are commands to Reset and Rebuild. It's not clear what's the difference between these two.

[Skrypt]

Need examples of using "Liquid" params {{ param }} and using conditions {% if param != null %}.
I think people need to know that they can use Liquid in their Lucene Queries and to understand the fact that a Lucene Query is a Liquid Template.

[dodyg]

It's also not obvious how to perform count query. When you are doing paging e.g. for blog posts, you need total count of all posts.

[Skrypt]

Doing a count is not scalable so it's been intentionally not documented / recommended.

[Skrypt]

Lucene Queries count issue is fixed with #4554

[agriffard]

Here is a basic Lucene query you can run from the admin:

{
  "from": {{ from }},
  "size": {{ size }},
  "query": {
    "terms": { "Content.ContentItem.FullText": ["{{ terms }}"] }
  }
}

with parameters:
{ from: 0, size: 10, terms: "terms" }

See also #4825

[hung-doan]

Hi guys, I have two questions relates to the Lucene as there is no clear documentation relating to the query.

1. As my understanding, we don't use the build-in query syntax of the Lusence. We borrow the DSL query syntax from the elasticsearch. Is that correct ?.

2. Is the current implementation of the query support all types of queries from the Elastic Search DSL ?.
   I tried to do a simple query that follow this documentation: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-bool-query.html
   However, It doesn't work. I received an error Invalid query: Invalid property 'filter' in boolean query.
   Here is my query:

{
  "query": {
    "bool": {
      "filter": {
        "term": {
          "Content.ContentItem.Published": "true"
        }
      }
    }
  }
}

I wonder if the current implementation support all query syntax. If It doesn't, is there any documentation tell us about the supported query ? or is there any workaround ?.

[dodyg]

Did you try using true instead of "true"? I haven't used the Lucene Query for a long time unfortunately. Nowadays I much prefer the database content index. It is far more predictable to use.

[hung-doan]

thanks @dodyg, unfortunately changing it to true didn't help. I got the same error tho.

{
  "query": {
    "bool": {
      "filter": {
        "term": {
          "Content.ContentItem.Published": true
        }
      }
    }
  }
}

This bellow simple query did work, that mean It only work with limited query template?. I don't know, but I don't want to risk myself with an approach with the un-controllable approach 😞

{
  "query": {
    "term": {
      "Content.ContentItem.Published": "true"
    },
  }
}

any thought on this @Skrypt ?

[hung-doan]

@dodyg good idea about the DB index, is there any guide on this ?. I want to be able to index multiple Content-Type in the same query. It must support Full-Text Search as well.

[Skrypt]

If you want full-text search then I advise you staying with Lucene. Though indexing also is a good idea for quicker SQL transactions. These are 2 different things.

Here, if the issue is that the Lucene Query "filter" is not working properly, then we need to fix it.
But finding specific Content Items of multiple or single Content Types is easily achievable with the ElasticSearch / Lucene Queries. I will, when I have time, produce some Lucene Query examples for most common needs but it is really in the end a learning process of the ElasticSearch Query by looking at the documentation. Of course, some things are not yet working because the ElasticSearch Query language is based on top of the Lucene Java implementation which also has more functionalities than it's .NET Lucene implementation.

"Filter" here should surely work in OC. I'll try to take a look.

[Skrypt]

Ok, found where is the issue (I think).
CreateFilteredQuery() in BooleanQueryProvider class.

[dodyg]

@hung-doan the DB Index is really useful for listing, etc but yeah for full text search, Lucene is the way to go.

These are the tables you can query https://docs.orchardcore.net/en/dev/docs/reference/modules/SQL/. It's all basic sql. Make sure the module is enabled.

[hung-doan]

Ok, found where is the issue (I think).
CreateFilteredQuery() in BooleanQueryProvider class.

@Skrypt @sebastienros do you want me to submit an issue to log it as a bug?.

[Skrypt]

I reported the issue here : https://github.com/OrchardCMS/OrchardCore/pull/2725
This should be fixed as part of that PR.