Elasticpress: Woocommerce orders indexing fails with version 2.2

[fidjius]

Hello,
I installed version 2.2 with "Protected content" activated in order to index woocommerce orders (order numbers and invoice numbers).
Syncing always stops at 13650/129310. The web server always needs to be restarted since it seems to be lost in an infinite loop.
After restart, the plugin shows "Sync paused 13650/129310". Trying to resume indexing does nothing. The web server needs to be restarted again.
I tried with different server capacity up to M4.10xlarge on AWS just to see if it is a server issue but the issue is always the same.
I also tried different Elasticsearch server capacities on qbox.io but it is always the same.
Indexing always stops at 13650.
Thank you for your help.
Regards

[ivankristianto]

@fidjius Can you copy and paste here the error log?
you can get it by running the indexing with the wpcli:
wp elasticpress index --debug

[fidjius]

It worked but indexing was made in an index named "https-1" instead of "domainname-1" which seems to be used by the plugin inside wordpress. Is there a way to use the command "wp elasticpress index --debug" with a setting defining the index name to be used?

[ivankristianto]

Can you paste what is the output of your elasticpress stats?
Run this command: wp elasticpress stats

To run index in specific sites (if you have multisite), you can pass the --url, sample:
wp elasticpress index --url=http://yourdomain.com

[fidjius]

1. It worked with the right index name. I executed this command from scratch with a brand new elasticsearch server. The result is that the index is fully created with the right name. However wordpress displays none of woocommerce orders in the dashboard.

2. I tried to reindex from wordpress dashboard. I saw that the same index is emptied first (so I am now sure that case 1 is using the right index name), and then docs are starting to be indexed. In this case, woocommerce orders are starting to be displayed well from dashboard. In this case also, I am still meeting the issue of indexing stopping at 14,350 and of server freezing before the end of indexing.

The question is : what is wrong in the case 1 that makes that woocommerce orders are not displayed at all in the wp dashboard ? Case 1 seems to be the only way to index everything with our large number of posts and orders but there is still something to find in the command "wp elasticpress index --url=yourdomain.com" so that orders are displayed correctly in the wp dashboard.

[fidjius]

Debug log is the following :

Debug (bootstrap): No readable global config found (0.009s)
Debug (bootstrap): No project config found (0.01s)
Debug (bootstrap): No package autoload found to load. (0.125s)
Debug (bootstrap): ABSPATH defined: /opt/bitnami/apps/wordpress/htdocs/ (0.125s)
Debug (bootstrap): Set URL: domainname.com (0.125s)  // -> I hide the domain name
Debug (bootstrap): Begin WordPress load (0.125s)
Debug (bootstrap): wp-config.php path: /opt/bitnami/apps/wordpress/htdocs/wp-config.php (0.126s)
Debug (bootstrap): Loaded WordPress (1.449s)
Debug (bootstrap): Running command: elasticpress index (1.449s)
Indexing posts...
Processed 350/129649 entries. . .

(...)

Processed 129649/129649 entries. . .
Number of posts indexed on site 1: 129649
Total time elapsed: 1,173.139
Success: Done!

[fidjius]

One precision :

Case 1 of my previous message : it was done with an Elastic cluster version 2.4.2

I just made another test with a new cluster with Elastic version 5.1.1 and this time all woocommerce orders are displayed correctly in the wp dashboard.

The only weird thing is that total of items in woocommerce orders page is wrong. It displays 170,271 items instead of 9,842 and each time I click on next page, the total of items is decreased by around 500 items and so on. If i come back to page 1, the total displays 170,271 items again.

[ivankristianto]

@fidjius I'm investigating this issue. and try to replicate your environment.
Can you confirm your environment?

WordPress Version:
ElasticPress Version:
Elasticsearch Version:
Woocommerce Version:

[fidjius]

WordPress Version: 4.7.2
ElasticPress Version: 2.2
Elasticsearch Version: 5.1.1 on Qbox.io
Woocommerce Version: 2.6.14

[ivankristianto]

Yes this issue is confirmed with Elasticsearch version 5.1.1.
I got this error when open the orders from the admin area:

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [post_date] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory."
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": "10updev-1",
        "node": "wFx2ipJERA2F6JIvlq8-Zg",
        "reason": {
          "type": "illegal_argument_exception",
          "reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [post_date] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory."
        }
      }
    ],
    "caused_by": {
      "type": "illegal_argument_exception",
      "reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [post_date] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory."
    }
  },
  "status": 400
}

Explanation:

Fielddata can consume a lot of heap space, especially when loading high cardinality text fields. Once fielddata has been loaded into the heap, it remains there for the lifetime of the segment. Also, loading fielddata is an expensive process which can cause users to experience latency hits. This is why fielddata is disabled by default.
Source: https://www.elastic.co/guide/en/elasticsearch/reference/5.0/fielddata.html#_fielddata_is_disabled_on_literal_text_literal_fields_by_default

But this issue doesn't happen in Elasticsearch 5.2.0.

@fidjius can you try this on ES 5.2.0 ?

[fidjius]

Hi,
I just created a new cluster with ES 5.2.0 on cloud.elastic.co. After reindexing, I got exactly the same issue. The number of items in the woocommerce orders page is still wrong.

By the way, I am using Elasticpress mainly to rapidly find orders by customer name, customer email or order number.

I see that the field meta.order_time_order_number is the field that stores the order numbers, but it is indexed as an array. For example :
{ "value": "12345-999-1234", "raw": "12345-999-1234", "boolean": false, "date": "1971-01-01", "datetime": "1971-01-01 00:00:01", "time": "00:00:01" }

That makes that there is no possible search from woocommerce dashboard based on order number, because search in arrays are not possible.

Is that possible to index the value meta.order_time_order_number as text and not as an array ? The value "12345-999-1234" should be indexed only, not the other datas.

[tlovett1]

@fidjius would you mind emailing me at taylor.lovett [at] 10up.com? I'd love to help you take a closer look at this one.

[lkraav]

Hey all. Am I still supposed to run into this error on ES 5.6.3 + EP 2.4?

EDIT wp elasticpress index --post_type=shop_order --setup apparently got things working.

[fidjius]

Hello,

Sorry, I don't use ElasticPress anymore.

Bien cordialement / Best regards / freundliche Grüße / Mat frëndlechen
Gréiss

Hervé Lancelin
7 rue Michel Rodange
L-2430 Luxembourg
T ([phone]

2017-11-07 15:27 GMT+01:00 Leho Kraav [e-mail]:

Hey all. Am I still supposed to run into this error on ES 5.6.3 + EP 2.4?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/10up/ElasticPress/issues/747#issuecomment-342498246,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAUaqQPU8WnnGdMvChYWK3tDZM8VE1Isks5s0GjJgaJpZM4MP88W
.