Magento2: magento ce 2.1.6 catalog_category_product_index is not build right way (complete investigation available)

[tsifra]

This bug investigation started in #9656

Here are results. Similar products of category are imported the same way during one import. But they get different rows in catalog_category_product_index table.

here are rows from catalog_category_product

entity_id	category_id	product_id	position
5535	101	1948	1
5549	101	1962	1

Here are rows from catalog_category_product_index while the category is still "healthy". Please note that the first product of the category has different rowset.

category_id	product_id	position	is_parent	store_id	visibility
2	1948	1	1	1	4
56	1948	10001	0	1	4
101	1948	1	1	1	4
2	1962	1	1	1	4
6	1962	10001	0	1	4
56	1962	10001	0	1	4
101	1962	1	1	1	4

Finnaly when i continue to import other products to other categories something happens with indexer so "old" products become unavailable in category with id 101 but available in parent category. Exactly as described in #9656

category_id	product_id	position	is_parent	store_id	visibility
2	1948	1	1	1	4
56	1948	10001	0	1	4
101	1948	1	1	1	4
2	1962	1	1	1	4
56	1962	10001	0	1	4

[s00071609]

You might want to run php bin/magento catalog:images:resize command. The store i was working on was showing broken image after installing magento 2.1.6. I rant this command and it seems to have fixed issues. But i am running compile and deploy at the moment. If you want you can check this multistore setting- if you find the image looks fine in these stores when you check run the command, else assume that it din't fix. Not sure how to apply the new patch that has been release? Hopefully someone will soon provide some instruction, how it works in shared hosting.
https://jostechecig.com.au/
https://jostechaustralia.com.au/

[tsifra]

This issue not about images.

[s00071609]

is this issue with product import only or all upgrades from previous versions to this version?

[tsifra]

This issue not about product import. Please check the first message. It's about category-product indexer. Import/product qty in catalog is only trigger and indicator for this situation. I've noticed this problem only after 2000 products.

[s00071609]

Ok hope someone provide you the answer soon. Did you use the patch CE-MAGETWO-67805.patch?

How do you install this patch? I uploaded it to public_hmtl and ran bash patch file name, but it does not seem to work. Is there any other command that would work its shared hosting environment

[tsifra]

I'm not affected by this problem.

[sunilkumar707]

@tsifra Did you get solution? I'm facing same issue

[tsifra]

No, there is no solution. Flat catalog option = true doen't make sence. So have to wait for core team.

[r0m30n]

My solution, that works for me:

1. Open "vendor/magento/module-catalog/Model/Indexer/Category/Product/AbstractAction.php"
   2) Search for function "isRangingNeeded"
   3) Change Code from "return true;" to "return false;"

After reindexing all products are displayed in the category.

[kilis]

Same problem. isRangingNeeded with return false doesn't help.

[rahimalizada]

Same problem on mass importing products

[magento-engcom-team]

Hi @tsifra
Could you please provide me dump and import file from #9656 for testing.
Thanks

[magento-engcom-team]

@tsifra, we are closing this issue due to inactivity. If you'd like to update it, please reopen the issue.

[rahimalizada]

The problem persists on 2.1.10

[basilikonet]

Any update about it? We have this problem in 2.1.10

[kervin]

Any updates on this anyone?

[portchris]

I was having a similar problem where not all products were showing in categories. The temporary fix was to set indexers: Category Products & Product Categories to Update on Save.

For the long term solution I found @r0m30n suggestion above worked for me in this case on Magento 2.1.6.

I've created a patch for those that may also be in my situation:

composer.json

......
"require": {
        ......
        "cweagans/composer-patches": "^1.6",
        ......
},
......
"extra": {
        "magento-force": "override",
        "patches": {
            "magento/module-catalog": {
                "GITHUB-9676 - Remove Indexer Batch Ranging To Index Full Category": "patches/GITHUB-9676.patch"
           }
     }
}

patches/GITHUB-9676.patch

--- a/vendor/magento/module-catalog/Model/Indexer/Category/Product/AbstractAction.php
+++ b/vendor/magento/module-catalog/Model/Indexer/Category/Product/AbstractAction.php
@@ -298,7 +298,7 @@
      */
     protected function isRangingNeeded()
     {
-        return true;
+        return false;
     }

     /**

Finally, run:

rm -rf vendor/
composer install
composer update --lock
php bin/magento indexer:reset
php bin/magento indexer:reindex
php bin/magento cache:flush