Shopsys: initial command for products export into elasticsearch doesn't release memory usage during run

[boris-brtan]

What is happening

if we do an initial export of products into elasticsearch the command consumes big amount of memory
php -dmemory_limit=-1 ./project-base/bin/console shopsys:product-search:export-products
php phing product-search-export-products
50 000 items -> 2GB
200 000 items -> 8GB

so the command fails on memory_limit which is set by default to 512MB.

Expected result

maybe we can do some fix based on this article ??? https://www.conradk.com/codebase/2018/04/22/memory-management-in-long-lived-symfony-commands/

[boris-brtan]

i am not sure but by implementation gc_collector it could cause slowdown of this command.

[henzigo]

@boris-brtan If you will call gc_collector every 1000 exported items then speed is ok

[grossmannmartin]

manually called garbage collection doesn't help me at all.
But if you disable sqlLogger and clear entity manager after each batch, the memory consumption is stable (no memory leak)