Magento2: Images imported via URL have crazy file paths

[hgpit]

Steps to reproduce

1. Install Magento v2.1 EE.

2. Import product CSV using URLs for images (i.e.: base_image: "http://datastore.mydomain.local/imageserver/40354768_big.jpg").

   Expected result

3. File is stored in Magento file system as [magento]/pub/media/catalog/product/4/0/40354768_big.jpg.

   Actual result

4. File is stored in Magento file system as [magento]/pub/media/catalog/product/h/t/httpdatastore.mydomain.localimageserver40354768_big.jpg

When importing hundreds of thousands of products/images you end up with literally hundreds of thousands of files just under one folder: [magento]/pub/media/catalog/product/h/t/. So the file distribution logic of directory hierarchy creation based on the first two characters of the filename is futile/wasted - as the files will _always_ be stored under /h/t/.

We are concerned that this could lead to speed issues accessing the /h/t/ folder. Certainly FTPing to the folder from a remote system could cause the FTP clients to hang (or stall for a long period).

Thank you

[hgpit]

Are we really the only users who find this approach for file-naming a bit... ridiculous? Surely it's not the intended functionality?

[Ctucker9233]

@hgpit You're not the only one. It seems excessive. But it might be because you can run multiple stores/websites on the same magento setup. If you have any product crossover, there may need to be a way for magento to create a unique file name.

[hgpit]

@Ctucker9233 I'm not really sure what you mean, sorry. We do actually run multiple websites and stores. But as every product image is imported via a URL, they're all still filed under [magento]/pub/media/catalog/product/h/t/ regardless of the site/store because of their source path/URL.

We've just invested in Unirgy's uRapidFlow product importer for EE v2.1, and that brings in images (via URLs) "properly", as-in my above expected result. So I still think M2's current folder creations method is wrong/broken.

[Ctucker9233]

@hgpit Ok, I understand now. What I meant is that might be part of Magento's Core structure to create a unique filename for each instance of an image. So if you have the same image on two sites, Magento can tell the difference between the two instances of that image. I'm just speculating this might be the case for the weird file naming.

[andimov]

@hgpit
Thank you for reporting!
datastore.mydomain.local is a weird domain for importing. Try to use local paths.

[hgpit]

@andimov
"Try to use local paths.". What kind of answer/suggestion is that? Magento 2 provides the facility to import from URLs but because I'm reporting an issue with it, you're suggesting not to use the feature?

Also, how is "datastore.mydomain.local" a weird domain for importing? Aside from the fact I have blatantly replaced the domain name with a generic one for the purpose of example; we have an internal server which serves all our image data, so for instance if we have a corporate product SKU of 12345 we can point any browser/system on our network to http://datastore.mydomain.local/imageserver/12345.jpg and it will return the image. There is nothing weird about it.

[andimov]

@hgpit
Try to use one of iana list of tlds for importing from URLs.
Please, let me know when your issue will be solved.

[hgpit]

@andimov
I can't (and would not) change my entire enterprise's internal domain naming conventions for Magento.

[shiftedreality]

Hi @hgpit

Thank you for reporting.
We've created internal ticket MAGETWO-58217 to resolve this issue

[magicsss]

Hi there.
Does anybody find solution for this issue?

[qx54]

We ran into the same issue. The Magento import should definitely not include the path of the external URL in the filename. Bad for SEO beside other reasons.

[joachimVT]

I can confirm this, having the same issue in version 2.1.8

pub/media/catalog/product/cache/image/e9c3970ab036de70892d86c6d221abfe/h/t/httpwww.vinesse.bemediacatalogproductpapapeclement2012.png

[magento-engcom-team]

@hgpit, thank you for your report.
We've created internal ticket(s) MAGETWO-58217 to track progress on the issue.

[piotrekkaminski]

This issue was moved to magento-engcom/import-export-improvements#57

[PieterCappelle]

Should be fixed. If accepted I'll create backport to 2.2 & 2.1.

[magento-team]

Hi @hgpit. Thank you for your report.
The issue has been fixed in magento/magento2#12872 by @PieterCappelle in 2.3-develop branch
Related commit(s):

• 8cdf04390e43184aef85a26855c40a9c0a33309e
• 53292d37074044e644eacdef9adfa79df5199662

The fix will be available with the upcoming patch release.