Magento2: Magento 2.1.0 - "General system exception happened" on Import .csv

[githeat]

Preconditions

1. PHP 7.0.11
2. MySQL 5.6.30
3. Magento 2.1.0 CE
4. Ubuntu 14.04.1 LTS
   

   Steps to reproduce

1. Login to Magento admin panel
2. Select "System" -> "Import"
3. Entity type = "products", Import Behaviour = "Add/Update" - "Stop on error", Allowed errors count = "10", Field Seperator = ",", Multi value seperator ",", Select file to import = myfile, Image files directory = "pub/media/import/"

(Remove quotation marks).

Expected result

1. .CSV should be validated successfully, with 4065 lines (not including headers). Then the products should be imported successfully after clicking "Import"

Actual result

1. After uploading .CSV, I see a red "General system exception happened"
   

My .CSV is encoded (Using Notepad++) in "UTF-8-BOM". I have also tried "UTF-8".

Magento has been updated to 2.1.0.

I have checked:
/var/log/apache2/error.log - No related entries
/var/log/mysql/error.log - No related entries
/var/www/html/magento/var/log/system.log - No related entries
/var/www/html/magento/var/log/debug.log - No related entries

/var/www/html/magento/pub/media/import has 777 permissions to rule out permission issue.
Images inside /var/www/html/magento/pub/media/import have 775 permissions to rule out permissions issue.

In php.ini - upload size is 8mb (.csv is only 4mb).
In php.ini - uploads are enabled

Aside from this, I don't understand the issue. My .CSV is valid because I have been able to import products before, and I can import 1 product at a time. When I try and import 4k+ I get the issue, but it's unfeasable to try and do it by batches of 100.

Thanks for your time

[sevos1984]

Could you please share file that you try to import?

[wclsandipan]

Hi,

We are having the same error. We are using the version Magento 2.1.0 CE.
While we are using the default sample csv.
CSV has been attached below. (in zip format)

Thanks
catalog_product.csv.zip

[sevos1984]

Attached file is working for me, no error appear. Try update to latest magneto version.

[wclsandipan]

Thanks!!

[sevos1984]

@githeat Have you tried updating your code base?

[wclsandipan]

No. It worked with 2.1.0 CE as well, there would be some issue with server configuration I guess, as we have transferred the site to another server, it started working. I need to ask the server administrator about the difference.
I guess you can close ticket as the original author of this ticket is not responding.

[koenner01]

This issue shouldn't be closed as there was no response to @githeat his post.
We are having this issue with MG2.1.4 CE, importing a file containing 4k+ products (data for 2 storeviews) with about 10 columns per row.

There should REALLY be something logged to the logs files because now there is not a single trace of what the problem could be...

[koenner01]

I've done some more digging and the place where the problem lies is in the validateSource function in Magento\ImportExport\Model\Import where the exception in the try catch is not being logged / outputted

public function validateSource(\Magento\ImportExport\Model\Import\AbstractSource $source)
{
    $this->addLogComment(__('Begin data validation'));
    try {
        $adapter = $this->_getEntityAdapter()->setSource($source);
        $errorAggregator = $adapter->validateData();
    } catch (\Exception $e) {
        $errorAggregator = $this->getErrorAggregator();
        $errorAggregator->addError(
            \Magento\ImportExport\Model\Import\Entity\AbstractEntity::ERROR_CODE_SYSTEM_EXCEPTION,
            ProcessingError::ERROR_LEVEL_CRITICAL,
            null,
            null,
            null,
            $e->getMessage()
        );
    }

    $messages = $this->getOperationResultMessages($errorAggregator);
    $this->addLogComment($messages);

    $result = !$errorAggregator->getErrorsCount();
    if ($result) {
        $this->addLogComment(__('Import data validation is complete.'));
    }
    return $result;
}

In our case the Magento\Framework\Stdlib\StringUtils was throwing a warning during strlen function because of an incorrect character

[miguelBarreiro85]

Same issue "General system exception happened", on magento 2.1.5, made a sample file download in the admin zone and tried to import it, just for testing, same error.

Just have logs on debug.log: main.DEBUG cache_invalidate

Does anyone know how to solve this?

[spontaneousdinesh]

I had this and after doing random 1 line imports, I found it was some characters in the descriptions. I had to open the csv in Notepad++ and go Encoding> Convert to UTF-8-BOM then save it as a copy. Went straight on then.

[torchtrust]

I have same issue with customer Main csv import. It will import about half a dozen. Any more than that and if falls over. It's not the actual data as I can then import that data a bit at a time. Once the records are created, then I can update the data, but this error is occuring on more than 6-7 new entries. I want to add 3000, so help!

[webshotNL]

Exactly the same problem on 2.1.7. It works only when you split them in batches of 500 products per import.

[franckgarnier21]

Magento, can you improve you error reporting in order to add the Exception Stack trace and not only catch the exception and return a general error message ?

[shayzalman]

I had the same issue with Hebrew strings and the conversion to UTF-8 solved the issue

[magento-team]

Internal ticket to track issue progress: MAGETWO-81594

[Ctucker9233]

@magento-engcom-team This is still an issue on 2.1.9. Please reopen until this is fixed in 2.1.x.

[ishosys]

Same issue in 2.2.0. importing grouped products.

General system exception happened
Additional data:
Notice: Undefined index: Band-Aid in E:\wamp64\www\sm\vendor\magento\module-grouped-import-export\Model\Import\Product\Type\Grouped.php on line 102 

if i remove associated_skus from CSV, it imports successfully.

I was able to successfully import the same CSV in ver 2.1.9

MISC.xlsx

[Ekkoe]

Same issue, not yet updated to 2.2. Issue occurs for me when trying to import only sku and crosssell_skus.

EDIT: I uploaded everything per 175 products, which did work. The issue MAY have been related to grouped/bundled products, or maybe too long sku names... I'm not sure.

[noindie]

same problem here, tried everything but no solution.
I am running 2.1.9 with mariadb 10 on debian jessie.
Could it be caused by some value already added in database?
my csv is simple 1 product only, UTF-8 no weird char on it

Even applied this in vendor/magento/module-import-export/ https://github.com/magento/magento2/commit/eeb0e6f16713b6a2cc84def34d4ab37b0f681ee9#diff-ea3120705d77399fa31b22b5d099c822

[magento-engcom-team]

Hi @githeat. Thank you for your report.
The issue has been fixed in magento-engcom/magento2ce#1351 by @magento-engcom-team in 2.3-develop branch
Related commit(s):

• 70dd5a3fa17e1ee1cdfaf7f0defd9276989ebbd0
• 54bc3a1dca861e11e46e12f5a15578e0cb7ceb6d
• 3debd3f87e27e23949b81cb2d7e881b5a269608c
• 72c496566bf0d2ff1ced0401514faa5cbaccc0f8
• e60ecdfeabe8b8839cf52fd5ba7fefcbb0b9bdc3
• 8703b3da3dbe28f71c1daeaba08bbe44887f0733
• bd88ed8cc4e1f98b791bd7ca8bed45c29b276ee1
• 1c70b18a339fddf8a51a476a5a251aabef951350
• 5fd7481d2cf8b298369837508a16a23ab4f49850
• 7c615065b36cdafa95d752a891c09dd4ab1a4dfa
• 2653b8ae55625637a6836d305451e292f881bd6a
• 917a03f89f98454cb2ecde524dc8557d7fdcc135
• eca73d0abd1ca6e110c364ef380adab86ab3cc1a
• 26f3109842ce89096a49b9c122af0bae6b672ba2
• c223484a75e38fdcece09d4586342528f06c6383
• 3b90ad71d791ba9cf8e81cc204fd306f41473a18
• ad317c25039de70c1a976e1139c57730899eb114
• 53e721ccc06f1ade05132690715575bd7d47d6b7
• 6114120c0a921b342871448dfb6c47bf34a65cdc
• 63b483925c18304a32441f66b0db8a9c70995c73

The fix will be available with the upcoming 2.3.0 release.

[tanko]

This is still an issue with Magento 2.2.3.
If grouped product with SKU 01.001A is added, it throws an error. SKU with small letter 'a' like 01.001a imports ok.

General system exception happened
Additional data:
Notice: Undefined index: 01.001A in vendor/magento/module-grouped-import-export/Model/Import/Product/Type/Grouped.php on line 102

[seabasst]

Make sure your URL_KEY 's are unique. This error is triggered by duplicate URL keys too.
That's the reason I got the error at least. Hope it helps!

[sahava314]

Having the same issue in 2.2.4

General system exception happened
Additional data:
Notice: Undefined index: 1100GRP in /home/brush/public_html/vendor/magento/module-grouped-import-export/Model/Import/Product/Type/Grouped.php on line 102

productexport_magento_UX_1100GRP.txt

[insanityinside]

Still occurring on 2.2.5. Definitely seems to be related to a capital letter on the sku column, specifically with grouped products. Will investigate further and comment.

[ghost]

@insanityinside, please update steps to reproduce

[insanityinside]

Creating minimal test case now on clean 2.2.5 install, and diving into the code to find out precisely what is occurring. Hang tight.

[insanityinside]

Minimal testcase confirmed, using System->Import, entity type products, import behaviour add/update, stop on error, error count 10, field separator comma, multiple value separator comma, field enclosure ticked or unticked (no difference).

Minimal testcase file CSV attached. Will now test against 2.2-dev.

magento_product_import_combined.txt

[insanityinside]

Confirmed solved in 2.2-dev as of current. Closing ticket.

[R-K-Shrivastaw]

This is still an issue with Magento 2.3.0 Same error throws on product upload.csv

[Misco]

Same to 2.3.1 please have look again. thanks

[alrururu]

Same error on 2.3.3
https://i.imgur.com/fpnAjv4.png

[alrururu]

when I imported 498 item instead, it goes ok
https://i.imgur.com/WxU4oTx.png