Sylius: Data import and export

[pjedrzejewski]

[umpirsky]

Interested.

[jjanvier]

Import from ? Export to ?

[stloyd]

@jjanvier I.e. to CSV/JSON, check any PIM (i.e. www.akeneo.com).

[amenophis]

Maybe this can be a good start : http://www.theodo.fr/blog/2013/10/how-to-import-export-data-with-an-etl-command-using-symfony2-in-tdd/

[umpirsky]

Interesting lib https://github.com/ddeboer/data-import

[winzou]

https://github.com/orocrm/platform/tree/master/src/Oro/Bundle/ImportExportBundle seems very interesting but these guys don't release their bundles as standalone composer packages... weird.

@umpirsky you're lib seems interesting as well, but we'll need to integrate it in a bundle, it has already been done in the Oro platform... But anyway, it we cannot get oro standalone bundle, let's go with it! What do you think?

[umpirsky]

@winzou Not sure. We must think twice before starting this. I hope oro will release their bundle. But still not sure if it's the best option.

[winzou]

What do you mean by best option? The fact of using their bundle or the fact of waiting them for releasing it in standalone?

[umpirsky]

@winzou Using their bundle.

[pjedrzejewski]

I've been looking at it as well and it seems quite nice, I'd really love to use it and avoid redoing it on our side. Only thing that I did not check, do we need to use their FlexibleEntityBundle with ImportExport? (this kinda complicates things)

[winzou]

Seems like the only dependency is their BatchBundle

[pjedrzejewski]

This BatchBundle is interesting as well. I'll get in touch with Oro guys to see when they plan to release the bundles standalone. /cc @orocrm

[winzou]

Any news @pjedrzejewski ? Otherwise we could go with the lib @umpirsky mentioned, it has actually already a bundle: https://github.com/ddeboer/DdeboerDataImportBundle

[nidup]

Hi guys,

FYI, the BatchBundle is now distributed as standalone package https://github.com/akeneo/BatchBundle, we maintain this bundle and we continue to add new features in context of Akeneo PIM, the contribution is welcomed :)

For the PIM we also developed some bundles, BaseConnectorBundle, TransformBundle, but they are part of the application and more or less strongly coupled with our CatalogBundle and our business rules. More details on the connector concept http://docs.akeneo.com/latest/index.html

About https://github.com/ddeboer/DdeboerDataImportBundle we used it by the past, imho, it's a simple and interesting bundle to ease a first implementation.

Anyway, we'll be glad to discuss and share on this topic. /cc @BitOne

[winzou]

Hi @nidup
Thanks for your feedback!
Do you have any roadmap regarding the release of ImportExportBundle as standalone?
And what was your limitation with DdeboerDataImportBundle?

[nidup]

Hi @winzou

As import / export is one of the main concern of the PIM, we defined many bundles to deal with,
ImportExportBundle : contains UI (grid, form) to configure an import / export, execute it and see the result
BaseConnectorBundle : contains basic Readers, Processors, Writers and the configuration of our jobs (as a product import)
TransformBundle : normalize / denormalize PIM objects (product, category, attribute)
BatchBundle : a generic batch architecture bundle inspired by Spring Batch

It's highly related to the PIM, imho, the only parts that could be extracted (for now) are basic readers / writers from the BaseConnectorBundle (quite close to those provided by DdeboerDataImportBundle).

Perhaps, you're thinking to the OroPlatform ImportExportBundle ?

[winzou]

Arf yes sorry, I was indeed thinking of the one from OroPlatform. The identical names are not helpful ;)
Why did you rewrite some readers/writers in BaseConnectorBundle as they already exist in OroPlatform/ImportExportBundle that you're using?

[BitOne]

Hello @winzou,

As the ImportExportBundle in Akeneo existed well before the creation of the one in OroPlatform, I would say "why they didn't use our Readers and writers ?" ;)

Moreover Oro needs are pretty differents. Import/Export on a CRM is quite a peripheral feature whereas in a PIM is a central feature. So our needs are more complex and deeper than what is covered by the ImportExport bundle, and recreating our ImportExport to use their would not have really brought any added value to Akeneo.

[winzou]

Fair enough, I didn't know about the chronology. Just wanted to make sure that Oro ImportExportBundle wasn't weak, leading you to rewrite some parts of it.
Thanks for you feedback.

[Richtermeister]

Sorry for hijacking this comment, but I just wanted to say I'm super excited about both Oro and Akeneo. Reading your code is mind-opening on so many levels, and that you show up for discussions such as this makes me feel all warm and fuzzy! Very cool!

/fanboy :blush:

[BitOne]

Thanks @Richtermeister for the kind words ! I can tell you we learned a lot too by looking at the Sylius code ;)

[mojzis]

any thoughts about which way sylius will go ? https://github.com/ddeboer/data-import or akeneo ?

[BitOne]

It's definitely two absolutely different things (Akeneo vs ddeboer/data-import). Akeneo is a full fledge standalone application with import/export facilities and UI to enrich, translate products information, with a big emphasize on data quality checking.

ddeboer/data-import is more about reading and converting data format. In fact ddeboer/data-import could be used in Akeneo connectors.

Another thing we were talking about is the akeneo/BatchBundle (https://github.com/akeneo/BatchBundle). This provides import and export infrastructure with monitoring. It has concepts close to ETL and could be used with ddeboer/data-import as well, as it's more on the framework side (it defines the import/export execution flow whereas ddeboer/data-import is more about concrete format manipulation).

If you don't need the full Akeneo application, a interesting solution would BatchBundle+ddeboer/data-import to manage your import and export.

[michalmarcinkowski]

In this PR https://github.com/Sylius/Sylius/pull/3156 you will find the initial version of data import and export for Sylius.

[olegsv]

Are there any news regarding the integration of batch import functionality into Sylius?

[Zales0123]

As far as I know there are no plans for now to implement such a functionality to _Sylius_, at least until release of stable version. Of course as we already have possibility to create plugins for Sylius there is a high possibility that plugin with such a functionality would be created ;)

[bitbager]

Is this still in the backlog or maybe has anybody implemented it and would like to share it with OpenSource? 😢

[czende]

This would be great! Any update on this?

[kochen]

I have a small Order Export solution I plan to release as a plugin as soon as time allows.

[bitbager]

Perfect. Has anyone thought about configurable product information export?

[czende]

@bitbager I'm working on import and export commands, but its really complicated, because of multiple relations and possibilities. For now I have just structure for exporting simple products and their relations (taxons, attributes, shippings, images paths etc) into simple CSV file.

[diimpp]

https://github.com/FriendsOfSylius/SyliusImportExportPlugin

There is this new cool plugin, which is based on portphp/portphp (Previously known as as ddeboer/data-import)

[lchrusciel]

The issue can be closed then, can't it?