Syndesis: [RFC] About Datashapes

[paoloantinori]

This is a...

[x] Feature request
[ ] Regression (a behavior that used to work and stopped working in a new release)
[ ] Bug report  
[ ] Documentation issue or request

Description

An interesting conversation has started on Google Chat about the limits of being force to define a datashape to use AtlasMap.

I'd like people to continue the conversation here.

[lburgazzoli]

@paoloantinori can you add the "meeting notes" here ? :)

[paoloantinori]

Zoran Regvart,

Atlasmap supports constants, the steps preceeding/following it need to define a shape in order for it to be useful

Nicola Ferraro,

"* in order for it to be used".. There are many use cases where you want to ignore the content of the exchange and just set a constant value in the body (e.g. webhooks, timer). That constraint of requiring a datashape in previous (for datamapper) and next (for template) step is a bit artificial to me..

Andrea Tarocchi,

+1

Zoran Regvart,

The webhook has a datashape for the request, it's probably missing a data shape for the response, if we were to allow insertion of Atlasmap step regardless of data shapes what would the mapper map then? IMHO Syndesis is not a be-all-end-all solution, if you're doing integrations without knowing the shapes of the data it's probably not well suited for that use case...

[christophd]

That constraint of requiring a datashape in previous (for datamapper) ...

I think this constraint has been removed already, see #4207

[mmuzikar]

Relates to #4207 and I believe many other issues, for me it makes sense even solely for the constants can provide. Requiring user to first use a step with datashape defined above and then using datamapper is quite cumbersome when creating integrations and also seems like the user is doing some nasty hack in order to get it working.

I am not familiar with the properties section it contains but it also seems like that could be useful even without datashapes... and with the introduction of conditional mapping I believe now is the right time to allow users to use atlasmap without previous definition of datashape.

[christophd]

I believe now is the right time to allow users to use atlasmap without previous definition of datashape

@mmuzikar as #4207 has been solved isn't that already possible as of today?

[mmuzikar]

@christophd ah cool didn't notice that because it was quite recent. Big thanks for that, that can really improve the workflow.

[lburgazzoli]

Partially related but there were some discussion in the past to have the data mapper capabilities (or a lightweight variant) embedded into steps

[christophd]

So if I get it right data mapper only requires an input shape of the subsequent step as target and this kinda makes sense to me as you want to map something from A to B. And if B is not specified you can not map anything.

I guess we need some other enrich step that just sets/adjusts the body/header without any mapping (so no atlasmap involved in this enrich step).

[mmuzikar]

@lburgazzoli By what I can imagine for example specifying the SQL query like SELECT * FROM contact WHERE name = $constants.name or name = $body.firstname $body.lastname?

That would be really cool IMHO. If there is a github issue or any other conversation regarding this topic could you attach some links to follow up? :)

[lburgazzoli]

@mmuzikar the camel sql component already support using the simple language, then I do not know if it is supported by the connector but that's another topic

[christophd]

I also like the idea to bind/embed data mapping to the step itself because we would avoid users destroying the mapping by adding more steps between data mapper and subsequent step.

By adding/removing steps around data mapper you often destroy the mapping as source and target changes accordingly. You end up redoing the mapping from scratch.

[lburgazzoli]

I think that a step - at least from an UX POV - should always have two additional capabilities:

• mapper
• logger

[nicolaferraro]

Happy to see the issue with datamapper and template have been fixed. Not able to use the react staging environment at the moment, but I wonder if also this issue has been fixed:

[mmuzikar]

@nicolaferraro No that issue still remains... I also think I'd be a valid use case

[zregvart]

cc @gaughan

[gaughan]

I really like the idea of adding mapping and logging to the step.
It looks like #4207 might need to be reopened based on zorans comment.

[dongniwang]

Thanks for the notify/uxd label. I'd love to understand more of the discussion here from a UX point of view. Would it be possible to have a quick meeting on this? Hoping to understand the requirements, the scope of work, and how we can get it done.

[stale[bot]]

This issue has been automatically marked as stale because it has not had any activity since 90 days. It will be closed if no further activity occurs within 7 days. Thank you for your contributions!