Syndesis: Create runtime side of Data Buckets POC

[igarashitm]

This issue is to create Data Buckets POC focused on runtime. Put the associated document into Camel Exchange Property with a Document ID as a key, also keeping primary Document as a IN Message body.

Key point is to build a mapping between Syndesis Steps and corresponding DataShapes via Document ID. The idea came from initial discussion was to do that in the Interceptor, for each Step the Inteceptor would capture the output data and push it into Exchange Property with Document ID as a key.

Assuming UI would put corresponding Document ID into datamapper mapping definition ahead, then camel-atlasmap only needs to retrieve it from Exchange Property with Document ID in the mapping as a key.

[chirino]

Added PR #1143 which should allow atlas map to do something like:

Map<String, Message> messages = exchange.getProperty("CAPTURED_OUT_MESSAGES_MAP", Map.class);
Message msg = messages.get("<<Step UUID>>")
String inputData = msg.getBody(String.class);

To get the out documents associated with each step id.

The nice thing about this is that you don't need to add any dependencies to atlasmap to pick this up.

[lburgazzoli]

@igarashitm @chirino I've added WIP label to the PR as I'm almost ready to open a PR for the yaml removal stuffs and I can include the step/processor ID stuff that may be useful outside this specific case for logging and debugging purpose if you want.

[chirino]

@lburgazzoli. ok. My PR for monitoring also depends on Id's being set on the processors. The sooner you get it in the better.

[lburgazzoli]

@chirino setting the processor id has been added to #1144

[igarashitm]

FYI - PR atlasmap/atlasmap#246 for camel-atlasmap bit

[igarashitm]

@chirino w/r/t #1191 - do we want to make a deep copy of the org.apache.camel.Message object in the interceptor, if the "preserve output message" option is specified for the step/connection?

[igarashitm]

As per today's call we don't implement #1191 at first place.

A memo for the future work:
How about holding a java.util.Map<String, Object> as a Data Buckets and push the body instead of Message? The Interceptor then could clone it if the preserve output message option is enabled && the body is Clonable.

[chirino]

Even cloning is not guaranteed to be a deep copy. Best solution would be to serialize/marshal to store the body and then deserialize/unmarshal to get it back. Not all bodys can be serialized. Perhaps we should create explicit steps that can do the marshal/unmarshal.