Syndesis: [conditional-flows] UI data shape handling of CBR step

[christophd]

• A CBR step can be inserted at any point in an existing flow. When added the step adopts the output data shape of the previous step and this is the fixed starting shape of all conditional flows added (= first step in conditional flow provides this shape as fixed output shape).
• By default the CBR step has no output shape defined so the conditional flows can produce any output and the primary flow is not able to access the shapes created within the conditional flow.
• Changes to the input/output data shapes of the CBR step must be propagated to all sub-flows

[christophd]

Could be also done with #5181

[lburgazzoli]

I think that a CBR step could be handled in a way similar to the "shapeless connectors" so the user is prompted to provide a shape when the CBR step is added to the flow.

In case a step already exists after the CBR one, the UI should offers also an option to set the output shape to the one of the subsequent step.

[lburgazzoli]

We eventually need to propagate warning on a flow up to the source so i.e. if the data shape of a CBR is changed, all the flow could potentially need to be updated thus the CBR step should report a warning till all the flows have been fixed.

[christophd]

I have to move the user defined data shape handling to a separate issue. Most likely this particular feature will not make it to 7.4.

[christophd]

@gashcrumb @mcada FYI this seems to be broken with react UI. The conditional flows step adopts the shape successfully but the shape is not set as fixed output shape to the 1st step of each conditional flow.

[gashcrumb]

Yep, you're right it's totally not doing that at the moment. I'll get that implemented tomorrow.

[mastepan]

@gashcrumb please change pipeline to Done if the issue's been fixed. Ignore this otherwise. Thanks.

[christophd]

@gashcrumb has this fix https://github.com/syndesisio/syndesis-react/pull/470 been backported to 1.7.x already? I still see the error on 1.7.x branch

[gashcrumb]

Hm, it should be that was done before we branched for release. I'll check
in a few, what integration did you test with?

On Tue, Jun 25, 2019, 5:31 AM Christoph Deppisch [e-mail]
wrote:

@gashcrumb https://github.com/gashcrumb has this fix
syndesisio/syndesis-react#470
https://github.com/syndesisio/syndesis-react/pull/470 been backported
to 1.7.x already? I still see the error on 1.7.x branch

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/syndesisio/syndesis/issues/5215?email_source=notifications&email_token=AACV3LB3HO2NAI7MFPGE63TP4HQVNA5CNFSM4HF5L46KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYPUJHY#issuecomment-505365663,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AACV3LDL4GGMXY6O5XH4GK3P4HQVNANCNFSM4HF5L46A
.

[christophd]

I wrote a new integration SQL => Split => Conditional Flow => Log The sub flow should be provided with the single element SQL shape as start connection

[gashcrumb]

This is where it's done on master at least => https://github.com/syndesisio/syndesis/blob/master/app/ui-react/packages/api/src/helpers/integrationFunctions.ts#L702-L709
lemme check the 1.7.x branch

[gashcrumb]

Looks the same there too. K, will investigate further.

[christophd]

I will also double check on my side

[gashcrumb]

Ah, I think I want to be passing the input data shape on this line, silly me -> https://github.com/syndesisio/syndesis/blob/master/app/ui-react/packages/api/src/helpers/integrationFunctions.ts#L705

I don't know why I can't keep these straight, I'm going to blame the attribute names :laughing:

[mcada]

Data shape handling is not working correctly.
First there is this issue https://github.com/syndesisio/syndesis/issues/6158

Second issue - I tried to create a condition with some output data shape, lets say create_lead_return from SQL stored procedure create_lead. It was propagated correctly to the primary flow. But editing the condition - deleting the create_lead step and adding general select * from contact did not change output data shape in the primary flow.

[heiko-braun]

@mcada Could it be an issue with the SQL connector, rather than the CBR step?

[heiko-braun]

@riccardo-forina @christophd This one seems to be related to #6158

[mastepan]

@heiko-braun P0 and Backlog? Is that right? Just checking

[heiko-braun]

What do you mean?

[mastepan]

What do you mean?

I'm not familiar with exact workflow but I've noticed that this issue has P0 label and it's been moved to Backlog state that sounds strange to me if it's urgent. So I just checked if there happened any error :)

[heiko-braun]

@mastepan No, backlog means it should be done now.

[christophd]

@mcada what is the difference in those two issues you have mentioned? Is it really related to conditional flows or is it a general problem that changing the data shape is not propagated to the next steps?

[mcada]

@christophd it may be the same issue but this one is about handling data shape in CBR and it does not work so I mentioned some scenarios which did prove that it does not work so this issue should not be done/closed as it is not working - you mentioned in the first comment: Changes to the input/output data shapes of the CBR step must be propagated to all sub-flows

[christophd]

@mcada thanks for the explanation. As the PR #6167 got merged today and this fixes #6158 can you please verify if this issue here is also fixed with it?

[christophd]

The PR also got backported to 1.7.x

[mcada]

@christophd I will check after we receive new 1.7.x tag

[heiko-braun]

@mcada The fix is on 1.7.x can you use this for a preflight check?

[mcada]

@heiko-braun I do not see 1.7.x image tag updated on docker-hub. Does it mean that I have to build the image from current 1.7.x syndesis branch?

[heiko-braun]

@christophd Can you help @mcada to verify from the 1.7.x branch?

[christophd]

@mcada let me push latest 1.7.x to https://christophd-test.b6ff.rh-idev.openshiftapps.com/ you can use this then to verify. I will let you know once the 1.7.x with the fixes is available there.

[christophd]

@mcada so the rh-idev project I have mentioned above is now using the latest 1.7.x and you can use that to verify

[heiko-braun]

Thanks @christophd.

@mcada Can you leave a remark here once you done the a quick check?

[mcada]

Data shape change is not propagated to conditional flow step.

[christophd]

my first findings is that this is a refresh issue when the shape changes. @gashcrumb is also having a look now

[christophd]

seems it is only in combination with split step