Syndesis: Salesforce connector: Missing option value for Id or sObjectIdName exception

[dsimansk]

This is a...

[ ] Feature request
[x] Regression (a behavior that used to work and stopped working in a new release)
[ ] Bug report  
[ ] Documentation issue or request

The problem

Running Periodic DB -> Datamapper -> Salesforce Create or update produces the following error.

db-sf-export.zip

 Missing option value for Id or sObjectIdName

2018-04-12 13:41:53.489 ERROR 1 --- [r://integration] o.a.camel.processor.DefaultErrorHandler  : Failed delivery for (MessageId: i-L9u4pA9_SfSdQKrTZ_fz on ExchangeId: i-L9u4p1__SfSdQKrTZ_cz). Exhausted after delivery attempt: 1 caught: org.apache.camel.component.salesforce.api.SalesforceException: Missing option value for Id or sObjectIdName
--
  |  
  | Message History
  | ---------------------------------------------------------------------------------------------------------------------------------------
  | RouteId              ProcessorId          Processor                                                                        Elapsed (ms)
  | [i-L9u4RoLxQYNXsa8Q] [i-L9u4RoLxQYNXsa8Q] [timer://integration?period=60000                                              ] [       740]
  | [i-L9u4RoLxQYNXsa8Q] [-L9u4BfOaKGOj3REgD] [Processor@0x7797f462                                                          ] [         0]
  | [i-L9u4RoLxQYNXsa8Q] [process2          ] [Processor@0x235502f8                                                          ] [         2]
  | [i-L9u4RoLxQYNXsa8Q] [process3          ] [Processor@0x7296d3cc                                                          ] [         0]
  | [i-L9u4RoLxQYNXsa8Q] [-L9u4JHUaKGOj3REgD] [atlas:mapping-step-2.json?sourceMapName=Syndesis.CAPTURED_OUT_MESSAGES_MAP    ] [       544]
  | [i-L9u4RoLxQYNXsa8Q] [-L9u4JHUaKGOj3REgD] [Processor@0x71935d8d                                                          ] [         0]
  | [i-L9u4RoLxQYNXsa8Q] [process4          ] [Processor@0x235502f8                                                          ] [         0]
  | [i-L9u4RoLxQYNXsa8Q] [process5          ] [Processor@0x396f9339                                                          ] [         0]
  | [i-L9u4RoLxQYNXsa8Q] [-L9u4EBjaKGOj3REgD] [salesforce-3                                                                  ] [         1]
  |  
  | Stacktrace
  | ---------------------------------------------------------------------------------------------------------------------------------------
  |  
  | org.apache.camel.component.salesforce.api.SalesforceException: Missing option value for Id or sObjectIdName
  | at io.syndesis.connector.salesforce.customizer.ForUpdateCustomizer.beforeProducer(ForUpdateCustomizer.java:57) ~[connector-salesforce-1.3-SNAPSHOT.jar!/:1.3-SNAPSHOT]
  | at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61) ~[camel-core-2.21.0.jar!/:2.21.0]
  | at org.apache.camel.processor.Pipeline.process(Pipeline.java:138) [camel-core-2.21.0.jar!/:2.21.0]
  | at org.apache.camel.processor.Pipeline.process(Pipeline.java:101) [camel-core-2.21.0.jar!/:2.21.0]
  | at org.apache.camel.processor.Pipeline.process(Pipeline.java:138) [camel-core-2.21.0.jar!/:2.21.0]
  | at org.apache.camel.processor.Pipeline.process(Pipeline.java:101) [camel-core-2.21.0.jar!/:2.21.0]
  | at io.syndesis.integration.component.proxy.ComponentProxyProducer.process(ComponentProxyProducer.java:44) ~[integration-component-proxy-1.3-SNAPSHOT.jar!/:1.3-SNAPSHOT]
  | at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148) ~[camel-core-2.21.0.jar!/:2.21.0]
  | at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548) [camel-core-2.21.0.jar!/:2.21.0]
  | at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) [camel-core-2.21.0.jar!/:2.21.0]
  | at org.apache.camel.processor.Pipeline.process(Pipeline.java:138) [camel-core-2.21.0.jar!/:2.21.0]
  | at org.apache.camel.processor.Pipeline.process(Pipeline.java:101) [camel-core-2.21.0.jar!/:2.21.0]
  | at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548) [camel-core-2.21.0.jar!/:2.21.0]
  | at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) [camel-core-2.21.0.jar!/:2.21.0]
  | at org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:711) [camel-core-2.21.0.jar!/:2.21.0]
  | at org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:634) [camel-core-2.21.0.jar!/:2.21.0]
  | at org.apache.camel.processor.MulticastProcessor.process(MulticastProcessor.java:248) [camel-core-2.21.0.jar!/:2.21.0]
  | at org.apache.camel.processor.Splitter.process(Splitter.java:114) [camel-core-2.21.0.jar!/:2.21.0]
  | at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548) [camel-core-2.21.0.jar!/:2.21.0]
  | at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) [camel-core-2.21.0.jar!/:2.21.0]
  | at org.apache.camel.processor.Pipeline.process(Pipeline.java:138) [camel-core-2.21.0.jar!/:2.21.0]
  | at org.apache.camel.processor.Pipeline.process(Pipeline.java:101) [camel-core-2.21.0.jar!/:2.21.0]
  | at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) [camel-core-2.21.0.jar!/:2.21.0]
  | at org.apache.camel.component.timer.TimerConsumer.sendTimerExchange(TimerConsumer.java:197) [camel-core-2.21.0.jar!/:2.21.0]
  | at org.apache.camel.component.timer.TimerConsumer$1.run(TimerConsumer.java:79) [camel-core-2.21.0.jar!/:2.21.0]
  | at java.util.TimerThread.mainLoop(Timer.java:555) [na:1.8.0_151]
  | at java.util.TimerThread.run(Timer.java:505) [na:1.8.0_151]

Expected behavior

Integration is deployed running without errors.

Tasks involved / Steps to Reproduce

1. Periodic SELECT * FROM CONTACT
2. Salesforce - Create or update record -> Contact with custom field
3. Datamapper - all fields mapped
4. Profit

Cc @zregvart

[zregvart]

@dsimansk I know the exception text is somewhat confusing, it's actually complaining that the value is missing for the ID property. Could it be NULL in the database for the row you're considering?

Could you add a log step before Salesforce to confirm?

[dsimansk]

@zregvart narrowed ti down to custom sObject that was set to null. If I create integration with Create or update record that, I'm required to select identifier field that in my case for TwitterScreenName but I didn't create any mapping to the field.
In general the answer is like: "Identifier field must be set/mapped"?

[zregvart]

@dsimansk yes, for the _Create or update_ action we absolutely need the value for the identifier, otherwise the API call to Salesforce cannot be made.

We might have to take a look at enhancing the data shape/schema generation and the UI of the mapper to notify users of required properties (cc @igarashitm, @pleacu).

[igarashitm]

Yep, handling required field is on the table - atlasmap/atlasmap#90 atlasmap/atlasmap#50

[dsimansk]

@zregvart thanks for bearing with me :)

@igarashitm greate news:). Should we keep this issue opened in Syndesis repo for tracking purposes?

[gashcrumb]

Wonder if there should be some mention on the page where you select the ID field that it needs to have data mapped to it, it's probably not that obvious currently why this is needed.

[igarashitm]

@dsimansk Yep, let's keep this open - we'll at least make sure that works in Syndesis, and possibly need some cosmetic changes in the connector schemas.

[zregvart]

@dsimansk, @igarashitm perhaps we should close this and open a new issue detailing what needs to be implemented in this new feature?

[igarashitm]

I'm OK with it, I'd just expect something open so that we won't forget :)

[zregvart]

I've created #2397 feel free to comment over there if I've missed anything.