Azure-functions-durable-extension: Some exceptions do not deserialize correctly

[cgillum]

Repro steps

[FunctionName("Repro")]
public static async Task Run(
    [OrchestrationTrigger] DurableOrchestrationContextBase context,
    TraceWriter log)
{
    try
    {
        await context.CallActivityAsync("ThrowHttpRequestException", null);
    }
    catch (Exception e)
    {
        log.Warning(e.ToString());
    }
}

[FunctionName("ThrowHttpRequestException")]
public static Task SayHelloAsync([ActivityTrigger] string name)
{
    new HttpResponseMessage(HttpStatusCode.Unauthorized).EnsureSuccessStatusCode();
    return Task.CompletedTask;
}

Exception observed in user code:

Microsoft.Azure.WebJobs.FunctionFailedException: The activity function 'ThrowHttpRequestException' failed: "Failed to deserialize exception from TaskActivity: {"ClassName":"System.Net.Http.HttpRequestException","Message":"Response status code does not indicate success: 401 (Unauthorized).","Data":null,"InnerException":null,"HelpURL":null,"StackTraceString":"   at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()

Exception from Durable Task Framework:

System.Reflection.TargetInvocationException
  HResult=0x80131604
  Message=Exception has been thrown by the target of an invocation.
  Source=mscorlib
  StackTrace:
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
   at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Newtonsoft.Json.Serialization.JsonContract.<>c__DisplayClass73_0.<CreateSerializationCallback>b__0(Object o, StreamingContext context)
   at Newtonsoft.Json.Serialization.JsonContract.InvokeOnDeserialized(Object o, StreamingContext context)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.OnDeserialized(JsonReader reader, JsonContract contract, Object value)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateISerializable(JsonReader reader, JsonISerializableContract contract, JsonProperty member, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
   at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonSerializer.Deserialize(JsonReader reader, Type objectType)
   at DurableTask.Core.Serializing.JsonDataConverter.Deserialize(String data, Type objectType) in C:\Code\GitHub\durabletask\src\DurableTask.Core\Serializing\JsonDataConverter.cs:line 95

Inner Exception 1:
InvalidCastException: Unable to cast object of type 'Newtonsoft.Json.Linq.JObject' to type 'System.Runtime.Serialization.ISafeSerializationData'.

This problem is discussed in this StackOverflow post: https://stackoverflow.com/questions/23771228/unable-to-cast-object-of-type-newtonsoft-json-linq-jobject-to-type-system-run

This is a bug in the exception serialization behavior of the durable task framework.

[thomas-schreiter]

Are there any updated on this issue? I am getting a lot of additional DurableTask.Core.Exceptions.OrchestrationFailureException because of this. This adds noise to the logs and alerts. (For now, I will explicitly filter this exception out).

[shibayan]

@cgillum It seems that most exceptions can be handled without problem by keeping type name to JSON and adding [Serializable] to exception class. 💯

The above Repro steps code also works properly.

Microsoft.Azure.WebJobs.FunctionFailedException: The activity function 'ThrowHttpRequestException' failed: "Exception of type 'System.Net.Http.HttpRequestException' was thrown.". See the function execution logs for additional details. ---> System.Net.Http.HttpRequestException: Exception of type 'System.Net.Http.HttpRequestException' was thrown.
   --- End of inner exception stack trace ---
   at Microsoft.Azure.WebJobs.DurableOrchestrationContext.CallDurableTaskFunctionAsync[TResult](String functionName, FunctionType functionType, String instanceId, RetryOptions retryOptions, Object input) in C:\Users\shibayan\Documents\GitHub\azure-functions-durable-extension\src\WebJobs.Extensions.DurableTask\DurableOrchestrationContext.cs:line 475
   at DurableFunctionSpike.DurableFunctionSpike.DurableSpike.Run(DurableOrchestrationContextBase context, TraceWriter log) in C:\Users\shibayan\source\repos\DurableFunctionSpike\DurableFunctionSpike\Class1.cs:line 25

[cgillum]

@shibayan Awesome! Thanks for contributing these fixes! I will mark this as fix-ready.

[cgillum]

Resolved in v1.8.0 release.

[cjjohansen]

What about handling excpetions from 3rd party dependencies that cannot be serialized?

[cgillum]

Indeed, we made this work for some exceptions but not all. I'll leave this issue open since it is a common issue that people run into and there needs to be an item tracking it.