Azure-functions-durable-extension: Details about activity function failures do not flow to the calling orchestrator function

[cgillum]

Summary

When an activity function throws an exception, the orchestrator function does not see the exception. Instead, it sees a generic TaskFailedException (defined in DurableTask.Core.dll) which does not include any details about the original exception.

Expected Behavior

1. TaskFailedException should not be directly exposed in orchestrator functions since it is defined in DurableTask.Core.dll
2. The details of the activity function exception should be preserved. At a minimum, it should be preserved as an inner exception.

Motivation

It's currently not possible to catch specific exception types when calling activity functions since the original exception information is lost. It's also not possible to fetch the details of the original exception when getting the failed orchestrator status - i.e. all that is shown is "Exception while executing function: {activity function name}". Exception details from orchestrator functions flow fine.

Other Notes

It's not yet clear how well (if at all) exceptions in sub-orchestrations flow to the parent orchestration. A fix for this issue should also be made to work with sub-orchestrations.

[markheath]

Yes, the inner exception is needed when using the RetryOptions.Handle func with CallActivityWithRetryAsync. Otherwise there's no easy way to make a decision whether to retry or not.

[cgillum]

Agreed. FYI @simonporter since I think we will need DTFx support for this.

[simonporter]

To stop throwing a TaskFailedException represents a breaking change so this would be a larger change.
To preserve the inner exception is an outstanding ask I can try to address in the short term.

[cgillum]

We just checked-in support for this after getting the changes we needed in DurableTask.Core (thanks Simon!). It will go out with the next release.

[markheath]

I've just been experimenting with this (using local dev environment) and although we have the InnerException available now, with the correct message, it does not have the same type as the exception thrown from the activity function - you just get a basic System.Exception. Is this intended behavior? It would be nice to do something like

new RetryOptions { Handle = ex => ex.InnerException is InvalidOperationException }

but that is not currently possible.

[cecilphillip]

I can confirm this in my local environment as well.

[vjhbtigoyal]

This is still happening in V2 version. I checked in my local environment. ex.InnerException is coming as System.Exception so my error handling scenario is not able to process it accordingly.
In my case i was raising a custom exception in scenarios where i need retries. Any suggested way to handle that exception in Handle method?

[cgillum]

@vjhbtigoyal This is a known issue being tracked here: https://github.com/Azure/azure-functions-durable-extension/issues/361. It's tricky because it may require changes to the Durable Task Framework.