Azure-webjobs-sdk: How to intentionally deadletter a message in azure functions v2

[bledbet]

Repro steps

Provide the steps required to reproduce the problem

1. ServiceBusTrigger fires

2. Function processing identifies a non-retryable error, and it is desirable to send the message straight to deadletter, rather than roll back and retry until the max receive count is exceeded.

3. The ServiceBus client assembly that AFv2 is integrated with can bind the incoming message to the Message class, but unlike the old BrokeredMessage class from previous ServiceBus assemblies, Message does not have a Deadletter() method.

4. Other than deadlettering when a non-retryable error is identified, I'm not looking to have to deal with all message lifecycle in the entire FunctionApp (the solution to turn this off for the entire function app, which could have many functions, when you only need to explicitly deadletter in a few cases, seems heav-handed).

Expected behavior

Couple of options I can think of:

A way to do something like Message.Deadletter(), and return from the function, and the Function runtime realizes that there is nothing further to be done with the message.

A way to throw a specific kind of exception (FunctionNotRetryableException?), and the Function runtime uses this as a signal to deadletter the message rather than just abandon it.

Known workarounds

I can find no other solution than to create my own "deadletter queue" for each subscription/queue that might need this. This is not desirable in large integrations solutions when you already have dozens of subscriptions/queues.

Related information

VisualStudio 15.6.7
Azure Functions and Web Jobs Tools 15.10.2046.0
Microsoft.Azure.WebJobs.Extensions.ServiceBus 3.0.1
Microsoft.Azure.ServiceBus 3.0.2

[bbougot]

Indeed it would be great to manage the message lifecycle from a WebJob: once the ServiceBusTrigger has been triggered, we should be able to Complete or Abandon it for example if the underlying code failed to process the message.

[bledbet]

I don't think I have a problem with letting the Functions runtime handle Complete and Abandon based on whether or not my function throws - in fact, I think I'd rather not (part of what I'm getting at with bullet #4) - all I really want is a way for my function logic to short-circuit the path to the deadletter queue in cases where abandoning and retrying will do no good.

[mpMelnikov]

Really need this feature too.
Is it going to be implemented? Is there any known workarround except creation of my own "deadletter queue"?
If somebody from azure-webjobs-sdk team could provide me some guidance, maybe I can implement this feature?

cc: @mathewc

[mathewc]

Related issue https://github.com/Azure/azure-webjobs-sdk/issues/1535.

In v3, you can bind to the MessageReceiver class, which exposes methods like DeadLetter, Abaondon, Complete, etc. Example:

public static async Task ProcessMessage(
    [ServiceBusTrigger("myqueue")] string message, int deliveryCount,
    MessageReceiver messageReceiver,
    string lockToken)
{
    . . .

    await messageReceiver.DeadLetterAsync(lockToken);

    . . .
}

In this example, the message is bound as a string and the various message properties including lockToken are bound as params. You can also bind the message as a Message Type and access the requisite message properties from there. In v2 the ServiceBus SDK exposed these message methods directly on the BrokeredMessage class itself, but in the latest version of their SDK those methods no longer exist, meaning you have to bind to MessageReceiver to access them.

[bledbet]

Perfect! I'll keep a lookout for v3 release, thanks!

[mathewc]

The v3 I'm referring to is the v3 version of the WebJobs SDK which v2 of Functions already uses. So the code I gave above is already live. Please try it out and let me know if it works for your scenarios.

[bledbet]

Oh, cool - thanks!

[mpMelnikov]

@mathewc Thank you so much! The message flow works perfectly now!

I still have an issue with "The lock supplied is invalid" in logs even with "autoComplete": false" but it's rather related to https://github.com/Azure/azure-functions-host/pull/3179

[mathewc]

@mpMelnikov please share your host.json configuration. Make sure your "serviceBus" section is within the "extensions" section of config, as per here: https://github.com/Azure/azure-functions-host/wiki/host.json-(v2)

[gwoody1984]

When I am debugging locally, I am unable to get a lockToken from the bound parameter, it just comes across as an empty string. I am invoking the function through the use of the admin HTTP endpoint, so I assume there is some way to pass it in there, but I can't find any documentation around how to do that.

It is also important to note that the host.json config does not seem to be recognizing the extensions section. When debugging starts it says the host.json only has the version key value pair, when in fact, the copy on disk has the full config.

Does anyone know of a good working reference?

host.json

{
  "version": "2.0",
  "extensions": {
    "serviceBus": {
      "messageHandlerOptions": {
        "autoComplete": false
      }
    }
  }
}

Related Information
Visual Studio Version: 15.9.0
Azure Functions and Web Jobs Tools 15.10.2046.0
Azure Functions Core Tools 2.2.46
Function Runtime Version: 2.0.12175.0
Microsoft.Azure.WebJobs.Extensions.ServiceBus 3.0.1

[mathewc]

For your first question, invoking the function through the admin invoke endpoint is a debug simulation - no ServiceBus message is involved, so message derived properties like lockToken won't be populated. That invoke API bypasses the message listener loop - no message is added or dequeued.

Your second qeustion is due to a known issue https://github.com/Azure/azure-functions-host/issues/3507. It's just a logging issue that we'll fix soon @brettsam .

[gwoody1984]

Thanks for the follow up @mathewc that makes a ton of sense. Am I able to rely on Microsoft.Azure.ServiceBus.Core.IMessageReceiver in the signature of my function instead of the concrete MessageReceiver class to ease unit testing?

I tried to do so, but debugging locally I am receiving an error that it is unable to bind the parameter. If I use the concrete class, everything works as expected.

Microsoft.Azure.WebJobs.Host: Error indexing method 'Function.Run'. Microsoft.Azure.WebJobs.Host: Can't bind parameter 'messageReceiver' to type 'Microsoft.Azure.ServiceBus.Core.IMessageReceiver'.

[celluj34]

I would also like to see the IMessageReceiver bound. Being able to unit test these functions is paramount to good development. As it stands, I either have to create topics and subscriptions on the fly (which is doable but _awful_) or try to shove functionality into another class (which really breaks inversion of control).

[maartenkools]

Related issue #1535.

In v3, you can bind to the MessageReceiver class, which exposes methods like DeadLetter, Abaondon, Complete, etc. Example:

public static async Task ProcessMessage(
    [ServiceBusTrigger("myqueue")] string message, int deliveryCount,
    MessageReceiver messageReceiver,
    string lockToken)
{
    . . .

    await messageReceiver.DeadLetterAsync(lockToken);

    . . .
}

In this example, the message is bound as a string and the various message properties including lockToken are bound as params. You can also bind the message as a Message Type and access the requisite message properties from there. In v2 the ServiceBus SDK exposed these message methods directly on the BrokeredMessage class itself, but in the latest version of their SDK those methods no longer exist, meaning you have to bind to MessageReceiver to access them.

It's important to know that the MessageReceiver parameter _must_ be called _messageReceiver_, any other name doesn't seem to work.

[chris-rogala]

@mathewc - We really need the binding to be on interfaces and not concrete implementations. Additionally, with the new DI release in Functions, how will this be impacted? I don't want to create another issue if this is already working within the new DI framework or if there are plans to have IMessageReceiver become a bounded parameter.

Thank you in advance for your response!

[mathewc]

Yes, I agree with you guys that you should be able to bind to IMessageReceiver OR MessageReceiver with the framework allowing the polymorphism. Currently the static binding contract stuff is pretty simple (code here). Built-in bindings like this have to match name/type exactly. We could relax the type I think with some work to allow you to bind to base interfaces of the type. Recommend you create a new issue with this suggestion, thanks.

[PingPongSet]

Great woks.

I would like know if binding to interface, e.g. IMessageReceiver vs MessageReceiver, is available now?

[chris-rogala]

mathewc- I did a PR the other week, but while my tests passed locally, the build in the pipeline failed. Can you take a quick look into it? Very simple change: https://github.com/Azure/azure-webjobs-sdk/pull/2218

[ben-burton]

This thread has been really useful for me trying to work out how to plug in a combination of message body and message metadata properties from Service Bus into a Cosmos DB input binding. Apart from digging through the code, is there some documentation somewhere which describes the parameters you can use with MessageReceiver? I can see message (string), deliveryCount and lockToken from the above example

[damienestewart]

@maartenkools thanks for saving my day (spent the last few hours trying to figure out why I couldn't bind to MessageReceiver). How do we know that the parameter can only be named messageReceiver? Did I miss some documentation?

[musa-pro]

Related issue #1535.

In v3, you can bind to the MessageReceiver class, which exposes methods like DeadLetter, Abaondon, Complete, etc. Example:

public static async Task ProcessMessage(
    [ServiceBusTrigger("myqueue")] string message, int deliveryCount,
    MessageReceiver messageReceiver,
    string lockToken)
{
    . . .

    await messageReceiver.DeadLetterAsync(lockToken);

    . . .
}

In this example, the message is bound as a string and the various message properties including lockToken are bound as params. You can also bind the message as a Message Type and access the requisite message properties from there. In v2 the ServiceBus SDK exposed these message methods directly on the BrokeredMessage class itself, but in the latest version of their SDK those methods no longer exist, meaning you have to bind to MessageReceiver to access them.

I'm trying using Java but It throws me error when I tried to get meta information using @BindingName("MessageReceiver") -

Stack: java.lang.NullPointerException
    at com.microsoft.azure.functions.worker.binding.BindingDataStore.getTriggerMetatDataByName(BindingDataStore.java:59)
    at com.microsoft.azure.functions.worker.broker.ParameterResolver.resolve(ParameterResolver.java:62)
    at com.microsoft.azure.functions.worker.broker.ParameterResolver.resolve(ParameterResolver.java:42)
    at com.microsoft.azure.functions.worker.broker.JavaMethodExecutorImpl.execute(JavaMethodExecutorImpl.java:52)
    at com.microsoft.azure.functions.worker.broker.JavaFunctionBroker.invokeMethod(JavaFunctionBroker.java:57)
    at com.microsoft.azure.functions.worker.handler.InvocationRequestHandler.execute(InvocationRequestHandler.java:33)
    at com.microsoft.azure.functions.worker.handler.InvocationRequestHandler.execute(InvocationRequestHandler.java:10)
    at com.microsoft.azure.functions.worker.handler.MessageHandler.handle(MessageHandler.java:45)
    at com.microsoft.azure.functions.worker.JavaWorkerClient$StreamingMessagePeer.lambda$onNext$0(JavaWorkerClient.java:92)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

Any clue?

[kvbutler]

Related issue #1535.
In v3, you can bind to the MessageReceiver class, which exposes methods like DeadLetter, Abaondon, Complete, etc. Example:

public static async Task ProcessMessage(
    [ServiceBusTrigger("myqueue")] string message, int deliveryCount,
    MessageReceiver messageReceiver,
    string lockToken)
{
    . . .

    await messageReceiver.DeadLetterAsync(lockToken);

    . . .
}

In this example, the message is bound as a string and the various message properties including lockToken are bound as params. You can also bind the message as a Message Type and access the requisite message properties from there. In v2 the ServiceBus SDK exposed these message methods directly on the BrokeredMessage class itself, but in the latest version of their SDK those methods no longer exist, meaning you have to bind to MessageReceiver to access them.

It's important to know that the MessageReceiver parameter _must_ be called _messageReceiver_, any other name doesn't seem to work.

Thanks very much that did it. This (messageReceiver naming convention) should be in BIG BOLD text somewhere in the documentation...