Azure-functions-durable-extension: When running two durable functions locally, App Lease blocking does not seem to kick in on the second instance

Created on 19 Oct 2020  路  9Comments  路  Source: Azure/azure-functions-durable-extension

Description

We're currently in the processing of testing how the new App Lease functionality works from 2.3.0.

However, when running locally, I can see that both our durable functions (with identical code) attempt to take our an app lease lock and both seem to have succeeded, despite both using the same taskhub. I'm not sure what's missing here from our implementation, if this is expected when running locally, or if this is a bug?

Any insight would be useful.

App Details

  • Durable Functions extension version (e.g. v1.8.3): 2.3.0
  • Azure Functions runtime version (1.0 or 2.0): 3.14.0
  • Programming language used: c#

Attached the console logs to show what I'd seen (Information logging is enabled)
Top is the first instance, bottom is the second instance I started later.

Untitled

bug

All 9 comments

The AppLease feature takes a lease on a container using app name as the lease id. Each app deployed will have a unique app name. If you're testing locally, we currently use machine name as the lease id. This is why each app is able to use the same task hub locally.

Although, I'm not sure why we chose to do this instead of a random guid. When testing this locally I chose guids for this lease id. In my opinion, it makes sense to be able to use this feature when testing locally. It would be confusing to get different behavior in local testing than when deployed.

@ConnorMcMahon @cgillum Any thoughts on this?

I believe the reason we moved away from using a random guid is because the app name is persistent, which has advantages if the app recycles for some reason. That way it can instantly regrab the lease, as opposed to waiting for it to expire before beginning to reprocess.

@amdeel @ConnorMcMahon So with that said, does this mean there isn't a way for us to test locally? Would of been useful to have seen it in action locally before deploying in to a DevTest environment

We could probably do something where if we don't detect the app name due to running in local mode, we use a random GUID.

Seems like that would be a simple enough change.

In the meantime @bendursley, try in your appsettings.json setting different values for WEBSITE_SITE_NAME for both instances you are running, and see if that works.

@ConnorMcMahon unfortunately it hasn't worked with setting WEBSITE_SITE_NAME with different values in the appsettings.json, but it does work when set in our local.settings.json file though.

Hmm, I am not very familiar with how our tooling would treat appsetting.json differently local.settings.json, but that does seem very odd to me. In general, setting this in local.settings.json is probably the right call anyways, as this is a setting you won't want to deploy.

It sounds like you at least have a workaround?

@ConnorMcMahon yeah it was enough for me to be happy to deploy to a DevTest environment.

Interestingly, I let our functions run overnight in Azure and while it mostly seems to have worked, there's four instances of orchestrations running on the function app without a lease. <- Worth raising a new ticket to see if there's anything you guys can see your end?

Have logged #1535 for the second part of my above message.

Thanks for filing that @bendursley. We will definitely take a look!

Was this page helpful?
0 / 5 - 0 ratings