Azure-functions-host: Allow runtime to be pre-warmed along with Premium Plan worker pre-warming

[alexkarcher-msft]

In the Premium Plan we allow workers to be "pre-warmed" to mitigate cold start. Currently this method does not pre-load the runtime and results in a cold start penalty when scaling out beyond your minimum pre-warmed instances while still within your minimum plan size.

This delay is most noticeable in languages with long initialization times like CSX, JS, etc.

The current proposals are:

1. Add an app setting to ping an HTTP endpoint and wait for a response before routing traffic to that app. This work would happen primarily in the platform side.
2. Add a custom trigger type that is always called before traffic is routed to the app.

In both of these proposals the host would need a loopback endpoint by default that is called if the user does not specify a custom pre-warming endpoint

[jeffhollan]

Thanks @alexkarcher-msft. @fabiocav / @alexkarcher-msft my preference is very much on #2. It'd be a bit of a pain for a customer to have to make sure to update an app setting on deployment of a function to some arbitrary route whenever they wanted this feature. Also means as I run the app across environments I have these different env variables I assume?

Anyway know it may be "easier" but I think the right choice from an experience and management point of view would be a custom trigger type. In this case we could either expose an API to invoke it, but also make it automatically invoke whenever the scale controller hits the "warm up" endpoint it does today with no scale controller work needed.

[fabiocav]

I'm assuming we'd limit to a single warm up function per app, correct? Need to design the experience around that (if that's the right assumption) as we don't really have a precedent for that.

[alexkarcher-msft]

Yes, we'd need to limit to one warmup function per app. If it's HTTP under the covers, we can leverage the existing checks to ensure you don't deploy two functions with the same HTTP route

[fabiocav]

If we're using a different trigger type, the validation will happen separately, but that's an implementation detail. What I'm hearing is that, if more than one function is using the warmup trigger, we'd fail indexing with an appropriate message, correct?

[alexkarcher-msft]

Correct. I can't think of anything but trouble and race conditions that would come from multiple warmup triggers existing.

[alexkarcher-msft]

Hello Fabio! Does the endpoint /admin/warmupfunction sound good to you? I don't see anything in this particular issue mentioning an endpoint

[fabiocav]

Let's go with /admin/warmup, as this will still work if a function is not setup to trigger on the warmup request

[jeffhollan]

Nit but I prefer just warmup. User may or may not extend with their own function and simpler.

[fabiocav]

@ankitkumarr FYI. This is the issue tracking the work I mentioned earlier today.

[ankitkumarr]

@jeffhollan / @alexkarcher-msft, What priority would you say this work-item has? If you don't have a hard and short deadline for this, I would be happy to start the work on this item once I am done with things on my plate.

[alexkarcher-msft]

Hi Ankit! This is a GA requirement and should be enabled by the time ANT 85 is in production. That gives you about 6 weeks of time, but it is relatively high priority :)

[jeffhollan]

And just to clarify that would be 6 weeks to "in production" - so with some buffer for the release rollout so we can reliably test

[ankitkumarr]

Ok, I will to pull this in for the next Sprint.

[ankitkumarr]

I did some initial investigation to see when exactly do we see cold starts in warmed instances. A couple of things that I want to leave here (Text Driven Development) --

1. Having a Warmup trigger run prior would definitely help with the first invocation, but the first invocation time may still not be super close to the subsequent invocation to the same function. This would make sense as there's language specific caching and optimizations that usually helps to run the same function faster the next time. For example, if there's require statements in node or import statements in Python at the function level, those imports typically get cached to improve the next run.
2. I think there's value in keeping this Warmup trigger as close to HttpTrigger as possible. This is because HttpTrigger is where cold-start is most reflective. And it would be good to warmup the code-path as close to HttpTrigger as possible.
   I did notice that if I had two HttpTriggers, and I execute both of them simultaneously, the second was fast. It was faster than if I have one TimerTrigger and one HttpTrigger and I execute the HttpTrigger after TimerTrigger had warmed things out. That said, both of these cases were faster than the first function execution in both cases.

[brettsam]

May need to break this one down into smaller items if it's going to span several sprints.

[astryger]

In the Premium Plan we allow workers to be "pre-warmed" to mitigate cold start. Currently this method does not pre-load the runtime and results in a cold start penalty when scaling out beyond your minimum pre-warmed instances while still within your minimum plan size.

I have this sentence to translate:
"While automatically scaling based on demand, use pre-warmed workers to run applications with no delay after being idle, run on more powerful instances, and connect to VNETs."

prewarmed workers?
Are "workers" referring to people?? Or something else?

[jeffhollan]

Fair question. In this case a worker is an instance of a server that is running their app. We often use the word “instance” as well - meaning an instance of your app and more instances are added as more requests come in

[brphelps]

@jeffhollan @ankitkumarr -- I think our usage of Functions started alerting on a large number of 404s when this was rolling out, we've since hidden these particular 404s from our alerting. Is this being tracked anywhere? This is basically just the default integration with App Insights, so I'm assuming this could've happened in other environments too.

[ankitkumarr]

@brphelps, this hasn't started rolling out yet and shouldn't be affecting any environment.
What type of 404s are you getting alerts from? Would you be able to provide some more information?

[brphelps]

Following up offline to not broadcast our azure subscription data :).

[gbilodeau]

@brphelps We are suspecting the same thing on our environment. Where are you in your investigations?

[brphelps]

@gbilodeau -- We saw the issue go away in the past couple of days, I'm assuming the infrastructure changes completed rolling out. @ankitkumarr can you confirm?

[astryger]

Another question you might know the answer to:

Why it is called “wrangling data flow”?

https://channel9.msdn.com/Shows/Azure-Friday/How-to-prepare-data-using-wrangling-data-flows-in-Azure-Data-Factory https://channel9.msdn.com/Shows/Azure-Friday/How-to-prepare-data-using-wrangling-data-flows-in-Azure-Data-Factory

To wrangle means to quarrel or to collect.

I don’t get it at all.

From: Jeff Hollan [mailto:[email protected]]
Sent: Friday, October 4, 2019 9:38 PM
To: Azure/azure-functions-host
Cc: astryger; Comment
Subject: Re: [Azure/azure-functions-host] Allow runtime to be pre-warmed along with Premium Plan worker pre-warming (#4588)

Fair question. In this case a worker is an instance of a server that is running their app. We often use the word “instance” as well - meaning an instance of your app and more instances are added as more requests come in

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/Azure/azure-functions-host/issues/4588?email_source=notifications&email_token=ANMOOE65E3FPNNVTY6IH24DQM6SRRA5CNFSM4HZCYF22YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAM2ITI#issuecomment-538551373 , or mute the thread https://github.com/notifications/unsubscribe-auth/ANMOOE46L5QPDXUH27LCU7DQM6SRRANCNFSM4HZCYF2Q .Bilde er fjernet av sender.

[ankitkumarr]

@brphelps / @gbilodeau, sorry I missed the pings here as this issue was closed. Yes it should have all been resolved.