Faas: Question: Async function invocation rate

[saad749]

I have an async function that takes around 2-3 seconds to complete and I am using jmeter to invoke the function with 100 threads (users). I then proceed to the Prometheus graphs and check rate(gateway_function_invocation_total[10s]). The rate never goes above 0.6, I have tested with more users as well. It seems the functions are queued and are invoked after the previous has completed, but the initial response is already received by jmeter. Even if I set a lower threshold for scaling, such as

sum
  by(function_name) (rate(gateway_function_invocation_total{code="200"}[10s]))
  > 0.01
for: 2s

The function invocation rate remains same.
If I have a rather fast running function, i.e. just prints Hello World, the function invocation rate can reach upto 85 (if I invoke it a 1000 times, or 10,000)

Expected Behaviour

• Set the number of functions to be invoked concurrently
• The invocation rate should increase when more replicas are available

Current Behaviour

It seems the functions are queued and are invoked after the previous has completed

Context

I have a function (X) that is invoked asynchronously, and it turn invoke other (separate) functions (Y). Currently, the Y functions that are invoked by the initial functions X seem to get into the queue, till all the X functions have completed and then they start to execute. My expectation is that, as X and Y are different functions, they can be invoked in parallel as soon as they are requested. Also it seems only 1 function is invoked at a time, Is it possible to set the number of functions that can be invoked concurrently. I found this open issue https://github.com/openfaas/faas/issues/408 but, I cannot figure how to apply it in this scenario.

Your Environment

• Docker version docker version (e.g. Docker 17.0.05 ):

Client:
 Version:      1.13.1
 API version:  1.26
 Go version:   go1.8.3
 Git commit:   092cba3
 Built:        Thu Oct 12 22:34:44 2017
 OS/Arch:      linux/amd64

Server:
 Version:      1.13.1
 API version:  1.26 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   092cba3
 Built:        Thu Oct 12 22:34:44 2017
 OS/Arch:      linux/amd64
 Experimental: false

• Are you using Docker Swarm or Kubernetes (FaaS-netes)? FaaS-netes

• Operating System and version (e.g. Linux, Windows, MacOS): Ubuntu 17.10

• Language: Python

I hope I was able to explain the my understanding and questions, if not kindly bear with me, I am a bit new to this.

[saad749]

[rgee0]

Hello @saad749 thank you for your query and for using OpenFaaS.

When you state you are using an asynchronous function do you mean that you are using the /async-function/ route to your function?

Your expected behaviour does seem to describe synchronous mode. Have you tried your functions in this mode rather than in async?

[saad749]

Yes by async function, I mean invoking it from /async-function/

[saad749]

E.g. http://192.168.100.19:31112/async-function/hello-python

[rgee0]

Async mode is intended for situations where you looking to:

• Enable longer time-outs
• Process work whenever resources are available rather than immediately
• Consume a large batch of work within a few seconds and let it process at its own pace

If I understand the detail correctly, you are looking to scale to process the work as quickly as possible which would suit synchronous mode. Have you tried the workload through synchronous mode? Have you moved to away from synchronous to async for a reason?

[saad749]

Synchronous mode timeouts because, the initial function X (as described in the context) might just take a couple of seconds or so, but the Y functions are long running tasks and can take several minutes.

I am not really looking to work as quickly as possible, but more towards, why I am observing the above scenario and if I can optimize my solution to be bit more efficient (In case I have some non-optimal configurations).

[alexellis]

Set the number of functions to be invoked concurrently

The queue worker has a default parallelism level of 1 item with a max 30 second timeout.

1. You can add more replicas of the queue worker to increase parallelism.
2. You can alter the inflight items from 1 to N to achieve level-N parallelism per queue worker, but pay attention to the maximum timeout.

The invocation rate should increase when more replicas are available

This is not necessarily true unless you add more nodes to your cluster or have a particularly CPU or memory-hungry function.

Does this answer the question?

Additional context which would be ideal here.

• What is your function doing?

Maybe share the source on GitHub or via a Gist

• Are you using the new watchdog yet? of-watchdog?

If not maybe try it also and follow the link through for the new high-throughput Python template

http://docs.openfaas.com/architecture/watchdog/#of-watchdog

[alexellis]

Synchronous mode timeouts because, the initial function X (as described in the context) might just take a couple of seconds or so, but the Y functions are long running tasks and can take several minutes.

Synchronous or asynchronous makes no difference you have to configure an appropriate timeout for both invocation modes. For more info see this docs page - http://docs.openfaas.com/deployment/troubleshooting/

For a duration of several minutes you will have to increase the timeouts appropriately

[saad749]

You can add more replicas of the queue worker to increase parallelism.

I think that's what I was looking for, All the other stuff seems to make sense now.

Thank you for the link to the docs, didn't knew they existed.

[alexellis]

I recommend joining Slack too - http://docs.openfaas.com/community/#slack-workspace

[saad749]

I recommend joining Slack too - http://docs.openfaas.com/community/#slack-workspace

Thank you. I have sent a email with the request.

[saad749]

Thank you