fastapi's async/await does not provide concurrency with pytorch library

[cherepanovic]

Hello dear fastapi community,

my problem is that all requests will be blocked until one function in a route ends although the async and await are declared.

I trigger the training process of an NN model by a request and use the pytorch library for that. This process blocks all other requests.

Is there any solution for that? I unfortunally could not find something which helps me.

Thanks in advance!

[Mause]

You should run the model in a different thread, by calling it from a synchronous endpoint

[raphaelauv]

use : starlette.concurrency.run_in_threadpool

tuto -> https://bocadilloproject.github.io/guide/async.html#tl-dr

or a backgroung task : https://fastapi.tiangolo.com/tutorial/background-tasks/

close your issue if you do not have a feature request or fastapi bug

thank

[cherepanovic]

will try it out today, thank everyone!