Fastapi: APIRouter is blocked when a high CPU and long time request is being requested

[ChristianMarca]

Description

I created a small example

fibonacci.py

import time

def fib(n):
    return 1 if n < 2 else fib(n - 1) + fib(n - 2)

def run_fib_for_n_seconds(max_number, sec):
    start_time = time.perf_counter()
    total_time = 0

    while True:
        fib(max_number)
        end_time = time.perf_counter()
        total_time = end_time - start_time

        if total_time >= sec:
            break

    return {"time": total_time}

This function is called by a router on fibonacci_route.py

from fastapi import APIRouter
from typing import Optional
from ..routes.fibonacci import run_fib_for_n_seconds

router = APIRouter()


@router.get("/fibonacci", response_model=FunctionExecutionNTimes)
async def get_response_after_high_cpu_usage(max_number_fibo: Optional[int] = 100, time_on_execution: Optional[int] = 30):
      return run_fib_for_n_seconds(max_number_fibo, time_on_execution)

and on the main.py

I have

from fastapi import FastAPI
from app.routes import fibonacci_route

app = FastAPI()

app.include_router(
    fibonacci_route.router,
    prefix="/cpu",
    tags=["cpu"],
    responses={404: {"description": "Not found"}},
)
app.include_router(
    health.router,
    prefix="/health",
    tags=["health"],
    responses={404: {"description": "Not found"}},
)

When I call once at http://localhost:8000/cpu/fibonacci the request starts resolving but when I try calling again at this endpoint or another for example http://localhost:8000/health the requests are blocker before resolve the first request and on the terminal, I get the message 307 Temporary Redirect and afterward 200 OK

[toidi]

This is not due to FastAPI but rather how asyncio and cooperative multitasking works.

Async code is being executed sequentially on a single thread.

So that when you run CPU intensive task - event loop thread is blocked and other async tasks(HTTP handlers for instance) are not executed.

To fix that - you can

1. run synchronous code in an executor
2. making you code asynchronous by async sleeping(await asyncio.sleep(0)) somewhere in your while loop (this approach, in general, is not the best practice)

[ChristianMarca]

This is not due to FastAPI but rather how asyncio and cooperative multitasking works.

Async code is being executed sequentially on a single thread.

So that when you run CPU intensive task - event loop thread is blocked and other async tasks(HTTP handlers for instance) are not executed.

To fix that - you can

1. run synchronous code in an executor
2. making you code asynchronous by async sleeping(await asyncio.sleep(0)) somewhere in your while loop (this approach, in general, is not the best practice)

🙏 Thank you so much, this approach works really well, 😅 I knew that it was my mistake...

[tiangolo]

Thanks for the help here @toidi ! :clap: :bow:

Thanks for reporting back and closing the issue @ChristianMarca :+1:

Also, that's why FastAPI (actually Starlette) supports normal def functions. If you have blocking or CPU intensive code you can use normal def functions and FastAPI (actually Starlette) will take care of running it in a threadpool (using an executor internally).

So, it would be something like:

from fastapi import APIRouter
from typing import Optional
from ..routes.fibonacci import run_fib_for_n_seconds

router = APIRouter()


@router.get("/fibonacci", response_model=FunctionExecutionNTimes)
def get_response_after_high_cpu_usage(max_number_fibo: Optional[int] = 100, time_on_execution: Optional[int] = 30):
      return run_fib_for_n_seconds(max_number_fibo, time_on_execution)

the only difference with yours is the lack of async.

You can read more about it in the docs: https://fastapi.tiangolo.com/async/