Fastapi: [BUG] Issue when debugging inside docker

[joaqo]

Describe the bug
When I run pdb in docker I run into issues if I have reload enabled.

To Reproduce
Steps to reproduce the behavior:

from fastapi import FastAPI
from starlette.requests import Request

import sys
sys.path.insert(0, '.')
from app.api.api import api_router
from app.core import config
from app.db.session import Session


app = FastAPI(title='App')
app.include_router(api_router, prefix=config.API_V1_STR)


@app.middleware("http")
async def db_session_middleware(request: Request, call_next):

    # This set_trace crashes only if I started uvicorn with `debug=True`
    import pdb; pdb.set_trace()

    request.state.db = Session()
    response = await call_next(request)
    request.state.db.close()
    return response


# For debugging
if __name__ == "__main__":
    import uvicorn
    uvicorn.run("main:app", host="0.0.0.0", port=80, reload=True, debug=True)

Expected behavior
I should be able to use pdb, but instead get an bdb.BdbQuit exception. This doesn't happen if I run outside docker or if I run inside docker but remove reload=True argument.

Environment:
FROM tiangolo/uvicorn-gunicorn:python3.7

[dmontagu]

I have no idea what could be causing this, but my guess is that it's more likely to be a uvicorn issue than a fastapi issue. I'd recommend trying with a non-fastapi (and ideally non-starlette-based) framework -- if you have the same problem hitting breakpoints, it would indicate the problem is in uvicorn.

@euri10 any other suggestions for debugging in docker? (If I recall you have some experience with this.)

[euri10]

I'm using pycharm which uses pydevd iirc so no experience with pdb.
My guess is that uvicorn reload uses another process and afaik you cant pbd in another process,hence the behavior with reload only.
But you should ask in uvicorn imho, if a simple asgi app crashes the same.

[dmontagu]

@euri10 that's a good insight -- I definitely think the separate process issue could explain the problem

[joaqo]

Hey, thanks for the answers! Separate processes could explain the issue, but its weird that reload + pdb works correctly outside docker, running it in macOS.

[dmontagu]

Hmm that's a good point. Can you run it outside of docker on a machine with the same operating system as the container? (E.g., on a cloud instance?)

[joaqo]

Found the bug! Tried it on a linux machine and it worked just fine. So I started reducing the error to the smallest code possible and it turns out that using "main:app" and reload=True in:

uvicorn.run("main:app", host="0.0.0.0", port=80, reload=True, debug=True)

is what triggers the issue. If I just use app and reload=True it works fine. Thanks for the help.

[dmontagu]

Weird! Glad you were able to figure it out! 🚀

[joaqo]

Turns out that there was a uvicorn warning I didn't catch:

WARNING: auto-reload only works when app is passed as an import string.

Checked, and its true. So I had just turned auto-reload off after all. Regrettably the problem persists. Here is a self contained script in case anyone wants to try it:

from fastapi import FastAPI
from starlette.requests import Request

app = FastAPI()


@app.get("/")
def read_root():
    return {"Hello": "World"}


@app.middleware("http")
async def db_session_middleware(request: Request, call_next):
    import pdb; pdb.set_trace()
    return await call_next(request)


if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=80, reload=True, debug=True)

Tried it in Ubuntu and macOS and it works perfectly, but on a tiangolo/uvicorn-gunicorn:python3.7 docker container it crashes. Cheers!

[dmontagu]

I am not able to reproduce this -- your script seems to work fine for me in my own backend-tests docker container built on top of the official python:3.7 image (which I believe is the base image used by tiangolo/uvicon-gunicorn:python3.7).

In particular, if I start the server, then run curl 0.0.0.0 from another shell in the same container, the debugger starts as usual. Then, if I enter the command c when the debugger comes up, everything goes through as expected. If I enter the command q I get the BdbQuit error, but I get that same error in both macos and in the docker container (and I don't think that is how you are supposed to continue code execution anyway, but I'm not really a pdb user).

Are you doing something specific besides the above that is triggering the error?

[joaqo]

I'm sorry, in the script I posted I used app instead of 'script_file_name:app' in the uvicorn.run command. So if you just pasted that you didn't have reload enabled. If you use 'script_file_name:app' you should get the same error I got. Sorry for wasting your time. This is the script that gives me an error while using Docker:

from fastapi import FastAPI
from starlette.requests import Request

app = FastAPI()


@app.get("/")
def read_root():
    return {"Hello": "World"}


@app.middleware("http")
async def db_session_middleware(request: Request, call_next):
    import pdb; pdb.set_trace()
    return await call_next(request)


if __name__ == "__main__":
    import uvicorn
    uvicorn.run('script_file_name:app', host="0.0.0.0", port=80, reload=True, debug=True)

To run it just do.

docker run -v /local_folder_with_script_above/:/test/ -p 80:80 tiangolo/uvicorn-gunicorn:python3.7

then run in another terminal:

docker exec -it {container_name} bash

and then inside the terminal just pip install fastapi and then python /test/script_file_name.py. Finally go to localhost in your browser. You should get an error in the above terminal instead of a pdb prompt.

[euri10]

I can reproduce it, that should be reported upstream imho, I hit
breakpoints fine in docker container without reload
I even tried with stdin_open: true and tty: true

On Wed, Sep 18, 2019 at 3:46 PM Joaquín Alori [e-mail]
wrote:

I'm sorry, in the script I posted I used app instead of 'script_name:app'
in the uvicorn.run command. So if you just pasted that you didn't have
reload enabled. If you use script_file_name:app' you should get the same
error I got. Sorry for wasting your time. This is the script that gives me
an error while using Docker:

from fastapi import FastAPIfrom starlette.requests import Request

app = FastAPI()

@app.get("/")def read_root():
return {"Hello": "World"}

@app.middleware("http")async def db_session_middleware(request: Request, call_next):
import pdb; pdb.set_trace()
return await call_next(request)

if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=80, reload=True, debug=True)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tiangolo/fastapi/issues/545?email_source=notifications&email_token=AAINSPWZ74GQTQ4BQJDGE5TQKIWJ3A5CNFSM4IXUESS2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7ADUGY#issuecomment-532691483,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAINSPR6RLFNYVM4BZVBITTQKIWJ3ANCNFSM4IXUESSQ
.

--
benoit barthelet
http://pgp.mit.edu/pks/lookup?op=get&search=0xF150E01A72F6D2EE

[joaqo]

Great, thanks. Not really sure what to report upstream means? Is this something I should take care of?

[euri10]

Sorry 🙏 I meant you should report it in uvicorn repository, I tested with a Starlette app and simple asgi app too, so it's more a uvicorn issue

[joaqo]

Gotcha, just one more question regarding etiquette, should I just link to this issue, or rewrite the whole thing there?

[euri10]

Your snippet is excellent I'd repost that and mention the issue here

Le jeu. 19 sept. 2019 à 10:53 PM, Joaquín Alori [e-mail]
a écrit :

Gotcha, just one more question regarding etiquette, should I just link to
this issue, or rewrite the whole thing there?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tiangolo/fastapi/issues/545?email_source=notifications&email_token=AAINSPVRXKBH75W6F3FF5BTQKPRGFA5CNFSM4IXUESS2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7EZPRI#issuecomment-533305285,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAINSPSODWFQ7MBXPBGSJBDQKPRGFANCNFSM4IXUESSQ
.

[joaqo]

Gotcha. Link in case anyone landing here from a search engine wants to follow the issue.

[alimcmaster1]

@joaqo / @tiangolo - we should be good to close this now. Thanks!

[tiangolo]

Thanks @dmontagu , @euri10 for the help here! :clap: :bow:

And thanks @joaqo for reporting back and closing the issue :+1: