Fastapi: Microservices FastAPI test in docker, return 404

Created on 5 Jul 2020  ·  6Comments  ·  Source: tiangolo/fastapi

First check

  • [x] I added a very descriptive title to this issue.
  • [x] I used the GitHub search to find a similar issue and didn't find it.
  • [x] I searched the FastAPI documentation, with the integrated search.
  • [x] I already searched in Google "How to X in FastAPI" and didn't find any information.
  • [x] I already read and followed all the tutorial in the docs and didn't find an answer.

Project Description

I created a simple library project in microservices to study and implement FastAPI.
Docker starts 5 main services:

  • books
  • db-book
  • author
  • db-author
  • nginx
    Everything works as expected, making requests with postman I have no problem.

Structure

.
├── author-service
│ ├── app
│ │ ├── api
│ │ ├── main.py
│ │ └── __pycache__
│ ├── Dockerfile
│ ├── requirements.txt
│ └── tests
│ ├── __init__.py
│ ├── __pycache__
│ └── test_author.py
├── book-service
│ ├── app
│ │ ├── api
│ │ ├── __init__.py
│ │ ├── main.py
│ │ └── __pycache__
│ ├── Dockerfile
│ ├── requirements.txt
│ └── tests
│ ├── __init__.py
│ ├── __pycache__
│ └── test_book.py
├── docker-compose.yml
├── nginx_config.conf
├── README.md
├── requirements.txt

Problem description

I added a test directory where I test endpoints.

Example of (incomplete) author test

from starlette.testclient import TestClient
from app.main import app
from app.api.author import authors
import logging
log = logging.getLogger('__name__')
import requests

client = TestClient(app)

def test_get_authors():
    response = client.get("/")
    assert response.status_code == 200

def test_get_author():
    response = client.get("/1")
    assert response.status_code == 200

$> docker-compose exec author_service pytest .
returns this

============================================================================================================= test session starts =============================================================================================================
platform linux -- Python 3.8.3, pytest-5.3.2, py-1.9.0, pluggy-0.13.1
rootdir: /app
collected 2 items                                                                                                                                                                                                                             

tests/test_author.py FF                                                                                                                                                                                                                 [100%]

================================================================================================================== FAILURES ===================================================================================================================
______________________________________________________________________________________________________________ test_get_authors _______________________________________________________________________________________________________________

    def test_get_authors():
        response = client.get("/")
>       assert response.status_code == 200
E       assert 404 == 200
E        +  where 404 = <Response [404]>.status_code

tests/test_author.py:12: AssertionError
_______________________________________________________________________________________________________________ test_get_author _______________________________________________________________________________________________________________

    def test_get_author():
        response = client.get("/1")
>       assert response.status_code == 200
E       assert 404 == 200
E        +  where 404 = <Response [404]>.status_code

tests/test_author.py:16: AssertionError
============================================================================================================== 2 failed in 0.35s ==============================================================================================================

I tried to start the tests directly from the container shell but nothing the same.
This problem occurs only with tests that are done following the documentation (using starlette / fastapi) and with requests

I would like to add another thing, following the FastAPI documentation there is written to import
from fastapi.testclient import TestClient
This gives me the following error

============================================================================================================= test session starts =============================================================================================================
platform linux -- Python 3.8.3, pytest-5.3.2, py-1.9.0, pluggy-0.13.1
rootdir: /app
collected 0 items / 1 error                                                                                                                                                                                                                   

=================================================================================================================== ERRORS ====================================================================================================================
____________________________________________________________________________________________________ ERROR collecting tests/test_author.py ____________________________________________________________________________________________________
ImportError while importing test module '/app/tests/test_author.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/test_author.py:1: in <module>
    from fastapi.testclient import TestClient
E   ModuleNotFoundError: No module named 'fastapi.testclient'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================================================================== 1 error in 0.19s ===============================================================================================================

You can find the complete project here
Library Microsrevices example

Environment

  • OS:[Linux Fedora 32]
  • FastAPI Version [0.55.1]:
  • Python: [Python 3.8.3]
answered question
Source
picture Plaoo

All 6 comments

@Plaoo its not fastapi issues, it's the way the imports work, hope this link helps: https://docs.pytest.org/en/stable/pythonpath.html

tomarv2 picture tomarv2 on 5 Jul 2020

Some points:

  • Author service is failing.

    • I just run the docker-compose and the author service doesn't start because there's an error.

  • FastAPI version on book service is 0.48.0, we are at 0.58.1...

    • Check docker exec library-microservices-example_author_service_1 pip freeze

  • You need to use the full URL in the tests, if your prefix is api/v1/authors/, then you need to have this in your request on the tests.

I didn't check more than that.

Kludex picture Kludex on 5 Jul 2020 
* You need to use the full URL in the tests, if your prefix is `api/v1/authors/`, then you need to have this in your request on the tests.

I didn't check more than that.
Made an update of FastAPI now is fastapi==0.58.1, I also tried to add full url, what I use with postman this is the result:

============================================================================================================= test session starts =============================================================================================================
platform linux -- Python 3.8.3, pytest-5.3.2, py-1.9.0, pluggy-0.13.1
rootdir: /app
collected 2 items                                                                                                                                                                                                                             

tests/test_author.py FF                                                                                                                                                                                                                 [100%]

================================================================================================================== FAILURES ===================================================================================================================
______________________________________________________________________________________________________________ test_get_authors _______________________________________________________________________________________________________________

    def test_get_authors():
>       response = client.get("http://0.0.0.0:8080/api/v1/authors/")

tests/test_author.py:11: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python3.8/site-packages/requests/sessions.py:543: in get
    return self.request('GET', url, **kwargs)
/usr/local/lib/python3.8/site-packages/starlette/testclient.py:413: in request
    return super().request(
/usr/local/lib/python3.8/site-packages/requests/sessions.py:530: in request
    resp = self.send(prep, **send_kwargs)
/usr/local/lib/python3.8/site-packages/requests/sessions.py:643: in send
    r = adapter.send(request, **kwargs)
/usr/local/lib/python3.8/site-packages/starlette/testclient.py:243: in send
    raise exc from None
/usr/local/lib/python3.8/site-packages/starlette/testclient.py:240: in send
    loop.run_until_complete(self.app(scope, receive, send))
/usr/local/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
    return future.result()
/usr/local/lib/python3.8/site-packages/fastapi/applications.py:171: in __call__
    await super().__call__(scope, receive, send)
/usr/local/lib/python3.8/site-packages/starlette/applications.py:102: in __call__
    await self.middleware_stack(scope, receive, send)
/usr/local/lib/python3.8/site-packages/starlette/middleware/errors.py:181: in __call__
    raise exc from None
/usr/local/lib/python3.8/site-packages/starlette/middleware/errors.py:159: in __call__
    await self.app(scope, receive, _send)
/usr/local/lib/python3.8/site-packages/starlette/exceptions.py:82: in __call__
    raise exc from None
/usr/local/lib/python3.8/site-packages/starlette/exceptions.py:71: in __call__
    await self.app(scope, receive, sender)
/usr/local/lib/python3.8/site-packages/starlette/routing.py:550: in __call__
    await route.handle(scope, receive, send)
/usr/local/lib/python3.8/site-packages/starlette/routing.py:227: in handle
    await self.app(scope, receive, send)
/usr/local/lib/python3.8/site-packages/starlette/routing.py:41: in app
    response = await func(request)
/usr/local/lib/python3.8/site-packages/fastapi/routing.py:196: in app
    raw_response = await run_endpoint_function(
/usr/local/lib/python3.8/site-packages/fastapi/routing.py:147: in run_endpoint_function
    return await dependant.call(**values)
app/api/author.py:10: in get_authors
    return await db_manager.get_all_authors()
app/api/db_manager.py:12: in get_all_authors
    return await database.fetch_all(query=query)
/usr/local/lib/python3.8/site-packages/databases/core.py:130: in fetch_all
    async with self.connection() as connection:
/usr/local/lib/python3.8/site-packages/databases/core.py:199: in __aenter__
    await self._connection.acquire()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <databases.backends.postgres.PostgresConnection object at 0x7f9a4eb2bf40>

    async def acquire(self) -> None:
        assert self._connection is None, "Connection is already acquired"
>       assert self._database._pool is not None, "DatabaseBackend is not running"
E       AssertionError: DatabaseBackend is not running

/usr/local/lib/python3.8/site-packages/databases/backends/postgres.py:132: AssertionError
_______________________________________________________________________________________________________________ test_get_author _______________________________________________________________________________________________________________

    def test_get_author():
>       response = client.get("http://0.0.0.0:8080/api/v1/authors/1")

tests/test_author.py:15: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python3.8/site-packages/requests/sessions.py:543: in get
    return self.request('GET', url, **kwargs)
/usr/local/lib/python3.8/site-packages/starlette/testclient.py:413: in request
    return super().request(
/usr/local/lib/python3.8/site-packages/requests/sessions.py:530: in request
    resp = self.send(prep, **send_kwargs)
/usr/local/lib/python3.8/site-packages/requests/sessions.py:643: in send
    r = adapter.send(request, **kwargs)
/usr/local/lib/python3.8/site-packages/starlette/testclient.py:243: in send
    raise exc from None
/usr/local/lib/python3.8/site-packages/starlette/testclient.py:240: in send
    loop.run_until_complete(self.app(scope, receive, send))
/usr/local/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
    return future.result()
/usr/local/lib/python3.8/site-packages/fastapi/applications.py:171: in __call__
    await super().__call__(scope, receive, send)
/usr/local/lib/python3.8/site-packages/starlette/applications.py:102: in __call__
    await self.middleware_stack(scope, receive, send)
/usr/local/lib/python3.8/site-packages/starlette/middleware/errors.py:181: in __call__
    raise exc from None
/usr/local/lib/python3.8/site-packages/starlette/middleware/errors.py:159: in __call__
    await self.app(scope, receive, _send)
/usr/local/lib/python3.8/site-packages/starlette/exceptions.py:82: in __call__
    raise exc from None
/usr/local/lib/python3.8/site-packages/starlette/exceptions.py:71: in __call__
    await self.app(scope, receive, sender)
/usr/local/lib/python3.8/site-packages/starlette/routing.py:550: in __call__
    await route.handle(scope, receive, send)
/usr/local/lib/python3.8/site-packages/starlette/routing.py:227: in handle
    await self.app(scope, receive, send)
/usr/local/lib/python3.8/site-packages/starlette/routing.py:41: in app
    response = await func(request)
/usr/local/lib/python3.8/site-packages/fastapi/routing.py:196: in app
    raw_response = await run_endpoint_function(
/usr/local/lib/python3.8/site-packages/fastapi/routing.py:147: in run_endpoint_function
    return await dependant.call(**values)
app/api/author.py:20: in get_author
    author = await db_manager.get_author(id)
app/api/db_manager.py:17: in get_author
    return await database.fetch_one(query=query)
/usr/local/lib/python3.8/site-packages/databases/core.py:136: in fetch_one
    async with self.connection() as connection:
/usr/local/lib/python3.8/site-packages/databases/core.py:199: in __aenter__
    await self._connection.acquire()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <databases.backends.postgres.PostgresConnection object at 0x7f9a4e510dc0>

    async def acquire(self) -> None:
        assert self._connection is None, "Connection is already acquired"
>       assert self._database._pool is not None, "DatabaseBackend is not running"
E       AssertionError: DatabaseBackend is not running

/usr/local/lib/python3.8/site-packages/databases/backends/postgres.py:132: AssertionError
============================================================================================================== 2 failed in 0.71s ==============================================================================================================

If this issue is not directly related to FastAPI, please let me know that I will close immediately. Or recommend a site to talk about it.

Plaoo picture Plaoo on 6 Jul 2020

By full url kludex meant this:

def test_get_authors():
    response = client.get('/api/v1/authors/')
    assert response.status_code == 200


def test_get_author():
    response = client.get('/api/v1/authors/1')
    assert response.status_code == 200

The url is like that because you add a prefix to this router in main.py

app.include_router(authors, prefix='/api/v1/authors', tags=['authors'])

You can add a http://0.0.0.0:8000 if you want but fastapi auto adds it (not sure if that's exactly the case if someone more competent can confirm i would be happy 🙂 )

def test_get_authors():
    response = client.get('http://0.0.0.0:8000/api/v1/authors/')
    assert response.status_code == 200


def test_get_author():
    response = client.get('http://0.0.0.0:8000/api/v1/authors/1')
    assert response.status_code == 200
ArcLightSlavik picture ArcLightSlavik on 6 Jul 2020

Thanks for the help here everyone! :clap: :bow:

If that solves the original problem, then you can close this issue :heavy_check_mark:

Otherwise:

  • From your code it seems you are using an unpinned version of FastAPI, so you probably have an old cached version. That would explain the error:
E   ModuleNotFoundError: No module named 'fastapi.testclient'

  • From the URL: client.get("http://0.0.0.0:8080/api/v1/authors/"), that would work to communicate from outside of the container to the a mapped port. Probably not from inside the container. But especially, if you are using the TestClient you shouldn't add the http://0.0.0.0:8080, as @Kludex and @ArcLightSlavik mention.

  • If you just started the stack, the DB is probably not ready yet for the time you run the tests. You would have to wait for the DB to be ready or use something like tenacity.

tiangolo picture tiangolo on 6 Dec 2020
👍1

Thanks to all of the answers. I was able to start the tests using simply requests and the ip address of the docker0 network 172.17.0.1. I can close the issue, thanks again even if a little OT

Plaoo picture Plaoo on 14 Dec 2020
🎉1
Was this page helpful?
0 / 5 - 0 ratings

Related issues

cjw296 picture cjw296  ·  3Comments
devtud picture devtud  ·  3Comments
iwoloschin picture iwoloschin  ·  3Comments
RenanAlonkin picture RenanAlonkin  ·  3Comments
scheung38 picture scheung38  ·  3Comments