Starlette: TestClient on mac takes exactly 30s for every .get call

Created on 15 Jun 2020 · 7Comments · Source: encode/starlette

I previously opened this issue at FastAPI: https://github.com/tiangolo/fastapi/issues/1567, but traced the problem to Starlette.

Details

On Mac with new install of Starlette, every call to TestClient.get() takes exactly 30s to complete. The response, when it arrives, is correct.

Environment

OS: Macbook Pro 10.13.6
Starlette version: 0.13.4
Python version: 3.6

Example

Copy and paste the example from the Starlette docs https://www.starlette.io/testclient/ into a brand new venv:

from starlette.responses import HTMLResponse
from starlette.testclient import TestClient


async def app(scope, receive, send):
    assert scope['type'] == 'http'
    response = HTMLResponse('<html><body>Hello, world!</body></html>')
    await response(scope, receive, send)


def test_app():
    client = TestClient(app)
    response = client.get('/')
    assert response.status_code == 200

Alternatively, you can clone the repo at https://github.com/charlesbaynham/fastapi_mwe and use the requirements.txt to duplicate my environment. Running python test_starlette.py takes 30s. This repo also contains a cProfile of the test which looks like this:

Source

charlesbaynham

Most helpful comment

In my case, each test takes around 7.5 seconds to run, by @erewok comment, I took a look at TestClient which defines base_url as 'http://testserver' by default, which is not resolvable of course.

So I changed the call to TestClient for this:

client = TestClient(app, base_url='http://localhost')

Now the test runs in 0.3s, this is running MacOS 10.14.6 in a MacBook Pro, Python 3.8.0.

Hope this helps some else.

mcmx on 5 Nov 2020

👍4

All 7 comments

Hello, on OSX 10.15.5 with python3.8 I could not reproduce. I cloned the repo you linked here and it completed so quickly that I installed pytest and ran it that way to see some output. With pytest, it took 0.13s for me. I also ran it in iPython:

In [3]: %timeit test_app()
1.23 ms ± 3.95 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

Thinking that maybe it could be linked to Python 3.6, I created a virtualenv using conda under Python 3.6.10 and ran all the above tests there:

❯ pytest test_starlette.py
======================================================== test session starts =========================================================
platform darwin -- Python 3.6.10, pytest-5.4.3, py-1.8.2, pluggy-0.13.1
rootdir: /Users/eaker/open_source/fastapi_mwe
collected 1 item

test_starlette.py .                                                                                                            [100%]

========================================================= 1 passed in 0.13s ==========================================================

And iPython again (slightly slower with a wider standard deviation):

In [3]: %timeit test_app()
1.51 ms ± 22.7 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

So I can't reproduce.

erewok on 17 Jun 2020

Hmmm yes that's exactly what the guys at fastapi found. I'll try using a different version of python.

charlesbaynham on 17 Jun 2020

Got it.

erewok on 17 Jun 2020

I've tried this now in a conda env using python 3.8 and I'm getting the same problem. I'm at a bit of a loss as to how to reproduce this for you without just posting you my laptop... any ideas?

charlesbaynham on 21 Jun 2020

Well, it certainly doesn't seem like it has much to do at all with this library, so you're better off continuing to investigate on your own. A glance at your cProfile, for instance, shows that the 30 second wait goes way further down the stack than this library. Indeed, a quick search of ''osx python socket gethostbyname slow", for instance, reports the following:

It seems that when the MacOS machine does not have a FQDN name manually configured using scutil, python will be fail to resolve domain names and timeout after ~30 seconds.

https://bugs.python.org/issue35164

And:

https://apple.stackexchange.com/questions/175320/why-is-my-hostname-resolution-taking-so-long

If I had to guess, you have something wrong with the hostname on your machine or you're hitting something weird with Python's socket library. Check the last reply on the issue in the Python bug-tracker:

I came across this today with the same timeout behaviour on macOS 10.14.6.

After some digging I tracked it down to the Search Domains setting of my local network, this was set to "local", removing that immediately fixed the issue.

Good luck solving it.

erewok on 22 Jun 2020

👍1

After some digging I tracked it down to the Search Domains setting of my local network, this was set to "local", removing that immediately fixed the issue.

That sounds familiar, I've been messing around with OpenWRT recently. Thanks for your help and sorry to have wasted your time!

charlesbaynham on 22 Jun 2020

👍1

So I changed the call to TestClient for this:

client = TestClient(app, base_url='http://localhost')

Now the test runs in 0.3s, this is running MacOS 10.14.6 in a MacBook Pro, Python 3.8.0.

Hope this helps some else.

mcmx on 5 Nov 2020

👍4

Was this page helpful?

0 / 5 - 0 ratings