Fastapi: [QUESTION] enhance serialization speed
Description
I have to return sometimes big objects, I'm constrained in the fact that chunking them is not an option
An example of such an object would be a dict {"key:: value} where value is a list of list, 20 list of 10k elements.
I wrote this simple test case that shows quite clearly the massive hit in several scenarios (run with pytest tests/test_serial_speed.py --log-cli-level=INFO)
Here's the output:
======================================================================================================================= 1 passed in 3.68 seconds =======================================================================================================================
(fastapi) ➜ fastapi git:(slow_serial) ✗ pytest tests/test_serial_speed.py --log-cli-level=INFO
========================================================================================================================= test session starts ==========================================================================================================================
platform linux -- Python 3.6.8, pytest-5.0.0, py-1.8.0, pluggy-0.12.0
rootdir: /home/lotso/PycharmProjects/fastapi
plugins: cov-2.7.1
collected 1 item
tests/test_serial_speed.py::test_routes
---------------------------------------------------------------------------------------------------------------------------- live log call -----------------------------------------------------------------------------------------------------------------------------
INFO tests.test_serial_speed:test_serial_speed.py:39 route1: 0.05402565002441406
INFO tests.test_serial_speed:test_serial_speed.py:18 app.tests.test_serial_speed.route1, 9.395180225372314, ['http_status:200', 'http_method:GET', 'time:wall']
INFO tests.test_serial_speed:test_serial_speed.py:18 app.tests.test_serial_speed.route1, 9.395131000000001, ['http_status:200', 'http_method:GET', 'time:cpu']
INFO tests.test_serial_speed:test_serial_speed.py:52 route1: 0.049863576889038086
INFO tests.test_serial_speed:test_serial_speed.py:18 app.tests.test_serial_speed.route2, 10.358616590499878, ['http_status:200', 'http_method:GET', 'time:wall']
INFO tests.test_serial_speed:test_serial_speed.py:18 app.tests.test_serial_speed.route2, 10.358592000000002, ['http_status:200', 'http_method:GET', 'time:cpu']
INFO tests.test_serial_speed:test_serial_speed.py:64 route1: 0.05589580535888672
INFO tests.test_serial_speed:test_serial_speed.py:18 app.tests.test_serial_speed.route3, 11.318845272064209, ['http_status:200', 'http_method:GET', 'time:wall']
INFO tests.test_serial_speed:test_serial_speed.py:18 app.tests.test_serial_speed.route3, 11.318446000000002, ['http_status:200', 'http_method:GET', 'time:cpu']
PASSED [100%]
====================================================================================================================== 1 passed in 31.60 seconds =======================================================================================================================
all routes do the same with slight variations:
- build a big object, it's a dict, one key and a list of list value with 3 sublists of 100k elements, a little bit extreme maybe but it's to show clearly the impact
- return the object
as you can see the time taken to build such an object is small, around 0.05s, but...
route1 just returns it, it takes 9s
route2 returns it but has the response_model=BigData in the signature, it takes 1s more
route3 is not intuitive to me, I thought that by already building a BigData object and returning it, there would be no penalty, but it's again slower
How can I [...]?
improve performance
edit: the tests are available at this branch, can PR should you want to https://github.com/euri10/fastapi/tree/slow_serial
euri10
All 28 comments
Just to confirm, you do have ujson installed? What type of machine are you running this on?
Route 1 is going to be fastest as it is just returning the JSONResponse directly.
Route 2 is taking the dict, running field.validate and then dumping it back out as JSON, so will be slower than route 1.
_I could be wrong on this one, have never really looked at the response portion of the fastapi code, but from a quick glance_
Route 3 constructs the pydantic model (which validates it), then calls field.validate on it again and then dumps it back out as json, making it the slowest.
FWIW, with ujson on a MBP:
test_serial_speed.py::test_routes
------------------------------------------------------------------------------------------------ live log call -------------------------------------------------------------------------------------------------
test_serial_speed.py 41 INFO route1: 0.028443098068237305
test_serial_speed.py 19 INFO app.test_serial_speed.route1, 2.768159866333008, ['http_status:200', 'http_method:GET', 'time:wall']
test_serial_speed.py 19 INFO app.test_serial_speed.route1, 2.7664400000000002, ['http_status:200', 'http_method:GET', 'time:cpu']
test_serial_speed.py 53 INFO route2: 0.02795100212097168
test_serial_speed.py 19 INFO app.test_serial_speed.route2, 3.3919589519500732, ['http_status:200', 'http_method:GET', 'time:wall']
test_serial_speed.py 19 INFO app.test_serial_speed.route2, 3.386338999999999, ['http_status:200', 'http_method:GET', 'time:cpu']
test_serial_speed.py 64 INFO route3: 0.03644418716430664
test_serial_speed.py 19 INFO app.test_serial_speed.route3, 3.939689874649048, ['http_status:200', 'http_method:GET', 'time:wall']
test_serial_speed.py 19 INFO app.test_serial_speed.route3, 3.9318400000000002, ['http_status:200', 'http_method:GET', 'time:cpu']
jekirl
on 4 Jul 2019
I don't have ujson installed and my machine was a dual i5-2697v2 (until today when it seems I fried my mb bios...) well.
Will try ujson even if iirc it's kind of stalled project, isn't it?
euri10
on 4 Jul 2019
I'm investigating it, a bit, with line_profiler.
Maybe calling fastapi.encoders.jsonable_encoder() part is main factor?, and too many calling this function (recursively).
File: /home/me/venvs/fastapi/lib/python3.7/site-packages/fastapi/encoders.py
Function: jsonable_encoder at line 8
Line # Hits Time Per Hit % Time Line Contents
==============================================================
8 @profile
9 def jsonable_encoder(
10 obj: Any,
11 include: Set[str] = None,
12 exclude: Set[str] = set(),
13 by_alias: bool = True,
14 skip_defaults: bool = False,
15 include_none: bool = True,
16 custom_encoder: dict = {},
17 sqlalchemy_safe: bool = True,
18 ) -> Any:
19 130008 85523.0 0.7 3.5 if include is not None and not isinstance(include, set):
20 include = set(include)
21 130008 97039.0 0.7 3.9 if exclude is not None and not isinstance(exclude, set):
22 exclude = set(exclude)
23 130008 172094.0 1.3 6.9 if isinstance(obj, BaseModel):
24 1 2.0 2.0 0.0 encoder = getattr(obj.Config, "json_encoders", custom_encoder)
25 1 1.0 1.0 0.0 return jsonable_encoder(
26 1 1.0 1.0 0.0 obj.dict(
27 1 0.0 0.0 0.0 include=include,
28 1 1.0 1.0 0.0 exclude=exclude,
29 1 1.0 1.0 0.0 by_alias=by_alias,
30 1 22756.0 22756.0 0.9 skip_defaults=skip_defaults,
31 ),
32 1 1.0 1.0 0.0 include_none=include_none,
33 1 1.0 1.0 0.0 custom_encoder=encoder,
34 1 6.0 6.0 0.0 sqlalchemy_safe=sqlalchemy_safe,
35 )
36 130007 116806.0 0.9 4.7 if isinstance(obj, Enum):
37 return obj.value
38 130007 136911.0 1.1 5.5 if isinstance(obj, (str, int, float, type(None))):
39 30001 18834.0 0.6 0.8 return obj
40 100006 76767.0 0.8 3.1 if isinstance(obj, dict):
41 50001 59332.0 1.2 2.4 encoded_dict = {}
42 70002 77972.0 1.1 3.1 for key, value in obj.items():
43 if (
44 (
45 20001 13416.0 0.7 0.5 not sqlalchemy_safe
46 20001 15464.0 0.8 0.6 or (not isinstance(key, str))
47 20001 18121.0 0.9 0.7 or (not key.startswith("_sa"))
48 )
49 20001 13778.0 0.7 0.6 and (value is not None or include_none)
50 20001 14021.0 0.7 0.6 and ((include and key in include) or key not in exclude)
51 ):
52 20001 13916.0 0.7 0.6 encoded_key = jsonable_encoder(
53 20001 12987.0 0.6 0.5 key,
54 20001 13242.0 0.7 0.5 by_alias=by_alias,
55 20001 13011.0 0.7 0.5 skip_defaults=skip_defaults,
56 20001 13224.0 0.7 0.5 include_none=include_none,
57 20001 12971.0 0.6 0.5 custom_encoder=custom_encoder,
58 20001 33429.0 1.7 1.3 sqlalchemy_safe=sqlalchemy_safe,
59 )
60 20001 14937.0 0.7 0.6 encoded_value = jsonable_encoder(
61 20001 13204.0 0.7 0.5 value,
62 20001 13382.0 0.7 0.5 by_alias=by_alias,
63 20001 13164.0 0.7 0.5 skip_defaults=skip_defaults,
64 20001 12975.0 0.6 0.5 include_none=include_none,
65 20001 13333.0 0.7 0.5 custom_encoder=custom_encoder,
66 20001 27140.0 1.4 1.1 sqlalchemy_safe=sqlalchemy_safe,
67 )
68 20001 15872.0 0.8 0.6 encoded_dict[encoded_key] = encoded_value
69 50001 32725.0 0.7 1.3 return encoded_dict
70 50005 54024.0 1.1 2.2 if isinstance(obj, (list, set, frozenset, GeneratorType, tuple)):
71 5 3.0 0.6 0.0 encoded_list = []
72 40009 30858.0 0.8 1.2 for item in obj:
73 40004 29258.0 0.7 1.2 encoded_list.append(
74 40004 29604.0 0.7 1.2 jsonable_encoder(
75 40004 28551.0 0.7 1.2 item,
76 40004 26669.0 0.7 1.1 include=include,
77 40004 27325.0 0.7 1.1 exclude=exclude,
78 40004 26618.0 0.7 1.1 by_alias=by_alias,
79 40004 26593.0 0.7 1.1 skip_defaults=skip_defaults,
80 40004 26568.0 0.7 1.1 include_none=include_none,
81 40004 27065.0 0.7 1.1 custom_encoder=custom_encoder,
82 40004 60135.0 1.5 2.4 sqlalchemy_safe=sqlalchemy_safe,
83 )
84 )
85 5 1.0 0.2 0.0 return encoded_list
86 50000 36219.0 0.7 1.5 errors: List[Exception] = []
87 50000 37246.0 0.7 1.5 try:
88 50000 36024.0 0.7 1.5 if custom_encoder and type(obj) in custom_encoder:
89 encoder = custom_encoder[type(obj)]
90 else:
91 50000 66823.0 1.3 2.7 encoder = ENCODERS_BY_TYPE[type(obj)]
92 return encoder(obj)
93 50000 38776.0 0.8 1.6 except KeyError as e:
94 50000 43301.0 0.9 1.7 errors.append(e)
95 50000 37506.0 0.8 1.5 try:
96 50000 114909.0 2.3 4.6 data = dict(obj)
97 50000 38964.0 0.8 1.6 except Exception as e:
98 50000 41277.0 0.8 1.7 errors.append(e)
99 50000 37823.0 0.8 1.5 try:
100 50000 54317.0 1.1 2.2 data = vars(obj)
101 except Exception as e:
102 errors.append(e)
103 raise ValueError(errors)
104 50000 37823.0 0.8 1.5 return jsonable_encoder(
105 50000 38198.0 0.8 1.5 data,
106 50000 36241.0 0.7 1.5 by_alias=by_alias,
107 50000 35441.0 0.7 1.4 skip_defaults=skip_defaults,
108 50000 34936.0 0.7 1.4 include_none=include_none,
109 50000 34819.0 0.7 1.4 custom_encoder=custom_encoder,
110 50000 75581.0 1.5 3.1 sqlalchemy_safe=sqlalchemy_safe,
111 )
Total time: 4.87313 s
File: /home/me/venvs/fastapi/lib/python3.7/site-packages/fastapi/routing.py
Function: serialize_response at line 37
Line # Hits Time Per Hit % Time Line Contents
==============================================================
37 @profile
38 def serialize_response(
39 *,
40 field: Field = None,
41 response: Response,
42 include: Set[str] = None,
43 exclude: Set[str] = set(),
44 by_alias: bool = True,
45 skip_defaults: bool = False,
46 ) -> Any:
47 1 1.0 1.0 0.0 if field:
48 1 1.0 1.0 0.0 errors = []
49 1 8834.0 8834.0 0.2 value, errors_ = field.validate(response, {}, loc=("response",))
50 1 1.0 1.0 0.0 if isinstance(errors_, ErrorWrapper):
51 errors.append(errors_)
52 1 0.0 0.0 0.0 elif isinstance(errors_, list):
53 errors.extend(errors_)
54 1 0.0 0.0 0.0 if errors:
55 raise ValidationError(errors)
56 1 0.0 0.0 0.0 r = jsonable_encoder(
57 1 0.0 0.0 0.0 value,
58 1 0.0 0.0 0.0 include=include,
59 1 0.0 0.0 0.0 exclude=exclude,
60 1 0.0 0.0 0.0 by_alias=by_alias,
61 1 4864294.0 4864294.0 99.8 skip_defaults=skip_defaults,
62 )
63 1 0.0 0.0 0.0 return r
64 else:
65 return jsonable_encoder(response)
podhmo
on 5 Jul 2019
Maybe, it is good that running include = set(include) only once.
And the cost of calling isinstance() is not cheap.
podhmo
on 5 Jul 2019
And ujson's effect is limited, maybe. because calling response_class is not slow.
fastapi/routing.py:getapp()
Line # Hits Time Per Hit % Time Line Contents
==============================================================
83 @profile
84 async def app(request: Request) -> Response:
85 1 2.0 2.0 0.0 try:
86 1 1.0 1.0 0.0 body = None
87 1 1.0 1.0 0.0 if body_field:
88 if is_body_form:
89 body = await request.form()
90 else:
91 body_bytes = await request.body()
92 if body_bytes:
93 body = await request.json()
94 except Exception as e:
95 logging.error(f"Error getting request body: {e}")
96 raise HTTPException(
97 status_code=400, detail="There was an error parsing the body"
98 ) from e
99 1 1.0 1.0 0.0 solved_result = await solve_dependencies(
100 1 1.0 1.0 0.0 request=request,
101 1 1.0 1.0 0.0 dependant=dependant,
102 1 0.0 0.0 0.0 body=body,
103 1 102.0 102.0 0.0 dependency_overrides_provider=dependency_overrides_provider,
104 )
105 1 1.0 1.0 0.0 values, errors, background_tasks, sub_response, _ = solved_result
106 1 1.0 1.0 0.0 if errors:
107 raise RequestValidationError(errors)
108 else:
109 1 1.0 1.0 0.0 assert dependant.call is not None, "dependant.call must be a function"
110 1 1.0 1.0 0.0 if is_coroutine:
111 raw_response = await dependant.call(**values)
112 else:
113 1 4045.0 4045.0 0.3 raw_response = await run_in_threadpool(dependant.call, **values)
114 1 1.0 1.0 0.0 if isinstance(raw_response, Response):
115 if raw_response.background is None:
116 raw_response.background = background_tasks
117 return raw_response
118
119 1 1.0 1.0 0.0 response_data = serialize_response(
120 1 0.0 0.0 0.0 field=response_field,
121 1 1.0 1.0 0.0 response=raw_response,
122 1 0.0 0.0 0.0 include=response_model_include,
123 1 1.0 1.0 0.0 exclude=response_model_exclude,
124 1 0.0 0.0 0.0 by_alias=response_model_by_alias,
125 1 1345932.0 1345932.0 99.0 skip_defaults=response_model_skip_defaults,
126 )
127 1 2.0 2.0 0.0 response = response_class(
128 1 1.0 1.0 0.0 content=response_data,
129 1 1.0 1.0 0.0 status_code=status_code,
130 1 8758.0 8758.0 0.6 background=background_tasks,
131 )
132 1 32.0 32.0 0.0 response.headers.raw.extend(sub_response.headers.raw)
133 1 1.0 1.0 0.0 if sub_response.status_code:
134 response.status_code = sub_response.status_code
135 1 0.0 0.0 0.0 return response
And this test case's input values are not valid encodable expression.
(e.g. fake.email is used instead of fake.email(), this is typo?)
So, the bottoms of each recursive call, exception is raised again and again, and, in generally, exception handling is high-cost, so this code is too slow.
podhmo
on 6 Jul 2019
yep it's a mistake, I corrected it and don't see same discrepancies, sorry for that !
it seems the original code I based this test case on might have same sort of bug that I didn't detect because of the Any type, which as you said raises exceptions
thanks for the hint, will dig more and see what's happening, Any is evil
euri10
on 8 Jul 2019
I am also experiencing the same/related issue whereby a large nested payload takes ~10 mins to reach a point in the api whereby i can assign it to a variable. I have removed pydantic validation etc to just leave the bare endpoint and tried ujson but this doesnt seem to have helped significantly. for reference, the same payload is processed in ~1.8 seconds in flask.
Charlie-iProov
on 30 Sep 2019
@Charlie-iProov any chance you could put together a minimal example that is much faster in flask? It could be a good starting point for performance work.
dmontagu
on 30 Sep 2019
Hi heres a quick example - not sure where to host the files so just made a public repo https://github.com/MarlieChiller/api-serialisation-comparison the test uses a payload that contains a nested base64 encoded image string as that was my initial use case where i discovered the difference. However, you can swap the image out for an array and i found the difference is still present (although the time difference was reduced). I think the larger the array length, the larger the time discrepancy though
MarlieChiller
on 1 Oct 2019
for reference, in the base example in that repo i was getting approximately 44 seconds in fastapi to 1.4 seconds in flask. When i switched test cases to an array, i used a length of 10000 instead of the image string. Hope that helps
MarlieChiller
on 1 Oct 2019
@MarlieChiller Something is definitely behaving strangely here. I'm getting similar results to you when I run your script. On the other hand, if I make use of the ASGI test client, I get performance in line with flask (maybe a little faster) -- ~1.3s of execution time on my machine (vs ~1.4 for the flask going through the server):
import base64
import json
import sys
from datetime import datetime
import requests as r
from fastapi import FastAPI
from starlette.testclient import TestClient
app = FastAPI(title="fast_api_speed_test")
@app.post("/test")
async def endpoint(payload: dict):
if payload:
print(type(payload))
return 200
test_client = TestClient(app)
def main():
iterations = 1000
with open("black.png", "rb") as image_file:
img = image_file.read()
img = base64.b64encode(img)
fast_api = send_request(iterations, img, 8000)
print("fastapi speed >>> ", fast_api)
def send_request(iterations, encoded_string_img, port):
payload = {"count": iterations, "payload": []}
for i in range(iterations):
payload["payload"].append(
{"arbitrary_field": f"{i}", "image": encoded_string_img.decode("utf-8")}
)
print(sys.getsizeof(json.dumps(payload)))
x = datetime.utcnow()
response = test_client.post("/test", json=payload)
y = datetime.utcnow() - x
print(response.content)
return y
if __name__ == "__main__":
main()
Because of this, I don't think the performance issue is with fastapi, but maybe uvicorn instead? I'm looking into it some more...
dmontagu
on 1 Oct 2019
out of the blue, can it be sync stuff ?
On Tue, Oct 1, 2019 at 11:48 AM dmontagu notifications@github.com wrote:
@MarlieChiller https://github.com/MarlieChiller Something is definitely
behaving strangely here. I'm getting similar results to you when I run your
script. On the other hand, if I make use of the ASGI test client, I get
performance in line with flask (maybe a little faster) -- ~1.3s of
execution time on my machine (vs ~1.4 for the flask going through the
server):import base64import jsonimport sysfrom datetime import datetime
import requests as rfrom fastapi import FastAPIfrom starlette.testclient import TestClientapp = FastAPI(title="fast_api_speed_test")
@app.post("/test")async def endpoint(payload: dict):
if payload:
print(type(payload))
return 200test_client = TestClient(app)
def main():
iterations = 1000with open("black.png", "rb") as image_file: img = image_file.read() img = base64.b64encode(img) fast_api = send_request(iterations, img, 8000) print("fastapi speed >>> ", fast_api)def send_request(iterations, encoded_string_img, port):
payload = {"count": iterations, "payload": []}
for i in range(iterations):
payload["payload"].append(
{"arbitrary_field": f"{i}", "image": encoded_string_img.decode("utf-8")}
)print(sys.getsizeof(json.dumps(payload))) x = datetime.utcnow() response = test_client.post("/test", json=payload) y = datetime.utcnow() - x print(response.content) return yif __name__ == "__main__":
main()Because of this, I don't think the performance issue is with fastapi, but
maybe uvicorn instead? I'm looking into it some more...—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/tiangolo/fastapi/issues/360?email_source=notifications&email_token=AAINSPWFAYZUTS6OUHVQQTTQMMMILA5CNFSM4H4YY6OKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAAVWBA#issuecomment-536959748,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAINSPWJKQOQ3KAWLPO656DQMMMILANCNFSM4H4YY6OA
.
--
benoit barthelet
http://pgp.mit.edu/pks/lookup?op=get&search=0xF150E01A72F6D2EE
euri10
on 1 Oct 2019
@euri10 I don't think that should be the issue (again, the ASGI TestClient speed seems to indicate the problem is not with application-level stuff). But this is deeply disturbing.
To be fair, it is a ~175MB payload, but I still think it should be significantly faster to process.
dmontagu
on 1 Oct 2019
I didnt see Flask code but printing stuff take lts of time
On Tue, Oct 1, 2019 at 11:54 AM dmontagu notifications@github.com wrote:
@euri10 https://github.com/euri10 I don't think that should be the
issue (again, the ASGI TestClient speed seems to indicate the problem is
not with application-level stuff). But this is deeply disturbing.—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tiangolo/fastapi/issues/360?email_source=notifications&email_token=AAINSPXAKJNGNU2UOKI37TLQMMM33A5CNFSM4H4YY6OKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAAWFHY#issuecomment-536961695,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAINSPXKXEPKX33ZS5GCUFLQMMM33ANCNFSM4H4YY6OA
.
--
benoit barthelet
http://pgp.mit.edu/pks/lookup?op=get&search=0xF150E01A72F6D2EE
euri10
on 1 Oct 2019
Okay, so if you change the annotation from dict to Any, the speed is still ridiculously slow. Also, the speed is relatively fast until the single request gets to be about 30+ MBs, at which point the response time starts scaling much-worse-than-linearly. So I think the problem is the server, not the fastapi / the validation. I'm trying to run a uvicorn in a profiler to find where it is slow now.
I think it may be worth trying to run the app using a different server (e.g., hypercorn) to see if that has any impact.
dmontagu
on 1 Oct 2019
@MarlieChiller I simplified your script a bit, isolating the problem as specifically the payload size (and using just starlette, not even fastapi):
import sys
from datetime import datetime
import requests
import uvicorn
from requests import Session
from starlette.applications import Starlette
from starlette.requests import Request
from starlette.responses import Response
from starlette.testclient import TestClient
app = Starlette()
@app.route("/", methods=["POST"])
async def endpoint(request: Request):
payload = await request.json()
assert isinstance(payload, dict)
return Response("success")
def _speed_test(session: Session, url: str):
payload = {"payload": "a" * 100_000_000}
start = datetime.utcnow()
response = session.post(url=url, json=payload)
elapsed = datetime.utcnow() - start
assert response.status_code == 200
assert response.content == b"success"
print(elapsed)
def asgi_test():
client = TestClient(app)
_speed_test(client, "/")
def uvicorn_test():
session = requests.Session()
_speed_test(session, f"http://127.0.0.1:8000/")
def main():
if "--asgi-test" in sys.argv:
asgi_test()
# 0:00:00.650825
elif "--uvicorn-test" in sys.argv:
uvicorn_test()
# 0:00:17.502396
# cProfile:
# Name Call Count Time (ms) Own Time (ms)
# body 391 16670 16649
else:
uvicorn.run(app)
if __name__ == "__main__":
main()
I'm going to post an issue on the starlette and the uvicorn repos about this.
dmontagu
on 1 Oct 2019
sounds good, thanks for the help
MarlieChiller
on 1 Oct 2019
I posted to starlette and uvicorn just now, I guess we'll see if there is any response there!
dmontagu
on 1 Oct 2019
@MarlieChiller I got to the bottom of this -- it was due to how the request body was being built by starlette.
I opened a PR to fix it: https://github.com/encode/starlette/pull/653
dmontagu
on 2 Oct 2019
Nice! Why was += causing a quadratic growth in T vs .join may i ask?
MarlieChiller
on 2 Oct 2019
It has to do with the way strings work in python — a new string is created in memory out of the two inputs every time you call +=. So you are basically making a copy of everything you’ve seen every time the += is called. The list-joining approach doesn’t do any copying until it puts all the pieces together at the end.
dmontagu
on 2 Oct 2019
Just ran into this as well.
The example below is taking about 30ms:
@app.get('/', response_class=UJSONResponse)
async def root():
return {'results': list(range(10000))}
doing the ujson dumps in the body cuts that down to around 3-5ms:
@app.get('/', response_class=UJSONResponse)
async def root():
return ujson.dumps({'results': list(range(10000))})
michalwols
on 2 Oct 2019
Here's some strange behavior I don't understand:
import time
from fastapi import FastAPI
from starlette.responses import UJSONResponse, Response
from starlette.testclient import TestClient
app = FastAPI()
@app.get('/a', response_class=UJSONResponse)
async def root():
content = {'results': list(range(10000))}
return content
@app.get('/b', response_class=Response)
async def root():
content = {'results': list(range(10000))}
return UJSONResponse.render(None, content)
# return ujson.dumps(content, ensure_ascii=False).encode("utf-8")
client = TestClient(app)
t0 = time.time()
for _ in range(100):
client.get("/a")
t1 = time.time()
print(t1 - t0)
# 1.7897768020629883
t0 = time.time()
for _ in range(100):
client.get("/b")
t1 = time.time()
print(t1 - t0)
# 0.32788991928100586
Seems like it's not using UJSONResponse properly; might be a bug.
EDIT: It is using UJSONResponse; the problem is that it is also applying the relatively poorly performing jsonable_encoder to a thing that is already valid json -- not good.
dmontagu
on 3 Oct 2019
I investigated -- the problem is that jsonable_encoder is very slow for objects like this since it has to make many isinstance calls for each value in the returned list.
This seems like a pretty substantial shortcoming -- I think there should be a way to override the use of jsonable_encoder with something faster in cases where you know you don't need its functionality. (Currently you can provide a custom_encoder, but it won't speed things up in cases where you are returning a list/dict since jsonable_encoder will still loop over each entry and perform lots of isinstance checks.)
A 6x overhead is not good!
dmontagu
on 3 Oct 2019
@dmontagu was about to mention that it's mostly likely the jsonable_encoder and all of the validation in serialize_response.
michalwols
on 3 Oct 2019
that's why I stopped using response_class, but I recon people may want to
use the swagger niceness of having it documented
On Thu, Oct 3, 2019 at 3:52 AM Mike notifications@github.com wrote:
@dmontagu https://github.com/dmontagu was about to mention that it's
mostly likely the jsonable_encoder and all of the validation code.—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tiangolo/fastapi/issues/360?email_source=notifications&email_token=AAINSPSGAXZTER7BXZNNBCTQMVF6ZA5CNFSM4H4YY6OKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAGXGCA#issuecomment-537752328,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAINSPQ2USMGKKEID7LF7IDQMVF6ZANCNFSM4H4YY6OA
.
--
benoit barthelet
http://pgp.mit.edu/pks/lookup?op=get&search=0xF150E01A72F6D2EE
euri10
on 3 Oct 2019
Yeah, it's also easy enough to write a decorator that performs the conversion to a response for endpoints you know are safe. Something like:
def go_fast(f):
@wraps(f)
async def wrapped(*args, **kwargs):
return UJSONResponse(await f(*args, **kwargs))
return wrapped
(Might want to use inspect.iscoroutinefunction to also handle def endpoints.)
dmontagu
on 3 Oct 2019
Related issues
danieljfarrell
·
27Comments
nishtha03
·
23Comments
somada141
·
21Comments
sm-Fifteen
·
22Comments
sandys
·
23Comments
Most helpful comment
Yeah, it's also easy enough to write a decorator that performs the conversion to a response for endpoints you know are safe. Something like:
(Might want to use
inspect.iscoroutinefunctionto also handledefendpoints.)