Fastapi: [QUESTION] How ot handle generic text.plain POST requests

[CockyAmoeba]

Description
I am trying to write an endpoint to accept a request that is only sending text/plain body. I am struggling ot understand from the documentation how to handle this and have been getting a raft of errors

Are you able to provide sage advice? Is this possible a due to FASTAPI being "typed" and expecting json data to fulfil pydantic model schema?

header of request begin made to my api:

host: <host>
Accept-Encoding: gzip
Content-Type: text/plain; charset=utf-8
User-Agent: Go-http-client/1.1
Content-Length: 46
Connection: keep-alive

example payload:

EURUSD Less Than 1.09092
{"Condition": "value"}
[3,4,5,] 
{}

Additional context
I first tried:

@app.post("/webhook")
async def the_webhook(body: dict):

But that would give - [2019-09-30 12:19:45 +0000] [8] [INFO] ('172.17.0.1', 55782) - "POST /webhook HTTP/1.1" 422

@app.post("/webhook")
async def the_webhook(request: Request):
    body = await request.body()

Which was also unsuccessfull.

I REALLY wish to use FastAPI going forward but i am struggling ot understand how to do VERY basic things as the framework is trying to enforce schema and structure - I know this is the ideal but in this scenario I have no control on the incoming webhook that I need to catch!

I hope someone can help!

Regards,

[euri10]

I think this has been asked several times already, maybe it was on the chat though 😉
You may try

body: Any = Body(...)

[CockyAmoeba]

@euri10 thanks for the hint but i get teh following error:

ERROR:root:Error getting request body: Expecting value: line 1 column 1 (char 0)
[2019-09-30 21:19:06 +0000] [10] [INFO] ('172.17.0.1', 55922) - "POST /webhook2 HTTP/1.1" 400

Any further advice? Any documentation I can read to better understand the Body class?

[dmontagu]

@euri10 actually, Body(...) is still parsed as json, which is why @CockyAmoeba is running into the error.

You can use Form(...) to prevent the json parsing, but this still requires the data to be structured as form data.

EDIT: Removed my original response showing the use of Form as I don't believe its what you want, but just reading the request.body() should work (see next comment for more detail).

[dmontagu]

@CockyAmoeba

I'm not sure why you ran into an error with trying to read the data off the body. The following code works for me:

from fastapi import FastAPI
from starlette.requests import Request
from starlette.responses import Response
from starlette.testclient import TestClient

app = FastAPI()


@app.post("/webhook", response_class=Response)
async def the_webhook(request: Request):
    return await request.body()


data = b"""EURUSD Less Than 1.09092
{"Condition": "value"}
[3,4,5,]
{}"""

client = TestClient(app)
response = client.post("/webhook", data=data)
print(response.content)
# b'EURUSD Less Than 1.09092\n{"Condition": "value"}\n[3,4,5,]\n{}'

Without the response_class=Response the data comes back json-encoded, but it doesn't fail, so I'm not sure what error you are running into.

Maybe you can share your error message if this isn't working for you?

[CockyAmoeba]

@dmontagu Thank you so much for refocusing me and confirming the approach. This does work for me . - I have no idea what went wrong previously.

For clarity could you please explain or direct me to the documentation about what the ... means in the ~Body(...)andForm(...)` - I am not familiar with this syntax as it does run....

[dmontagu]

@CockyAmoeba

tl;dr: Body(...) means a body parameter that is required. The first argument to Body, Form, etc., is the default value, and ... is just used to signify that there is no default value. You might think None would be more conventional, but the problem is that None is often a desirable default value. I've included more detail below.

---

Body, Form, Query, etc. are all subclasses of pydantic.Schema. pydantic.Schema (which, for future reference, will be renamed to Field in pydantic v1) is intended as a way of describing the contents that should be contained in a field of a model. FastAPI essentially takes the schemas that occur in your endpoint definition, and converts them into a pydantic model specific to the appropriate endpoint that is populated from the HTTP request. The fields on that model are then passed to the endpoint function (or used to solve dependencies, which are passed to the endpoint function).

---

The pydantic Schema class takes the default value for the field as the first positional argument to its initializer. However, if we want the argument to be required, we need a way to specify that there is no default value. One common convention for this is to make the argument optional with a default value of None, and treat it as though it wasn't provided if it takes the value None.

The problem with this is that None is often a useful default value! So we need a different sentinel value to use if the goal is to say "this field is required, but it has no default". That's where Ellipsis (aka ...) comes in -- it just signifies that there is no default value, but the field should be required.

(Note, there are other ways in python to achieve this goal that might not involve the use of ..., but they each have other tradeoffs that may make them undesirable.)

---

Since Body, Form, Query, etc. are all subclasses of pydantic.Schema, the idea is that Body(...) means you are specifying a body parameter that is required. Body(None), which you may also see frequently, means that the default value is None -- pydantic will automatically translate this to meaning the value should be treated as Optional (this is the same as what typing.get_type_hints does if you set the default value of a parameter to None even if you don't include Optional in the annotated type).

You can also make the field optional by with a non-None default: -- for example, if you had the endpoint function

@app.post("/")
def f(x: Dict[str, Any] = Body({}), y: Dict[str, Any] = Body({})):
    ...

Then the variable x provided by the framework when calling f for the request would take the value {} if "x" didn't occur as a key in the json-parsed body. For comparison, if Body({}) was replaced with Body(None), the default value would be None, and if Body({}) was replaced with Body(...), you'd get a 422 response (due to a validation error) if "x" didn't occur as a key in the json parsed body.

---

This use of Ellipsis (or ...) is actually more-or-less conventional beyond fastapi and pydantic -- for example, it can be used for a similar purpose when declaring overloaded methods for mypy using typing.overload.

[dmontagu]

This is also described in various places in the docs, though not quite as head-on as the description above.

For example, this is explained in the section titled "Use Query as the default value` near the top of this docs page. (It is also mentioned in other parts of the docs, but that was the first I found.)

[CockyAmoeba]

@dmontagu Thank you so much for the clarification and detailed explanation. I will continue my missing of advocating FastAPI !

Regards!

[jbkoh]

Hello @CockyAmoeba and @dmontagu , I am new to FastAPI. Do you know how text.plain in POST request behaves in OpenAPI doc? When I follow the example you created, request does not show up as a parameter in the documentation /docs.

Thanks!

[phy25]

@jbkoh request is FastAPI-internal and thus will never be exposed to openapi end. You may want to use Body(..., media_type="text/plain") to let OpenAPI know. (Didn't test this, not sure if this is a good way)

Reference: https://github.com/tiangolo/fastapi/pull/439

[jbkoh]

@phy25
Thanks for the pointer! Unfortunately, I am still struggling with it; not sure if I am using the API correctly. I tried two things:

request: Dict[str, Any] = Body('', media_type="text/plain"),
request: Any = Body(..., media_type="text/turtle"),

These seem to add it to OpenAPI documentation, but now the proper request is rejected and saying:

ERROR:    Error getting request body: Expecting value: line 1 column 1 (char 0)

What's the right schema for text/plain? Thanks in advance!

[dmontagu]

I believe the problem here is that when you use Body, it tries to load the body as JSON regardless of the media type. (Not 100% sure about this, but if you are getting a JSONDecodeError that's probably the problem).

If you just want the text of the body, try using body = await request.body() with request: starlette.requests.Request in your endpoint.

It should be possible to override the openapi spec generation to ensure this gets documented properly. @tiangolo may have some better idea.

[jbkoh]

@dmontagu Thanks for the information. I was able to use Request as the input type, and, as you said, Request seems overriding the openapi spec.

@phy25

• With request: Request = Body(..., media_type='text/plain'), the API is working, but Body does not show up in openapi doc.
• With request: str = Body(..., media_type='text/plain'), Body does show up in openapi doc, but the API is not working ("Error getting request body: Expecting value: line 1 column 1 (char 0) 400 Bad Request") So I think @dmontagu's observation is right although it doesn't say it's a JSONDecodeError. Actually it was JSONDecodeError.

[jbkoh]

@tiangolo I could update https://github.com/tiangolo/fastapi/blob/9c3c9b6e78768374868d690bc05918d58481e880/fastapi/routing.py#L114
like this:

if body_bytes and request.headers['Content-Type'] == 'application/json':
# if body_bytes and request.headers.get('Content-Type', 'application/json'): # if the content type should be assumed.
    body = await request.json()
else:
    body = body_bytes

This works for me but not sure if there would be any other implications. Let me know if you'd like me to PR this one.

[dmontagu]

@jbkoh I think this is worth a separate issue. I'm not sure if your approach is the best/safest way to handle this (given who knows what people might be doing now with the content-type header), but even if not it might still be worth providing some easy way to get the OpenAPI media type documented properly.

Actually, it wouldn't surprise me if this was already possible in some way that was a little closer to your request: Request = Body(..., media_type='text/plain') (though obviously not exactly that). But it's probably worth a dedicated issue, even if there is already a solution.

[jbkoh]

@dmontagu I will create a new Issue and continue this discussion there. Thanks for the guidance.

[tiangolo]

Thanks for the help here everyone! :clap: :bow:

Thanks for reporting back and closing the issue @CockyAmoeba :+1: