Fastapi: multipart/form-data: Unable to parse complex types in a request form

[Vivalldi]

First check

• [x] I added a very descriptive title to this issue.
• [x] I used the GitHub search to find a similar issue and didn't find it. - _See additional context_
• [x] I searched the FastAPI documentation, with the integrated search.
• [x] I already searched in Google "How to X in FastAPI" and didn't find any information.
• [x] I already read and followed all the tutorial in the docs and didn't find an answer.
• [x] I already checked if it is not related to FastAPI but to Pydantic.
• [x] I already checked if it is not related to FastAPI but to Swagger UI.
• [x] I already checked if it is not related to FastAPI but to ReDoc.
• [x] After submitting this, I commit to one of:
  
  
  
  • Read open issues with questions until I find 2 issues where I can help someone and add a comment to help there.
  
  
  • I already hit the "watch" button in this repository to receive notifications and I commit to help at least 2 people that ask questions in the future.
  
  
  • Implement a Pull Request for a confirmed bug.
  
  

Example

Here's a self-contained, minimal, reproducible, example with my use case:

import inspect
from typing import Dict, Type

from fastapi import Depends, FastAPI, File, Form
from pydantic import BaseModel

app = FastAPI()


def as_form(cls: Type[BaseModel]):
    """
    Adds an as_form class method to decorated models. The as_form class method
    can be used with FastAPI endpoints
    """
    new_params = [
        inspect.Parameter(
            field.alias,
            inspect.Parameter.POSITIONAL_ONLY,
            default=(Form(field.default) if not field.required else Form(...)),
            annotation=field.outer_type_,
        )
        for field in cls.__fields__.values()
    ]

    async def _as_form(**data):
        return cls(**data)

    sig = inspect.signature(_as_form)
    sig = sig.replace(parameters=new_params)
    _as_form.__signature__ = sig
    setattr(cls, "as_form", _as_form)
    return cls


@as_form
class Item(BaseModel):
    name: str
    another: str
    opts: Dict[str, int] = {}


@app.post("/test", response_model=Item)
def endpoint(item: Item = Depends(Item.as_form), data: bytes = File(...)):
    print(len(data))
    return item


if __name__ == "__main__":
    import json
    import os

    from fastapi.testclient import TestClient

    tc = TestClient(app)

    item = {"name": "vivalldi", "another": "mause"}

    data = bytearray(os.urandom(1))
    files = {"data": ("data", data, "text/csv")}

    r = tc.post("/test", data=item, files=files)
    assert r.status_code == 200
    assert r.json() == {"name": "vivalldi", "another": "mause", "opts": {}}

    files["opts"] = (None, json.dumps({"a": 2}), "application/json")
    r = tc.post("/test", data=item, files=files)
    assert r.status_code == 200
    assert r.json() == {"name": "vivalldi", "another": "mause", "opts": {"a": 2}}

Description

tl;dr

Complex data types (objects) are not supported in multipart/form-data by FastAPI. There are workarounds for top level fields in pydantic models; nested objects are not supported. Save the above script as nested.py. Run python ./nested.py to see failing assertions. And call uvicorn nested:app to run the FastAPI app locally. Local tests can be run using httpie script in this gist

full

If you look through the StackOverflow & FastAPI issues you'll find plenty of examples of how to convert a model into a form. This issue is to address the shortcomings of those workarounds. The main focus of this is to determine the best way to work with multi-content multipart/form-data requests.

Per the OpenAPI Special Considerations for multipart Content "boundaries MAY be used to separate sections of the content being transferred." This indicates that you can specify the content type of each individual part. An example of a combination multipart request can be found in this gist. The gist was generated using httpie.

Going forward I intend to investigate what can be done from a workaround standpoint. Initially, I think we can adjust the workarounds to set object types to a File and parse those files as JSON (might require some model tweaks as well). Additionally, if this issue gains traction I will look into making changes that allow FastAPI to better support multi-content multipart requests.

Environment

• OS: macOS
• FastAPI Version: 0.58.1
• Python version: 3.8.6

Additional context

This is a spin-off of #2365 at Mause's request

Since the root cause is complex types (objects) in forms aren't supported, this may be a duplicate of #2295

[Mause]

Try this:

import inspect
from typing import Dict, Type, TypeVar, Protocol, Generic, NewType

from fastapi import Depends, FastAPI, File, Form
from pydantic import BaseModel, validator, BaseSettings, Json

app = FastAPI()

StringId = NewType('StringId', str)


def as_form(cls: Type[BaseModel]):
    """
    Adds an as_form class method to decorated models. The as_form class method
    can be used with FastAPI endpoints
    """
    new_params = [
        inspect.Parameter(
            field.alias,
            inspect.Parameter.POSITIONAL_ONLY,
            default=(Form(field.default) if not field.required else Form(...)),
        )
        for field in cls.__fields__.values()
    ]

    async def _as_form(**data):
        return cls(**data)

    sig = inspect.signature(_as_form)
    sig = sig.replace(parameters=new_params)
    _as_form.__signature__ = sig
    setattr(cls, "as_form", _as_form)
    return cls


@as_form
class Item(BaseModel):
    name: str
    another: str
    opts: Json[Dict[str, int]] = '{}'


@app.post("/test")
async def endpoint(item: Item = Depends(Item.as_form)):
    return item.dict()


if __name__ == "__main__":
    import json
    import os

    from fastapi.testclient import TestClient

    tc = TestClient(app)

    item = {"name": "vivalldi", "another": "mause"}

    data = bytearray(os.urandom(1))
    files = {"data": ("data", data, "text/csv")}

    r = tc.post("/test", data=item, files=files)
    assert r.status_code == 200, r.text
    assert r.json() == {"name": "vivalldi", "another": "mause", "opts": {}}

    files["opts"] = (None, json.dumps({"a": 2}), "application/json")
    r = tc.post("/test", data=item, files=files)
    assert r.status_code == 200
    assert r.json() == {"name": "vivalldi", "another": "mause", "opts": {"a": 2}}

The main two changes were using the Json class from pydantic, and removing the annotation from the as_form method, as otherwise pydantic would be validating the data twice - once for as_form, and once for the model itself

[tricosmo]

Hi, I am just wondering why not add something like this. The error(type is not dict) is the value of Item comes in as string when it is multipart/form-data. This solution also works with nested classes, it also won't unpack all fields of Item into form fields, which is not very nice if this Item class is a little complicated. Happy to create a PR if this is really a good solution.

[Mause]

That snippet alone isn't really sufficient, as it doesn't include any pydantic validation. But feel free to submit a pr with appropriate documentation and tests if you think it's worth it.

[tricosmo]

@Mause please could you review the PR above? thanks

[Vivalldi]

The above snippet works for me. The only thing (as mentioned) is that if you want to share a base model across endpoints (some with form-encoding and some with pure JSON) then you'll run into needing to nest JSON. Not a huge issue, and one I can workaround for now. Many thanks @Mause

[havardthom]

FYI you can also wrap nested models in the Json class e.g. opts: Json[OptsModel]
And if you want pydantic validation errors to propagate correctly as a fastapi request validation error instead of 500 error, you can add:

async def _as_form(**data):
    try:
        return cls(**data)
    except pydantic.ValidationError as e:
        raise fastapi.exceptions.RequestValidationError(e.raw_errors)

I'm trying to add a swagger example value for json field in the form with no luck, anyone have ideas?
edit:
For adding swagger examples in form fields:

openapi_schema = cls.schema()
new_params = []
for field in cls.__fields__.values():
    example = openapi_schema.get("example", {}).get(field.name)
    default = Form(field.default, example=example) if not field.required else Form(..., example=example)
    new_params.append(
        inspect.Parameter(
            field.alias,
            inspect.Parameter.POSITIONAL_ONLY,
            default=default,
        )
    )