Tortoise-orm: How about adding FileField?

[asyncins]

Why is there no Field for file upload?

https://tortoise-orm.readthedocs.io/en/latest/fields.html

Such as FileField

What should I do if I need Fields to upload files now?

[abondar]

Hi

At the moment we haven't implemented FileField because it doesn't bring much to core functionality of tortoise, which we are developing at the moment.
If you are in need of FileField you could try implementing it on your own and use in your project. It should look something like this:

TEXTCHARS = bytearray({7,8,9,10,12,13,27} | set(range(0x20, 0x100)) - {0x7f})


def is_binary_file(file_path: str):
    with open(file_path, 'rb') as f:
        content = f.read(1024)
        return bool(content.translate(None, TEXTCHARS))


class FileField(TextField):
    def __init__(self, *, upload_root: str, **kwargs):
        super().__init__(**kwargs)
        if not os.path.exists(self.upload_root):
            raise ConfigurationError('No such directory: {}'.format(self.upload_root))

        self.upload_root = upload_root

    def _is_binary(self, file: IO):
        return not isinstance(file, io.TextIOBase)

    def to_db_value(self, value: IO, instance):
        is_binary = self._is_binary(value)
        if hasattr(value, 'name'):
            name = value.name
        else:
            name = str(uuid.uuid4())

        if os.path.isfile(os.path.join(self.upload_root, name)):
            name = '{}-{}'.format(str(uuid.uuid4()), name)

        mode = 'w' if not is_binary else 'wb'

        path = os.path.join(self.upload_root, name)

        with open(path, mode) as f:
            f.write(value.read())

        return path

    def to_python_value(self, value: str):
        if is_binary_file(value):
            mode = 'rb'
            buffer = io.BytesIO()
        else:
            mode = 'r'
            buffer = io.StringIO()

        buffer.name = os.path.split(value)[-1]

        with open(value, mode) as f:
            buffer.write(f.read())

        buffer.seek(0)
        return buffer

Note that this code wasn't tested by me at all, and it could really use some refactoring. I am providing it only to give you idea how it could be made and show most obvious pitfalls you could encounter.

I think some time in future we would add FileField to Tortoise, but I think now it's not the moment, at least for myself.
You are also welcome to contribute, we would help you as much as we can.

[grigi]

I think the issue with FileField is that it depends on a CDN hook, that we just don't really want to have.
It is to reference metadata on an externally stored blob.
I feel we should only implement this ontop of some sort of storages system.

[mojimi]

How about just add a binary/blob field? Just so we can leverage the native types

[grigi]

There is plans to introduce it as part of the #81 work. Although enough of that is implemented so that it would work as long as one doesn't filter on the binary content. So could do that sooner rather than later.