Tortoise-orm: Trying to run tortoise with quart async rest api

[shaloba]

Describe the bug
Hi,

I have simple tortoise example that work fine when i create the python event loop and run it.
but when i run Quart API server I get strange error, I braked my head trying to figure out what is the issue
(I'm running python3.7)
The script I ran is below:

import asyncio
from tortoise import Tortoise, fields
from tortoise.models import Model

from quart import Quart, request, abort

app = Quart(__name__)
app.config.from_object(config)

class Users(Model):
    id = fields.IntField(pk=True)
    status = fields.CharField(20)
    def __repr__(self):
        return str(self.id)

class Workers(Model):
    id = fields.IntField(pk=True)
    status = fields.CharField(20)
    def __repr__(self):
        return str(self.id)

@app.before_first_request
async def initial_db():
    await Tortoise.init({
        'connections': {
            'default':  "postgres://localhost:5432/db",
            'second':  "postgres://localhost:5432/db",
        },
        'apps': {
            'users': {
                'models': ['__main__'],
                'default_connection': 'default',
            },
            'workers': {
                'models': ['__main__'],
                'default_connection': 'second',
            }
        }
    })


def run_quart():
    app.run(port=5002)


def run_tortoise():
    loop = asyncio.get_event_loop()
    loop.run_until_complete(initial_db())

if __name__ == '__main__':
    # run_quart() # not working
    run_tortoise() # working

To Reproduce
run one time the "run_quart" see the error
then run "run_tortoise" see the run succeed

Expected behavior
tortoise ORM will use the Quart event loop

Actual behavior
Tortoise raise "NotADirectoryError" error

Additional context
This is my first time using Tortoise ORM and its looking very cool, I will appreciate any help with this issue

Thanks in advanced!

[grigi]

I have not used Quart before, I know with aiohttp you have a init function where you can add async initialisers to use. It also contained the object state in the app, so you had no other globals.

That code looks like it should work, I will have to try it locally.

[grigi]

Here is a working sample app. (It probably is un-quark-ish as it is my first Quark app)
The info you were looking for is: https://pgjones.gitlab.io/quart/startup_shutdown.html
as https://tortoise-orm.readthedocs.io/en/latest/setup.html says one needs to set-up and tear-down the ORM properly.

I hope this helps :smile:

import asyncio
import json
from random import choice

from quart import Quart

from tortoise import Tortoise, fields
from tortoise.models import Model

STATUSES = ["New", "Old", "Gone"]
app = Quart(__name__)


class Users(Model):
    id = fields.IntField(pk=True)
    status = fields.CharField(20)

    def __str__(self):
        return "User {}: {}".format(self.id, self.status)


class Workers(Model):
    id = fields.IntField(pk=True)
    status = fields.CharField(20)

    def __str__(self):
        return "Worker {}: {}".format(self.id, self.status)


@app.before_serving
async def init_orm():
    await Tortoise.init(db_url="sqlite://:memory:", modules={"models": ["__main__"]})
    await Tortoise.generate_schemas()


@app.after_serving
async def close_orm():
    await Tortoise.close_connections()


@app.route("/")
async def list_all():
    users, workers = await asyncio.gather(Users.all(), Workers.all())
    return json.dumps(
        {
            "users": [str(user) for user in users],
            "workers": [str(worker) for worker in workers],
        },
        indent=4,
    )


@app.route("/user")
async def add_user():
    user = await Users.create(status=choice(STATUSES))
    return str(user)


@app.route("/worker")
async def add_worker():
    worker = await Workers.create(status=choice(STATUSES))
    return str(worker)


if __name__ == "__main__":
    app.run(port=5000)

[shaloba]

@grigi Thanks for your quick response! :)
unfortunately its raise me the same "NotADirectoryError" error, does the example work for you ?

Running on http://127.0.0.1:5000 (CTRL + C to quit) Traceback (most recent call last): File "/opt/pycharm-2017.3.4/helpers/pydev/pydevd.py", line 1668, in <module> main() File "/opt/pycharm-2017.3.4/helpers/pydev/pydevd.py", line 1662, in main globals = debugger.run(setup['file'], None, None, is_module) File "/opt/pycharm-2017.3.4/helpers/pydev/pydevd.py", line 1072, in run pydev_imports.execfile(file, globals, locals) # execute the script File "/opt/pycharm-2017.3.4/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile exec(compile(contents+"\n", file, 'exec'), glob, loc) File "/home/shlomy/PycharmProjects/playground/playground.py", line 80, in <module> app.run(port=5000) File "/home/shlomy/PycharmProjects/playground/venv37/lib/python3.7/site-packages/quart/app.py", line 1392, in run asyncio.run(serve(self, config), debug=config.debug) # type: ignore File "/usr/lib/python3.7/asyncio/runners.py", line 43, in run return loop.run_until_complete(main) File "/usr/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete return future.result() File "/home/shlomy/PycharmProjects/playground/venv37/lib/python3.7/site-packages/hypercorn/asyncio/__init__.py", line 31, in serve await worker_serve(app, config) File "/home/shlomy/PycharmProjects/playground/venv37/lib/python3.7/site-packages/hypercorn/asyncio/run.py", line 184, in worker_serve await gathered_tasks File "/home/shlomy/PycharmProjects/playground/venv37/lib/python3.7/site-packages/hypercorn/utils.py", line 75, in observe_changes mtime = Path(filename).stat().st_mtime File "/usr/lib/python3.7/pathlib.py", line 1151, in stat return self._accessor.stat(self) NotADirectoryError: [Errno 20] Not a directory: '/home/shlomy/PycharmProjects/playground/venv37/lib/python3.7/site-packages/tortoise_orm-0.11.6-py3.7.egg/tortoise/__init__.py'

if so maybe we have inconsistency with the environment ...
I'm using python 3.7.2 with the following packages version that might effect the run

1. tortoise-orm==0.11.6
2. Quart==0.8.1
3. asyncpg==0.18.3

Again thanks for your feedback :)

[grigi]

It ran for me. I used py3.6.6 and
Quart==0.6.12
tortoise-orm==0.11.8
asyncpg==0.18.3

So I should try this on Py3.7 I suppose they changed their api. And I see py3.6 is limited to the older version.

[shaloba]

thanks I will check it out !
unfortunately I have another library that need python 3.7.X so i will need to see what can i do .
your answer was very helpful ! :)

[grigi]

I tested it on py37 and the following pins (latest of everything):

aenum==2.1.2
aiofiles==0.4.0
aiosqlite==0.9.0
blinker==1.4
ciso8601==2.1.1
Click==7.0
h11==0.8.1
h2==3.1.0
hpack==3.0.0
Hypercorn==0.5.3
hyperframe==5.2.0
itsdangerous==1.1.0
Jinja2==2.10
MarkupSafe==1.1.1
multidict==4.5.2
PyPika==0.26.0
pytoml==0.1.20
Quart==0.8.1
sortedcontainers==2.1.0
tortoise-orm==0.11.8
typing-extensions==3.7.2
wsproto==0.13.0

And it worked fine with no changes at all. The only outstanding difference is that my example was using SQLite instead of PostgreSQL.

[shaloba]

very strange ... cause i just run it using python3.6 and the integration worked! but when comeback to python3.7 the error raised aggain .. maybe my virtual env is massed up or something I will create new one and check it again

Thakns @grigi :)

[grigi]

Whilst playing with a more complete quart sample, I discovered that using "__main__" as a model namespace also causes problems, as its behaviour changes depending on how the code is run.
If I do a python quart_example.py then it points to quart_example.
If I do a hypercorn quart_example:app then it points to hypercorn.

We should put a warning about usage of it.

[grigi]

I updated the sample at #121 could you review it for sanity?