Gino: Pytest Problem and give Gino engine is not initialized.

[saeedghx68]

• GINO version: 0.8.0
• Python version: 3.7.0
• asyncpg version: 0.17.0
• aiocontextvars version: 0.2.0
• PostgreSQL version: 10

Description

I want to write tests by pytest-sanic
I saw some weird response after writing or uncommenting my second test.
In other words, It cannot handling more than two test.
Tell us what happened, what went wrong, and what you expected to happen.
At the end, I give this error:

self = <gino.api._PlaceHolder object at 0x7f4b596f95b8>, item = 'close'

    def __getattribute__(self, item):
        if item == '_exception':
            return super().__getattribute__(item)
>       raise self._exception
E       gino.exceptions.UninitializedError: Gino engine is not initialized.

../../.pyenv/versions/3.7.0/envs/twitter/lib/python3.7/site-packages/gino/api.py:501: UninitializedError

What I Did

My test file is:

import server
import pytest
import json
from models.user import User

pytestmark = pytest.mark.asyncio


@pytest.yield_fixture
def app():
    return server.get_app()


@pytest.fixture
def test_cli(loop, app, test_client):
    return loop.run_until_complete(test_client(app))


async def test_register_user(test_cli):
    data = {
        'username': 'test',
        'password': 'test',
        'full_name': 'test'
    }
    resp = await test_cli.post('/api/v1/register', data=json.dumps(data))
    assert resp.status == 201
    await User.delete.where(User.username == data['username']).gino.status()


async def test_register_user_exist(test_cli):
    data = {
        'username' : 'test',
        'password': 'test',
        'full_name': 'test'
    }
    resp = await test_cli.post('/api/v1/register', data=json.dumps(data))
    assert resp.status == 201
    resp = await test_cli.post('/api/v1/register', data=json.dumps(data))
    assert resp.status == 422
    await User.delete.where(User.username == data['username']).gino.status()


async def test_register_invalid_data(test_cli):
    data = {
        'username': 'test2',
    }
    resp = await test_cli.post('/api/v1/register', data=json.dumps(data))
    assert resp.status == 400


async def test_auth(test_cli):
    data = {
        'username': 'test',
        'password': 'test',
        'full_name': 'test'
    }
    resp = await test_cli.post('/api/v1/register', data=json.dumps(data))
    assert resp.status == 201
    del data['full_name']
    resp = await test_cli.post('/api/v1', data=json.dumps(data))
    assert resp.status == 200
    assert json.loads(await resp.read()).get('access_token') != ''
    await User.delete.where(User.username == data['username']).gino.status()

And my server file is:

 import aioredis
 import asyncio
 import uvloop
 from app import app
 from logic.db import initial_db
 from routes import routes


 @app.listener('before_server_start')
 async def before_server_start(app, loop):
     # Initial DB (create tables)
     await initial_db()
     # create redis pool connection and set app config to access anywhere
     app.redis_pool = await aioredis.create_redis_pool(
         (app.config.REDIS_HOST, app.config.REDIS_PORT),
         minsize=5,
         maxsize=10,
         loop=loop,
     )


 @app.listener('after_server_stop')
 async def after_server_stop(app, loop):
     app.redis_pool.close()
     await app.redis_pool.wait_closed()


 def get_app():
     asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
     loop = asyncio.get_event_loop()
     loop.run_until_complete(initial_db())
     routes.load_router()
     return app


 if __name__ == "__main__":
     routes.load_router()
     app.run(host="0.0.0.0", port="2080", debug=app.config)

my logic db file is:

 from app import db, app


 async def initial_db():
     """
     Initial tables
     :return:
     """
     try:
         db.init_app(app)
         db_config = app.config

         # Create tables)
         await db.set_bind(f'postgresql://'
                           f'{db_config.DB_USER}:{db_config.DB_PASSWORD}@'
                           f'{db_config.DB_HOST}:{db_config.DB_PORT}/'
                           f'{db_config.DB_DATABASE}')
         # Create tables
         await db.gino.create_all()
     except Exception as ex:
         global_logger.write_log('error', f"error: {ex}")

my app file is:

from sanic import Sanic
from gino.ext.sanic import Gino
from gino.crud import CRUDModel, UpdateRequest, DEFAULT
from datetime import datetime
import settings


class HookedUpdateRequest(UpdateRequest):
    async def apply(self, bind=None, timeout=DEFAULT):
        self.update(last_modified=datetime.utcnow())
        return await super().apply(bind, timeout)


class HookedCRUDModel(CRUDModel):
    _update_request_cls = HookedUpdateRequest  # requires latest master


db = Gino(model_classes=(HookedCRUDModel,))
app = Sanic(__name__, strict_slashes=True)
settings.app.load_settings(app)

my settings app file is:

def load_settings(app):
    print('loading setting app ...')
    import os
    from dotenv import load_dotenv

    load_dotenv(verbose=True)

    app.config.CORS_ORIGINS = os.getenv('CORS_ORIGINS', '*')

    app.config.DB_HOST = os.getenv('DB_HOST', 'localhost')
    app.config.DB_DATABASE = 'twitter_db'
    app.config.DB_USER = os.getenv('DB_USER', 'saeed')
    app.config.DB_PASSWORD = os.getenv('DB_PASSWORD', '123')
    app.config.DB_PORT = os.getenv('DB_PORT', 5432)

    app.config.REDIS_HOST = os.getenv('REDIS_HOST', 'localhost')
    app.config.REDIS_PORT = os.getenv('REDIS_PORT', '6379')

    app.config.API_VERSION = os.getenv('API_VERSION')
    app.config.API_TITLE = os.getenv('API_TITLE')
    app.config.API_DESCRIPTION = os.getenv('API_DESCRIPTION')
    app.config.API_TERMS_OF_SERVICE = os.getenv('API_TERMS_OF_SERVICE')
    app.config.API_PRODUCES_CONTENT_TYPES = os.getenv('API_PRODUCES_CONTENT_TYPES')
    app.config.API_CONTACT_EMAIL = os.getenv('API_CONTACT_EMAIL')
    app.config.DEBUG = os.getenv('DEBUG', False)

[wwwjfy]

I don't see where gino engine is initialised. If possible, could you post a minimised but complete test case that could reproduce this issue?

[saeedghx68]

I edit my issue, I don't how to fix it

[wwwjfy]

Thanks, I can reproduce the error. Will take a look later today. :)

[wwwjfy]

The reason you get this error is because initial_db gets called more than once for the same app instance.
Each time db.init_app(app) is called, an after_server_stop callback is registered.
So in the teardown step, the cleanup task is performed more than once. The second time the db binding is already released, thus the above error.

And that's why it only appears only when more than one test is run.

Here are the places db.init_app is called:

• before_server_start
• get_app, it's called before each test

I'd suggest

• call db.init_app(app) right after app = Sanic(). It's not an asynchronous call, just to set up hooks.
• no need to call set_bind manually. It's already taken care of by before_server_start hook.
• use a dedicated process to manage db schemas/migrations, but not to call gino.create_all() at server start every time. The reason is while it's OK for unit tests, it'll be a pain to manage that if you want to do migrations other than table creation like adding a column.

[saeedghx68]

Thanks dear Tony

[wwwjfy]

Glad to help. :)