Flask-migrate: Flask not generating any migration

[michaelbukachi]

Hello there,
I'm in a huge pickle. Flask-Migrate is not generating any migrations. I keep getting the following:

INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.env] No changes in schema detected.

My directory structure is as follows:

.
├── app
│   ├── config.py
│   ├── credentials.json
│   ├── database.py
│   ├── errors.py
│   ├── factory.py
│   ├── fake.json
│   ├── models
│   │   ├── helium.py
│   │   ├── __init__.py
│   │   ├── logs.py
├── app.db
├── credentials.json
├── env-example
├── fake.json
├── migrations
│   ├── alembic.ini
│   ├── env.py
│   ├── env.pyc
│   ├── README
│   ├── script.py.mako
│   └── versions
├── pytest.ini
├── README.md
├── requirements.txt
├── run.py
├── sample-loan.json
├── tasks.py
├── test.sh
├── worker.sh
├── wsgi.py

Here's my main __init__.py file:

from .factory import Factory


def create_app(environment='development'):
    factory = Factory(environment)
    factory.set_flask()
    factory.set_celery()

    factory.set_db()
    factory.set_migration()
    from . import models


    factory.register(lms_blueprint)
    factory.register(mifos_blueprint)
    factory.register(br_blueprint)

    return factory.flask

My model's __init__.py file is as follows:

__all__ = ['helium', 'logs']

My database.py file is as follows:

from flask_migrate import Migrate
from flask_sqlalchemy import SQLAlchemy

db = SQLAlchemy()
migrate = Migrate()

They are later initialized in factory.py below:

from __future__ import absolute_import

import logging
import logging.config
import os
import sys
from logging.handlers import RotatingFileHandler

from flask import Flask

from .config import config
from .database import db, migrate
from .tasks import celery_app, redis


class Factory:

    def __init__(self, environment='development'):
        self._environment = os.environ.get('APP_ENVIRONMENT', environment)

    @property
    def environment(self):
        return self._environment

    @environment.setter
    def environment(self, environment):
        self._environment = environment
        self.flask.config.from_object(config[self._environment])

    def set_flask(self, **kwargs):
        self.flask = Flask(__name__, **kwargs)
        self.flask.config.from_object(config[self._environment])
        # setup logging
        file_handler = RotatingFileHandler('br.log', maxBytes=10000, backupCount=1)
        file_handler.setLevel(logging.INFO)
        self.flask.logger.addHandler(file_handler)
        stdout = logging.StreamHandler(sys.stdout)
        stdout.setLevel(logging.DEBUG)
        self.flask.logger.addHandler(stdout)
        # werkzeug = logging.getLogger('werkzeug')
        # werkzeug.setLevel(logging.DEBUG)
        # self.flask.logger.addHandler(werkzeug)
        return self.flask

    def set_celery(self, **kwargs):
        return celery_app

    def set_db(self):
        db.init_app(self.flask)
        return db

    def set_migration(self):
        migrate.init_app(self.flask, db)
        return migrate

I can't get it to work with sqlite or postgres. I'm not sure if my directory structure is the problem or I'm missing a step. Some guidance would be highly appreciated.

Thanks.

[miguelgrinberg]

What are the contents of the file app/models/__init__.py?

[michaelbukachi]

These are the contents __all__ = ['helium', 'logs']

[miguelgrinberg]

Did you import helium and logs?

[michaelbukachi]

Just did, made the following changes in the app's __init__.py file:

# from . import models
    from .models import Log, FollowupMessagesModel, SMSMessages, MezzanineRequestLogModel, FarmerModel, LoanModel

Also the made the following changes in the models __init__.py file:

#__all__ = ['helium', 'logs']
from logs import Log
from helium import FollowupMessagesModel, SMSMessages, MezzanineRequestLogModel, FarmerModel, LoanModel

It didn't work at first. I commented out the following lines in env.py

        if getattr(config.cmd_opts, 'autogenerate', False):
            script = directives[0]
            if script.upgrade_ops.is_empty():
                directives[:] = []
                logger.info('No changes in schema detected.')

and run the migrations and it worked, I then uncommented the lines again and it worked. It's quite puzzling really, is there some kind of cache that happens, that we should be aware of? This problems tend to appear for a new dev as soon as they've cloned the project ad are setting up.

[miguelgrinberg]

Not sure, but I think I was right with my guess that your models weren't being imported when you run flask db commands. The changes that you made caused the models to be imported, which is required for Alembic to be able to find changes.

[michaelbukachi]

Okay. I think I need to do a bit more research on the occasional odd behaviour. Just to clarify one thing though, models should be imported after the instantiation of db and migrate, right?

[miguelgrinberg]

Your model classes inherit from db.Model, so you have to have db defined or imported before your models. Likewise, your Migrate instance needs to take db as an argument in the constructor. The Migrate instance and the models do not have any dependency between them, but it is common to have Migrate defined right after db, so to summarize, the usual order is db is defined/imported first, then your Migrate instance, and then your models.

Note that if you use a linter, it may flag the models as being unused imports. This is fine, importing the models is necessary anyway, as Alembic and SQLAlchemy use introspection to find the models among all the imported symbols. If the models are not imported, then Alembic will think you have no models in your database schema, which is what happened to you.

[michaelbukachi]

Thanks for the clarification.

[johndiego]

Hello @miguelgrinberg

I 'm understand , how i can make in factory app?
your have examples?

[miguelgrinberg]

@johndiego see https://github.com/miguelgrinberg/flasky for an example.

[johndiego]

Thanks @michaelbukachi !!
I'm new in flask!

[johndiego]

@miguelgrinberg I'm try create new project , but don't work the flask migrate!
For three days I've been trying and unsuccessful, you can help-me please!!
my project is https://github.com/johndiego/project
but dont apply migrations with mysql

/app # flask db upgrade
/usr/local/lib/python3.7/site-packages/flask_sqlalchemy/__init__.py:774: UserWarning: Neither SQLALCHEMY_DATABASE_URI nor SQLALCHEMY_BINDS is set. Defaulting SQLALCHEMY_DATABASE_URI to "sqlite:///:memory:".
  'Neither SQLALCHEMY_DATABASE_URI nor SQLALCHEMY_BINDS is set. '
/usr/local/lib/python3.7/site-packages/flask_sqlalchemy/__init__.py:794: FSADeprecationWarning: SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and will be disabled by default in thefuture.  Set it to True or False to suppress this warning.
  'SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and '
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade  -> 0f215e8ced34, empty message
/app # flask db migrate
/usr/local/lib/python3.7/site-packages/flask_sqlalchemy/__init__.py:774: UserWarning: Neither SQLALCHEMY_DATABASE_URI nor SQLALCHEMY_BINDS is set. Defaulting SQLALCHEMY_DATABASE_URI to "sqlite:///:memory:".
  'Neither SQLALCHEMY_DATABASE_URI nor SQLALCHEMY_BINDS is set. '
/usr/local/lib/python3.7/site-packages/flask_sqlalchemy/__init__.py:794: FSADeprecationWarning: SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and will be disabled by default in thefuture.  Set it to True or False to suppress this warning.
  'SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and '
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
Traceback (most recent call last):
  File "/usr/local/bin/flask", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/flask/cli.py", line 894, in main
    cli.main(args=args, prog_name=name)
  File "/usr/local/lib/python3.7/site-packages/flask/cli.py", line 557, in main
    return super(FlaskGroup, self).main(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/flask/cli.py", line 412, in decorator
    return __ctx.invoke(f, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/flask_migrate/cli.py", line 90, in migrate
    rev_id, x_arg)
  File "/usr/local/lib/python3.7/site-packages/flask_migrate/__init__.py", line 197, in migrate
    version_path=version_path, rev_id=rev_id)
  File "/usr/local/lib/python3.7/site-packages/alembic/command.py", line 176, in revision
    script_directory.run_env()
  File "/usr/local/lib/python3.7/site-packages/alembic/script/base.py", line 427, in run_env
    util.load_python_file(self.dir, 'env.py')
  File "/usr/local/lib/python3.7/site-packages/alembic/util/pyfiles.py", line 81, in load_python_file
    module = load_module_py(module_id, path)
  File "/usr/local/lib/python3.7/site-packages/alembic/util/compat.py", line 83, in load_module_py
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "migrations/env.py", line 87, in <module>
    run_migrations_online()
  File "migrations/env.py", line 80, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/usr/local/lib/python3.7/site-packages/alembic/runtime/environment.py", line 836, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/usr/local/lib/python3.7/site-packages/alembic/runtime/migration.py", line 321, in run_migrations
    for step in self._migrations_fn(heads, self):
  File "/usr/local/lib/python3.7/site-packages/alembic/command.py", line 156, in retrieve_migrations
    revision_context.run_autogenerate(rev, context)
  File "/usr/local/lib/python3.7/site-packages/alembic/autogenerate/api.py", line 415, in run_autogenerate
    self._run_environment(rev, migration_context, True)
  File "/usr/local/lib/python3.7/site-packages/alembic/autogenerate/api.py", line 427, in _run_environment
    raise util.CommandError("Target database is not up to date.")
alembic.util.exc.CommandError: Target database is not up to date.
/app #

I used two commands
"flask db init"
"flask db migrate"
flask migrate do not apply migrations

[miguelgrinberg]

@johndiego The "target is not up to date" error means that you have migrations that haven't been applied to the database. You need to look in your migrations/versions directory to see if you have any extra migrations that should not be there, or else you may need to run "flask db upgrade" to apply any migrations that are part of your project.

[4vadim4]

@michaelbukachi , i had same trouble and i found solve for me in alembic env.py file - pay attention for this lines:
_# add your model's MetaData object here_
_# for 'autogenerate' support_
_# from myapp import mymodel_
_# target_metadata = mymodel.Base.metadata_
i imported my model class redefined _target_metadata_ , see below:
from src.models.UserModel import UserModel
target_metadata = UserModel.metadata
documentation: https://alembic.sqlalchemy.org/en/latest/autogenerate.html

[michaelbukachi]

@4vadim4 I was able to fix the issue. It seems my models weren't being imported properly.

[5hraddha]

@michaelbukachi , i had same trouble and i found solve for me in alembic env.py file - pay attention for this lines:
_# add your model's MetaData object here_
_# for 'autogenerate' support_
_# from myapp import mymodel_
_# target_metadata = mymodel.Base.metadata_
i imported my model class redefined _target_metadata_ , see below:
from src.models.UserModel import UserModel
target_metadata = UserModel.metadata
documentation: https://alembic.sqlalchemy.org/en/latest/autogenerate.html

Thanks..!! This worked for me too..!! Now, flask db migrate is working fine and generating migrations..

[miguelgrinberg]

@5hraddha @michaelbukachi all you are doing is explicitly import your models in the Alembic config file. This would not be necessary if the models were imported somewhere in your app, which was my diagnosis when this issue started:

Not sure, but I think I was right with my guess that your models weren't being imported when you run flask db commands

Given that you guys are now functional I'm going to close this issue. Please reopen if there are still problems.

[pranaykhilari]

Hi Miguel,

I am using flask-migrate library for migration. I have created models in models package not in my manage.py file. When I tried to migrate the model using

python3 manage.py db migrate
_INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
INFO [alembic.env] No changes in schema detected._

is my output. It means it is not detecting the models which I define in model folder.

This is my manage.py content

_from flask_migrate import Migrate, MigrateCommand
from flask_script import Manager

from app import app, db

migrate = Migrate(app, db)
manager = Manager(app)
manager.add_command('db', MigrateCommand)

if __name__ == '__main__':
manager.run()_

This is my model file content

_from manage import db

class Employee(db.Model):
__tablename__ = 'Employee'

id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(128))
email = db.Column(db.String(128))_

Please help me.

[bqube]

Hi there, I am unable to figure out why my migration files not generating for flask multidb. Since the last two days I have tried, and gone through all the comments, but unable to resolve. Can you help me with this, please!!

│   ├── module1
│   │   ├── generators
│   │   ├── helpers
│   │   ├── __init__.py
│   │   ├── resources
│   │   ├── schemas
│   │   └── services
│   ├── __init__.py
│   └── module2
│       ├── generators
│       ├── helpers
│       ├── __init__.py
│       ├── resources
│       ├── schemas
│       └── services
├── automation.py
├── config.py
├── database
│   ├── data_mart
│   ├── data_warehouse
│   │   ├── base.py
│   │   ├── fact_crawl_data.py
│   ├── __init__.py

this is the config.py

import os
from dotenv import load_dotenv

basedir = os.path.abspath(os.path.dirname(__file__))
load_dotenv(os.path.join(basedir, '.env'))


class Config(object):
    # Database
    SQLALCHEMY_DATABASE_URI = f"{os.environ.get('DATA_MART_DRIVER')}" \
                              f"://{os.environ.get('DATA_MART_USER_NAME')}" \
                              f":{os.environ.get('DATA_MART_PASSWORD')}" \
                              f"@{os.environ.get('DATA_MART_HOST')}" \
                              f"/{os.environ.get('DATA_MART_DB_NAME')}"
    SQLALCHEMY_BINDS = {
        'db_dw': f"{os.environ.get('WAREHOUSE_DRIVER')}://{os.environ.get('WAREHOUSE_USER_NAME')}"
                 f":{os.environ.get('WAREHOUSE_PASSWORD')}"
                 f"@{os.environ.get('WAREHOUSE_HOST')}/{os.environ.get('WAREHOUSE_DB_NAME')}",
    }
    SQLALCHEMY_TRACK_MODIFICATIONS = False
    SQLALCHEMY_RECORD_QUERIES = True

    # Mail
    MAIL_SERVER = os.environ.get('MAIL_SERVER')
    MAIL_PORT = os.environ.get('MAIL_PORT')
    MAIL_USERNAME = os.environ.get('MAIL_USERNAME')
    MAIL_PASSWORD = os.environ.get('MAIL_PASSWORD')
    MAIL_USE_TLS = os.environ.get('MAIL_USE_TLS')
    MAIL_USE_SSL = os.environ.get('MAIL_USE_SSL')
    MAIL_SUPPRESS_SEND = os.environ.get('MAIL_SUPPRESS_SEND')

    @staticmethod
    def init_app(app):
        pass

this is my app/__init__.py

import logging
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate
from flask_mail import Mail
from config import config

db = SQLAlchemy()
migrate = Migrate()
mail = Mail()

def create_app(config_name):
    app = Flask(__name__)
    app.config.from_object(config[config_name])
    db.init_app(app)
    migrate.init_app(app, db)
    mail.init_app(app)
    app.logger.setLevel(logging.INFO)
    app.logger.info('App started')
    return app


from database.data_warehouse.base import Base
from database.data_warehouse.fact_crawl_data import FactCrawlData

this is base.py

from sqlalchemy.ext.declarative import declared_attr, as_declarative
from sqlalchemy import DDL, Column, Integer, DateTime
from sqlalchemy.sql import func

updated_at = DDL(
    "CREATE OR REPLACE FUNCTION update_changetimestamp_column()"
    "RETURNS TRIGGER AS $$"
    "BEGIN"
        "NEW.changetimestamp = now();"
        "RETURN NEW;"
    "END;"
    "$$ language 'plpgsql';"
)


@as_declarative()
class Base(object):
    @declared_attr
    def __tablename__(cls):
        return cls.__name__.lower()

    id = Column(Integer, primary_key=True, autoincrement=True)
    created_at = Column(DateTime(timezone=True), default=func.now())
    updated_at = Column(DateTime(timezone=True), default=func.now(), onupdate=func.now())

this is fact_crawl_data.py

from sqlalchemy import event
from app import db
from database.data_warehouse.base import Base, updated_at


class FactCrawlData(Base, db.Model):
    __bind_key__ = 'db_dw'
    __tablename__ = 'dw_fact_crawl_data'
    crawl_time = db.Column(db.DateTime)
    crawl_data_json = db.Column(db.JSON)
    product_id = db.Column(db.Integer)
    etailer_id = db.Column(db.Integer)

Now while running flask db migrate it is giving

[2020-06-23 21:20:35,992] INFO in __init__: Automation startup
INFO  [alembic.env] Migrating database <default>
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.env] Migrating database db_dw
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.env] No changes in schema detected.

Please help me to debug my code. If I am missing something, please tell me or my structure alteration is doing any problem, please let me know. I would be highly appreciated to get any suggestions.

Thanks,

[miguelgrinberg]

"No changes in schema detected" means that your models and the database have the same structure. Two possible reasons:

• you upgraded the database on your own sidestepping Flask-Migrate
• you are not importing your models in your application

The 2nd one has some variants. I've seen people create two database objects, and the models are imported into one, but Flask-Migrate is configured with the other. I thought I'd mention this, seeing that you are using a very strange mix of Flask-SQLAlchemy with regular SQLAlchemy.

[xiayulu]

$ python -m manage db migrate
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.

When I changed something in my models, migrate didn't work.
I solved this problem by remove some files like 91c7e9304377_.py away from the versions folder.
Then I run $ python -m manage db migrate, it worked.
I hope this may help you.