Alembic: add virtualenv basics to the tutorial

[darthmall]

I set up Alembic per the instructions in the tutorial, but when I run alembic revision --autogenerate I get a ModuleNotFoundError when my base MetaData object is imported into env.py.

My project is structured like this:

myproj
    alembic.ini
    alembic/
        env.py
        versions/
    myapp/
        __init__.py
        db/
            __init__.py
            models.py

In alembic/env.py I have set up the target_metadata like this:

from myapp.db.models import Base
target_metadata = Base.metadata

If I run alembic revision -m "Add a table", it works fine, but if I run alembic revision --autogenerate -m "Add a table" I get ModuleNotFoundError: No module named 'myapp'.

Environment

Python

3.6.9

Alembic

1.3.1

[zzzeek]

hi there -

your project has to be in PYTHONPATH or otherwise importable. the most common way to do this is to create a virtualenv, install your project into the virtualenv using "/path/to/virtualenv/bin/python setup.py -e ." then also install alembic into that virtualenv. that way everyone is in the same environment so when you call upon /path/to/virtualenv/bin/alembic, it will also see your project in the same environment.

https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/

[darthmall]

I see. Did I miss that in the docs somewhere? I was pretty sure I followed the tutorial exactly. If it's not in the docs, it seems like something worth mentioning.

Thanks for the help.

[zzzeek]

it's not in the docs, no. alembic's original target audience were those for whom tools like these were second nature. May I ask what your Python background is? because adjusting the docs for a new target audience means I'd need to understand that audience. Also, if i write in the docs to use pip and virtualenv, there's already controversy there, as there are other tools like "Poetry" and such that are used instead and I really don't want to endorse any specific development method.

[zzzeek]

it's in the docs now. please review the documentation at https://alembic.sqlalchemy.org/en/latest/front.html#installation and report any typos or lack of clarity to this issue. thanks!

[darthmall]

alembic's original target audience were those for whom tools like these were second nature. May I ask what your Python background is? because adjusting the docs for a new target audience means I'd need to understand that audience.

I've been working with Python for a little over 15 years now. So I did understand that the module wasn't on my PYTHONPATH. I didn't expect that Alembic was set up so that when it runs it wouldn't be able to find the modules in the project without having them installed in my site-packages. My expectation was that Alembic would set itself up inside my project directory in such a way that when it executed my modules would be resolvable. I've worked with Django's ORM a lot more than I've worked with SQLAlchemy, and it works without installing anything into site-packages.

The other thing that threw me a little bit was that I had followed the tutorial, I thought, pretty closely. So I didn't expect to hit any errors unless I'd either messed something up or there was a bug. After digging around online for a bit without finding anything, I figured: bug. 😁 Once I knew that it wasn't a kludge to install the project so that it could be imported from anywhere, I did that and got it working right away. 🙏

So that's why I was suggesting having a note in the docs would be useful (thanks for adding it) just to point out that because of where env.py is put in the project structure you won't be able to import any of the modules from the project unless you either alter your PYTHONPATH or install your project in the PYTHONPATH with pip install -e ..

Also, if i write in the docs to use pip and virtualenv, there's already controversy there, as there are other tools like "Poetry" and such that are used instead and I really don't want to endorse any specific development method.

I don't think you need to endorse any specific development method. Just point out that it's expected behavior that the files in the alembic/ directory, when executed, won't be able to find your other modules; that it's up to the developer to find a solution to that. Although having at least one option for solving the problem (even if the developer isn't using pip and virtualenv) is helpful to point people in the right direction.

Anyway, thanks again. I appreciate how responsive you've been. 🙏

[zzzeek]

the tutorial does not cover "autogenerate", so to that extent, you don't actually need your local project modules to be importable if you're on just tiutorial.rst, what error did you get?

[darthmall]

Right, sorry, I wasn't being clear. I continued on from the tutorial to the next section, which was auto-generating migrations, followed those instructions, and hit the ModuleNotFoundError. The only reason it threw me was that I expected alembic init to configure Alembic so that it could find my SQLAlchemy classes because that's how the other ORMs I've used in the past work. That's all.

[zzzeek]

autogen used to be in the tutorial so i might want to adjust the docs I just added to match this more closely