Poetry: Invalid file path when running commands in poetry shell

[vitaliy-diachkov]

• [ ] I am on the latest Poetry version.
• [x] I have searched the issues of this repo and believe that this is not a duplicate.

• [ ] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Ubuntu 20.04

• Poetry version: 1.0.5
• Link of a Gist with the contents of your pyproject.toml file:

[tool.poetry]
name = "events-api"
version = "0.1.0"
description = ""
authors = ["vitaliy-diachkov <[email protected]>"]

[tool.poetry.dependencies]
python = "^3.8"
flask = "^1.1.2"
sqlalchemy = "^1.3.18"
gunicorn = "^20.0.4"
pg8000 = "^1.16.3"
python-dateutil = "^2.8.1"
psycopg2-binary = "^2.8.5"
pydantic = "^1.6.1"
google-cloud-secret-manager = "^1.0.0"
o365 = "^2.0.10"
google-api-python-client = "^1.10.0"
rfc3339 = "^6.2"
google-auth-oauthlib = "^0.4.1"
flask-swagger-ui = "^3.25.0"
pyyaml = "^5.3.1"
pytz = "^2020.1"
pytest = "^6.0.1"
python-dotenv = "^0.14.0"

[tool.poetry.dev-dependencies]
ptpython = "^3.0.5"

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

Issue

Hi, I have a project structure like this

.
|__ pyproject.toml
|__ poetry.lock
|__ src/
      |__ main.py

The issue is, when I'm running

poetry shell

cd src

python3 main.py

I got this error:

 * Serving Flask app "main" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: on
 * Running on http://0.0.0.0:8080/ (Press CTRL+C to quit)
 * Restarting with stat
/home/vitaliy/.cache/pypoetry/virtualenvs/events-api-QpbODKYT-py3.8/bin/python3: can't open file '.../events-api/main.py': [Errno 2] No such file or directory

So, as you see it is starting the Flask app first and then I got an error because it is trying to open events-api/main.py instead of events-api/src/main.py. What could be the reason of such behavior?

Everything works fine when I'm in project root directory and starts the flask app using python3 src/main.py.

[finswimmer]

Hello @vitaliy-diachkov,

hard to say what's going on exactly without having a look on the source code. But I have to two things you should check/change:

1. If you use the src layout, there must be a subfolder with the package name in src. And in this subfolder you put your code:

.
├── poetry.lock
├── pyproject.toml
└── src
    └── events_api
        └── main.py

1. Run poetry install to make sure your project gets installed in editable mode.

fin swimmer

[vitaliy-diachkov]

In main.py I have something like this

import flask


app = flask.Flask(__name__)


if __name__ == '__main__':
      app.run(host='0.0.0.0', port=8080, debug=True)

I don't pretend to use events_api as a library, that is just my web app code inside src folder. Why do I need to create a subfolder then?

[sinoroc]

@vitaliy-diachkov
Maybe you want to set your pyproject.toml like this:

[tool.poetry]
packages = [
    { include = 'main.py', from 'src' },
    # ...
]
# ...

_[Edited: I had forgotten the .py suffix, since it is a single module and not a package]_

This is an unconventional setup though. That is why _poetry_ requires more information in the pyproject.toml about the structure of the project.

Then you will want to call your code like so:

python3 -m main

_[Have not tested it, but it should work.]_

---

Why do I need to create a subfolder then?

Because of conventions...

If you have a project Something, then the convention dictates that there should be 1 and only 1 top level package (or module) called something (or something.py). This convention is important if you intend to publish the project on _PyPI_ for example. If the project is intended for private internal use only, then set your own conventions and feel free to name the top level modules and packages however you want.

In your current setup main.py is a top level module. The convention would thus assume that the project is named main, but there is already a project main on _PyPI_, so it would clash.

In @finswimmer's suggestion, theevents_api in src/events_api/main.py would be the top level package. And that would conform to the convention since you project seems to be named events_api (which by chance is currently not on _PyPI_, in case you want to publish it there).

---

Reference:

• https://python-poetry.org/docs/pyproject/#packages

[vitaliy-diachkov]

As I mentioned it wouldn't be a PyPi package or any library, it's just a simple Flask project and I was asking why poetry disallowes my to execute python3 main.py when I'm in src directory, but everything works fine when I'm running python3 src/main.py from project root directory.

[sinoroc]

@vitaliy-diachkov I will try to recreate the issue. How does the pyproject.toml file looks like?

There is a thing that bothers me in your code, it is your usage of __name__. I am not familiar with Flask, but the code seems strange to me. If __name__ == '__main__' is indeed true, which should be the case when calling python3 main.py as you claim to be doing, then I do not understand how flask.Flask(__name__) could possibly work... But I do not know enough about flask.

[vitaliy-diachkov]

The content is on details in thread

[tool.poetry]
name = "events-api"
version = "0.1.0"
description = ""
authors = ["vitaliy-diachkov <[email protected]>"]

[tool.poetry.dependencies]
python = "^3.8"
flask = "^1.1.2"
sqlalchemy = "^1.3.18"
gunicorn = "^20.0.4"
pg8000 = "^1.16.3"
python-dateutil = "^2.8.1"
psycopg2-binary = "^2.8.5"
pydantic = "^1.6.1"
google-cloud-secret-manager = "^1.0.0"
o365 = "^2.0.10"
google-api-python-client = "^1.10.0"
rfc3339 = "^6.2"
google-auth-oauthlib = "^0.4.1"
flask-swagger-ui = "^3.25.0"
pyyaml = "^5.3.1"
pytz = "^2020.1"
pytest = "^6.0.1"
python-dotenv = "^0.14.0"

[tool.poetry.dev-dependencies]
ptpython = "^3.0.5"

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

[vitaliy-diachkov]

I can create an example repo tomorrow (with a hello world Flask application), so you can test this on your own.

[sinoroc]

This is what I have come up with:

Directory structure:

events-api
├── pyproject.toml
└── src
    ├── __init__.py
    └── main.py

pyproject.toml:

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

[tool.poetry]
name = "events-api"
version = "0.1.0"
authors = []
description = ""

# seems like `packages` can be omitted if `poetry build` is not used
packages = [
    { include = "main.py", from = "src" },
]

[tool.poetry.dependencies]
python = "^3.8"
flask = "*"

src/main.py

import flask

app = flask.Flask(__name__)

if __name__ == '__main__':
      app.run(host='0.0.0.0', port=8080, debug=True)

After installing with poetry install, it can be used with:

python3 -m main

---

I was not able to recreate the original error message.

Even entering the src directory and calling the script directly seems to work:

.../events-api$ cd src
.../events-api/src$ python3 main.py

As well as:

.../events-api$ python3 src/main.py

[vitaliy-diachkov]

I've found that I only have issues when I'm using python-dotenv package, so probably not a poetry issue

Here is the example: https://github.com/vitaliy-diachkov/flask-poetry-dotenv-issue-example

[sinoroc]

@vitaliy-diachkov Would you mind closing the issue, then? It will help the maintainers keep a better overview of what issues require attention. Feel free to add any new info and/or reopen the ticket any time.

[vitaliy-diachkov]

Yes, hope will resolve this issue at python-dotenv's repo. Thanks for assistance.