Pipenv: Make setup.py optional.

[feluxe]

I started using Pipenv/Pipfile and I like it a lot. It's similar to npm/yarn/cargo etc. and I think that's cool, since I'm already used to those. But I have a hard time figuring out how Pipenv/Pipfile and setup.py play together.

In JS you have:

• npmjs.com package registry
• npm command line tool for installing/publishing packages etc.
• package.json to describe your package and manage dependencies.

In Rust it's pretty much the same

• crates.io package registry
• cargo command line tool for installing/publishing packages etc.
• Cargo.toml to describe your package and manage dependencies.

In Python:

• pypi.python.org package registry
• Pipenv command line tool for installing packages etc.
• setup.py to describe your package and dependencies. (also for publishing packages)
• Pipfile to describe dependencies and environment for pipenv.

I think it's confusing that you have similar values in both Pipfile and setup.py. In most projects I have seen setup.py acts as a pure config file. I think it would be more intuitive if all of it was configured in Pipfile, like in package.json, Cargo.toml... setup.py could be fully optional.

In Pipfile you could then use:

• [install_requires] to describe pypi package dependencies. (if needed)
• [packages] to describe dependencies for installation from source
• [dev-packages] to describe dependencies for dev-installation from source

If [install_requires] is set and [packages] is omitted, pipenv could use [install_requires] for the install command. That would solve the issue with syncing [packages] and install_requires a lot of people have.

What do you think? :)

[gsemet]

i use pbr for that. Don't know if pipenv should handle that

[techalchemy]

I see where you're coming from, and this issue comes up a lot, which probably why there is a fantastic article about it by Donald Stufft, creator of pip. Setup.py is not a configuration file. It is executed when you install a package. A Pipfile is a configuration file.

If you want to skip Donald's post, you can read this comment by one of our maintainers which articulates what the workflow should be. What you are proposing is fundamentally altering the way python works, which is definitely out of scope for pipenv. Thanks for the considered proposal though, this is definitely the kind of input we like to see :)

[feluxe]

Thanks for your responses!

@techalchemy I'm not saying the mechanism behind setup.py should change or disappear.

The usual setup.py is mostly config data for setuptools. My idea was that this config could be loaded from Pipfile similar to what @gsemet suggests with setup.cfg.

I think it would be awesome if Pipenv became the go-to interface for installing, building and uploading Python packages and Pipfile the single config of truth for package and dependency related config data. This could be build utilizing the nice standards/tools that are already there (setup.py, setuptools, wheels etc)... But that's just me dreaming on the couch ;)

[uranusjr]

@feluxe While I don’t necessarily strongly disagree with your points, this may not be the best venue to hold the discussion. I would suggest raising this issue on mailing lists like python-ideas to get more feedback.

Some background: Python has historically made a clear distinction between library and application dependencies (as mentioned several times previously). Pipfile has been developed to handle specifically the application part of the story, with Kenneth repeatedly emphasising Pipfile should be “the single source of truth” (this would never be truth if Pipfile is intended to use for library dependencies). There has been similar movements in lib-dep packaging to move away from old formats as well. The most significant one currently is the implementation of setup.cfg as an alternative metadata provider to setup.py. There has been proposals to furthur improve the format with the introduction of pyproject.toml.

There is still time to try unifying the formats. With PEP 390 rejected and 426 withdrawn, dependency specification is still not taken in the next-generation format, and Pipfile can still fill that role. But you’ll need to raise this issue in a more visible way to push the discussion toward the direction you want.

I don’t necessarily like the idea of mixing library and application dependency specifications, especially in Python, but that’s a whole other topic. I would very much love to see discussion around this issue, however, with more input from more knowledgable people, such as Nick and Donald.

[jtratner]

PBR (from openstack) currently handles parsing requirements.txt to generate the set of dependencies from your app. Once the pypa/pipenv repo gets fully baked, could be a good place to add support for pulling in dependencies from Pipfile (tho as I said on another ticket, I think you can get most of the way there by specifying pipenv install -e . and then using that pipfile)

[gsemet]

@jtratner I have started the patch a few months ago: https://review.openstack.org/#/c/524436/ I would be happy to use the new parser :)