Setuptools: Helper function to read __version__

[timhoffm]

Maintaining a consistent version number across different parts of a package is a burden for authors. It may be needed in different places such as the package/module itself, setup.py or a sphinx conf.py.

The packaging guide describes a total of 7 different approaches.

I propose to support at least a few of them with a new get_version() function in setuptools.

~~~
def get_version(filename, rel_to_filname=None, method='regexp'):
"""
Read the content of the variable __version__ from a file.

Arguments:
    filename: 
        The file containing __version__
    rel_to_filename:
        If given, filname will be interpreted to be relative to
        the directory of this file. Otherwise, it will be relative
        to the working directory.
        In most cases, you will want to pass __file__.
    method:
        If 'regexp', the value of __version__ will be read determined
        via a regular expression. This requires, that it's a string.
        If 'exec', the file will be executed and the local variable
        __version__ will be read out. This if __version__ is defined
        in a more complex way. However, the disadvatage is that the
        file will be executed, including possible side-effects. 

Returns:
    The __version__ value or None, if it could not be determined.

~~~

The basic implementation should support methods 1 (parse the file for a regexp) and 3 (execute the file in a separate context). IMO, these are the best approaches unless you have or need further functionality like extra tools (2) or VCS support (7).

The idea is to have __version__ set in your source code and fetch that value in helper scripts like setup.py or conf.py with a single line of code.

Example uses:

• get_version('mypackage.__init__.py', __file__)
• get_version('../mypackage._version.py, __file__, method='exec')

I figure this is a reasonable functionality to provide for setuptools. If there is interest, I can provide a PR.

[pganssle]

There seems like a lot of overlap between that and pkg_resources.parse_version.

[timhoffm]

As far as I understand, pkg_resources.parse_version is just a parser str -> Version.

I'm not intending to do this in the proposed function. It's really just getting the value of the __version__ variable from the source code. So I don't see any overlap.

[pganssle]

Oh I see what you mean, you want to get the version from the source without importing the module.

I think probably it's not a great idea. The 7 different mechanisms you describe for single-sourcing a version all generate a __version__ within the module. Anything else that needs the version can import the module and retrieve modulename.__version__, or call python -c 'from mymodule import __version__; print(__version__), and that should solve the problem in almost all cases.

[timhoffm]

The 7 different mechanisms you describe for single-sourcing a version all generate a __version__ within the module.

I beg to differ. This is only true for some of the approaches. The mechanisms hold the original version information in different places:

• __version__ in module / package (1, 3, 6)
• VERSION text file (4)
• setup.py (5)
• somewhere else (2, 7)

Each time, the purpose is to distribute the information to __version__ and setup.py.

Anything else that needs the version can import the module and retrieve modulename.__version__, or call python -c 'from mymodule import __version__; print(__version__), and that should solve the problem in almost all cases.

That's basically solution 6, and not necessarily a good solution (see the comment there). A major point of the proposed function is that you don't have to import the package to get the version information.

[timhoffm]

Motivation

Maybe it's helpful to state my motivation in a bit more detail.

I want to:

• Define the version only in one place
• Distribute it to __version__, setup.py and sphinx conf.py
• Don't require to import the module in setup.py and sphinx conf.py.

To me defining the version in the source code (1, 3) seem to be the best solution for this unless you need/want extra tools (2, 7). 4 and 5 don't have an advantage compared to 1, 3 but the added difficulty that I have to somehow set __version__ in the source code. 6 does not qualify because of the import.

The proposed function helps to make the value of __version__ available in other scripts like setup.py and conf.py without the need to write extra boilerplate code.

[benoit-pierre]

Note you can already use something like version = attr: src.VERSION in setup.cfg (see documentation).

[pganssle]

That's basically solution 6, and not necessarily a good solution (see the comment there). A major point of the proposed function is that you don't have to import the package to get the version information.

No, the seven solutions all lead to the same outcome - that import mymodule.__version__ contains the version information. Regardless of where in the repository that information is kept, you can rely on this fact. In the vast majority of cases where you'd be able to use this function, it is not a problem to import the module.

• Define the version only in one place
• Distribute it to __version__, setup.py and sphinx conf.py
• Don't require to import the module in setup.py and sphinx conf.py.

For setup.py, there are already 7 separate solutions in the document you linked, you are saying you want to add an eighth solution? I thought the point was to retrieve the version when it is stored in the repo in one of the other 7 ways.

As for using it in conf.py, I'm not sure why it's a problem to import the module. That tends to happen if you are using autodoc anyway and that's not a huge burden. The only reason to avoid importing the module in setup.py is that it creates a chicken-and-egg problem wherein you need the version to build the package but you need the package to get the version. You don't generally have that problem in conf.py.

[timhoffm]

Ok, drop conf.py. Import may work there.

For setup.py, there are already 7 separate solutions in the document you linked, you are saying you want to add an eighth solution?

No. The bullet points just describe the motivation. They can be achieved with some of the 7 solutions.

The function just streamlines these solutions so that one does not have to copy boilerplate code.

[pganssle]

I suppose. I personally don't think there's a huge amount of boilerplate code involved in a lot of these solutions, so I am -1 on creating and maintaining in setuptools or pkg_resources a new function for this purpose. It seems too complicated for very little payoff.

[timhoffm]

I accept you point of view but don't share it. To me, it's actually little effort for a good payoff.

When trying to find out how to do this correctly, I found a lot of users with the same question and a number of different answers (let alone the 7 in the guide). Apparently, this is something many people stumble over.

setuptools could really help package maintainers here by providing a simple standard solution.

I would be willing to provide a PR and I don't think it's much of a burden to maintain. Essentially I don't see why the function should need any future changes. It's really simple and self-contained.

Of course, the decision is up to you.

[pganssle]

setuptools could really help package maintainers here by providing a simple standard solution.

If there were a single standard solution there wouldn't be 7 items on this list. And it's really more than 7, because item 2 is "use a version manager that handles it for you". I personally think you get the best "value for money" from setuptools_scm (which has all the functionality you describe), but there are definite trade-offs.

You always have the option of creating a new package that does exactly what you say. If it's very useful presumably it could become the standard way of finding versions, but it would basically be competing with all the options in 2 and 7.

[timhoffm]

I've intentionally come to setuptools and not created a new package, because I think an additional dependency just for this function is not neccessary and it's actually quite the topic of setuptools. Additionally setuptools is basically available everywhere. setuptools_scm is certainly great, but an additional dependency nonetheless (not everybody wants that).

To me, the list could be more opinionated (For simple cases use A. If you additionally need feature x use B. Don't use C unless you really need feature y). The items just stand there side by side and you have to figure out yourself, when to use what. - If there is interest I could propose a more targeted rewrite.

Anyway thanks for the discussion. Keep up the great work!

[jaraco]

At one point, setuptools had built-in support for the most popular source code management system (Subversion). In fact, it was maintaining the relationship between SVN and Setuptools that drew me into contributing to the project in the first place.

I'm actually quite pleased with the direction things have been going, decoupling the functionality but making it increasingly convenient for a project to opt in. The latest triumph is the addition of PEP 518 support to Pip 10 (which is available in beta today, just pip install -U git+https://github.com/pypa/pip@release/10.0.0), which allows for a plugin like setuptools_scm to be readily included on demand and without any extra complication. And until that support is widely available, the setup_requires directive is a decent fallback.

Maybe it's helpful to state my motivation in a bit more detail.

I do share your motivations. And I should point out that if your motivation is to have the version in only one place and if you tag your source code with version numbers, that tag necessarily has to be the one place.

Regarding the Sphinx docs conf, I've found a solution with which I'm very happy. See the conf.py which relies on jaraco.packaging.sphinx to load the config (version, author, name, url) from the package metadata. Just make sure jaraco.packaging is in your docs requirements.

This approach allows the version to be specified exactly once, atomically, by tagging a commit. And the version number cascades from that tag (and even increments on intermediate commits). The main downside I face with this whole approach is the presentation of the version in packagename.__version__, and that only because import pkg_resources is slow (too slow for command-line clients). Otherwise, this approach is pretty robust and gets more robust with pip 10. I've been using setuptools_scm or something like it since 2010; I consider it a robust and recommended solution.

[jwodder]

Note you can already use something like version = attr: src.VERSION in setup.cfg

This, like other solutions, involves importing the module. Importing is generally not an option when the project has dependencies, as the dependencies will have to be installed first, which requires reading the metadata to determine the dependencies, which also involves reading the package's version, leading to a circular, um, dependency.

[pganssle]

This, like other solutions, involves importing the module. Importing is generally not an option when the project has dependencies, as the dependencies will have to be installed first, which requires reading the metadata to determine the dependencies, which also involves reading the package's version, leading to a circular, um, dependency.

I don't think this is generally true when all you need is to import __version__, though I suppose that might be an artifact of the way I've structured projects where even if I load dependencies, just doing from myproject import __version__ wouldn't import any of them.

That said, there's also #1359, which supports loading the version information directly from a file.

[jaraco]

The main downside I face with this whole approach is the presentation of the version in packagename.__version__.

I'm pleased to say this issue is addressed by importlib_metadata, slated to be importlib.metadata in Python 3.8. See path.py for an example of that being used to pull the version from package metadata into the module at run time.

So between the two options for defining a version for a project:

• derive version from metadata using setuptools_scm (my recommendation and preferred approach), or
• store the version in a file and use declarative config to load that value as importlib_metadata currently does.

You have straightforward ways to define the version in the package metadata, and then with importlib_metadata, you have elegant ways to present that version at run time.

[jwodder]

FYI to anyone still interested in a function that reads __version__ from source for use in setup.py: I've just released a package on PyPI for doing exactly that: https://pypi.org/project/read-version/

[smarie]

For what's worth you can now use getversion to get the version of any module or submodule along with an explanation of why this version was returned (which PEP/strategy was used)