Renovate: Support python

[rarkins]

As suggested by @khornberg and @hbetts in #121

This issue is for building up (a) support (please +1 if you would like to see python support) and (b) requirements, e.g. helping suggest what scope is required and what APIs can be used.

[rarkins]

Files

File names: requirements.txt
Can be multiple in same repo: yes

Registry

PyPi

Example URL: https://pypi.python.org/pypi/Versio/json

Versioning

Similar to semver, defined in https://www.python.org/dev/peps/pep-0440/

[rarkins]

Either write python version logic in JS, or call external python script, e.g. https://stackoverflow.com/questions/23450534/how-to-call-python-function-from-nodejs
Example: https://ourcodeworld.com/articles/read/286/how-to-execute-a-python-script-and-retrieve-output-data-and-errors-in-node-js

The regex is mega: https://stackoverflow.com/questions/37972029/regex-to-match-pep440-compliant-version-strings

[khornberg]

Since suggesting python, I have found https://github.com/pyupio/pyup
I have found it to be easy to work with and seems to be basically the same except targeting Python.

[rarkins]

Some discussions relevant to the python requirements process:
https://news.ycombinator.com/item?id=11210370
https://www.reddit.com/r/Python/comments/3uzl2a/setuppy_requirementstxt_or_a_combination/

[mxey]

The officially recommended tool for Python is now Pipenv, which uses Pipfile and Pipfile.lock: https://docs.pipenv.org/basics/#example-pipfile-pipfile-lock

[hutson]

For context, I found pipenv used as the official tool in the official Python documentation - https://packaging.python.org/tutorials/managing-dependencies/

[keimlink]

Python packaging tools and requirement files that should be supported:

• conda - used by many scientific projects
  
  
  
  • environment.yml
  
  
• pip - Python packaging tool currently used by the majority of projects
  
  
  
  • requirements.txt (could be also called requirements.pip, some projects use several files for production, testing and development requirements)
  
  
  • setup.cfg - see setuptools documentation
  
  
  • tox.ini - used by tox
  
  
• pipenv - The latest Python packaging tool, more projects start using it
  
  
  
  • Pipfile
  
  
  • Pipfile.lock
  
  

[rarkins]

We've added basic requirements.txt support just now - it works on pinned versions (e.g. v1.2.3) only. More advanced support like npm, as well as other files/managers will be added later. To enable or test this initial support:

  "pip_requirements": {
    "enabled": true
  }

[rarkins]

Next step is to implement (port) pep440 to JS so that we can use its validation and comparison functions instead of semver.

We may do this in a stand-alone repo to avoid any licensing concerns with related python implementations.

[khornberg]

Thanks @rarkins

[rarkins]

Useful article on pipenv: https://buff.ly/2KcoT7S

[rarkins]

Python support has now been expanded to support all stable "equals" values (e.g. ==x, ==x.y, ==x.y.z) plus all related core Renovate capabilities (e.g. major/minor separation, etc).

It remains disabled by default, so you need to manually enable it as mentioned previously.

I have opened #2085 to specifically track support for advanced PEP440 specifiers.

[dserodio]

Thank you for your work @rarkins. It seems that the basic support for Python is already done, but it isn't mentioned in the documentation. Are you waiting for some milestone to be reached, or are you just waiting for a volunteer to document it? :)

[ghost]

Are you waiting for some milestone to be reached, or are you just waiting for a volunteer to document it? :)

I'm not @rarkins, but I'd love to see documentation for Renovate's Python support. We've already started to use it in our organization, and I don't personally have any concerns with advertising it's usage to our wider community through dedicated, top-level, Renovate documentation.

[rarkins]

I left this issue open because the list of requirements in it was only partially complete. However it is true that pip requirements.txt support is complete (including ranges) and available. I think what we should do is:

• [x] Break further functionality described here into new/separate issues
• [x] Add documentation for Python
• [x] Flip the switch to make it enabled by default
• [x] Mark this issue as resolved

@dserodio I wasn't explicitly waiting for a volunteer but you are very welcome to jump in :)

[rarkins]

I've now added a few new issues to track remaining functionality for python. All have the new #python hashtag

[rarkins]

Python support is now enabled by default, so closing this issue. Thanks for everyone's help and suggestions