Pip-tools: Add config file

[merwok]

I end up writing wrapper scripts to pass options (like no-header) and support multiple .in files (#532).

If pip-compile knew to find its options in .pip-tools.cfg or tox.ini (many Python development tools such as pytest, flake8, coverage can find their config in tox.ini, even when they’re not run inside tox), I would put the options there and tell my co-workers they can simply run pip-compile file.in.

Idea taken from pip-tools’ fork prequ.

[vphilippon]

I would suggest to go with a pip-tools.cfg file or something alike, rather than tox.ini. I feel like options in tox.ini should be about testing and such.

Nice suggestion 👍

[suutari-ai]

How about a [pip-tools] section in setup.cfg? Most Python tools can read their config from setup.cfg (e.g. pytest, flake8, isort, etc.) and IMHO it's nicer to have fewer files in the project root.

[vphilippon]

I haven't looked much at the setup.cfg file. If it's meant to be extensible for other tools like that, then that sounds more reasonable to me.

[merwok]

It’s not meant to (the section names are made to match setup.py commands), but some tools look there too.

A difference is that a tox.ini section like pytest needs to be tool:pytest in setup.cfg.

[johnthagen]

Perhaps the standardized pyproject.toml should be used?

[tool.pip-tools]
# ...

[elcolie]

Any progress on this so far?
Since I am using base.txt, local.txt, and production.txt in my repository. pip-compile is amazing
base.txt

pytz==2020.1  # https://github.com/stub42/pytz
python-slugify==4.0.1  # https://github.com/un33k/python-slugify
Pillow==7.2.0  # https://github.com/python-pillow/Pillow
argon2-cffi==20.1.0  # https://github.com/hynek/argon2_cffi
redis==3.5.3  # https://github.com/andymccurdy/redis-py
hiredis==1.1.0  # https://github.com/redis/hiredis-py
celery==4.4.7  # pyup: < 5.0  # https://github.com/celery/celery
django-celery-beat==2.0.0  # https://github.com/celery/django-celery-beat
flower==0.9.5  # https://github.com/mher/flower
uvicorn==0.11.8  # https://github.com/encode/uvicorn
django-extensions==3.0.7
arrow==0.16.0
tqdm==4.48.2
django-filter==2.3.0

# Django
# ------------------------------------------------------------------------------
django==3.1.1  # pyup: < 3.1  # https://www.djangoproject.com/
django-environ==0.4.5  # https://github.com/joke2k/django-environ
django-model-utils==4.0.0  # https://github.com/jazzband/django-model-utils
django-allauth==0.42.0  # https://github.com/pennersr/django-allauth
django-crispy-forms==1.9.2  # https://github.com/django-crispy-forms/django-crispy-forms
django-redis==4.12.1  # https://github.com/jazzband/django-redis
# Django REST Framework
djangorestframework==3.11.1  # https://github.com/encode/django-rest-framework
django-cors-headers==3.5.0 # https://github.com/adamchainz/django-cors-headers

local.txt

-r base.txt

Werkzeug==1.0.1 # https://github.com/pallets/werkzeug
ipdb==0.13.3  # https://github.com/gotcha/ipdb
psycopg2==2.8.5 --no-binary psycopg2  # https://github.com/psycopg/psycopg2
watchgod==0.6  # https://github.com/samuelcolvin/watchgod

# Testing
# ------------------------------------------------------------------------------
mypy==0.770  # https://github.com/python/mypy
django-stubs==1.5.0  # https://github.com/typeddjango/django-stubs
pytest==6.0.1  # https://github.com/pytest-dev/pytest
pytest-sugar==0.9.4  # https://github.com/Frozenball/pytest-sugar

# Documentation
# ------------------------------------------------------------------------------
sphinx==3.2.1  # https://github.com/sphinx-doc/sphinx
sphinx-autobuild==2020.9.1 # https://github.com/GaretJax/sphinx-autobuild

# Code quality
# ------------------------------------------------------------------------------
flake8==3.8.3  # https://github.com/PyCQA/flake8
flake8-isort==4.0.0  # https://github.com/gforcada/flake8-isort
coverage==5.2.1  # https://github.com/nedbat/coveragepy
black==20.8b1  # https://github.com/ambv/black
pylint-django==2.3.0  # https://github.com/PyCQA/pylint-django
pylint-celery==0.3  # https://github.com/PyCQA/pylint-celery
pre-commit==2.7.1  # https://github.com/pre-commit/pre-commit

# Django
# ------------------------------------------------------------------------------
factory-boy==3.0.1  # https://github.com/FactoryBoy/factory_boy

django-debug-toolbar==2.2  # https://github.com/jazzband/django-debug-toolbar
django-extensions==3.0.7  # https://github.com/django-extensions/django-extensions
django-coverage-plugin==1.8.0  # https://github.com/nedbat/django_coverage_plugin
pytest-django==3.9.0  # https://github.com/pytest-dev/pytest-django

production.txt

# PRECAUTION: avoid production dependencies that aren't in development

-r base.txt

gunicorn==20.0.4  # https://github.com/benoitc/gunicorn
psycopg2==2.8.5 --no-binary psycopg2  # https://github.com/psycopg/psycopg2
Collectfast==2.2.0  # https://github.com/antonagestam/collectfast
sentry-sdk==0.17.3  # https://github.com/getsentry/sentry-python

# Django
# ------------------------------------------------------------------------------
django-storages[boto3]==1.10  # https://github.com/jschneier/django-storages
django-anymail[mailgun]==7.2.1  # https://github.com/anymail/django-anymail

[jcushman]

If anyone wants to take this on, I think a really good option would be to copy the code from the black package, which reads config options from pyproject.toml and uses the click library to parse arguments the same as pip-tools. It's just this click setting and adapting the read_pyproject_toml function and its support functions to pip-tools:

@click.option(
    "--config",
    type=click.Path(
        exists=True,
        file_okay=True,
        dir_okay=False,
        readable=True,
        allow_dash=False,
        path_type=str,
    ),
    is_eager=True,
    callback=read_pyproject_toml,
    help="Read configuration from FILE path.",
)

(I would remove the find_user_pyproject_toml() stuff from black's implementation, since I think having a user-level settings file is less useful for pip-tools than for black.)