Black: Offer option other than pyproject.toml for config

[altendky]

https://github.com/pypa/pip/issues/6163
https://github.com/pypa/setuptools/issues/1642

It seems that pyproject.toml is a trigger for enabling PEP 517 isolated builds. This seems a not good thing to tie to configuration of black. It broke pyinstaller and others with the pip-19.0 release.

[JelleZijlstra]

Sorry for the issues you saw! #688 suggests also supporting setup.cfg, which seems reasonable to me.

[altendky]

There's been a lot of follow up discussion https://github.com/pypa/setuptools/issues/1642 and https://github.com/pypa/pip/issues/6163. I followed in the beginning but can't really give a summary anymore, sorry. It might be worth reading up on where they are going with all this, or maybe it's not and just having another config-file option is appropriate.

I personally don't use black (yet?) so I was mostly just trying to point out the scenario for consideration.

[cjerdonek]

I believe the latest plan is to roll back pyproject.toml triggering PEP 517 builds in a new pip 19.0.x release and to try again in pip 19.1 once the implementation is more tested and known to be more stable.

[leplatrem]

As I wrote in https://github.com/ambv/black/issues/688#issuecomment-458995754, setup.cfg is a good candidate! It's used by setuptools (thus by pretty much everyone), it's been there for years, and it's used by flake8, coverage, mypy, zest.releaser, ...

Note: see an example in our project where we had to introduce the pyproject.toml just because of black :( BTW our setup.py is only 2 lines long :wink:

[zsol]

I'd really like to avoid supporting multiple configuration styles. I think if we go with setup.cfg, we should remove pyproject.toml.

[adamantike]

I can give it a try if nobody has started working on it. However, I'd try to keep both pyproject.toml and setup.cfg options. Otherwise, it will be a breaking change with a major release associated, and will require project maintainers to update their configs, only to keep using the latest version of black.

Is that what everybody wants?

[altendky]

In general, is embedding into something else's config a good thing? Why? Why not a separate file just for black? Yes, coverage can use setup.cfg, it can also use its own file.

[Natim]

I guess having a .black file would be fine.

A while ago the Pythonic world converge in putting everything related to a python project tooling config into setup.cfg (flake8, coverage, wheel, zest.releaser) the whole ecosystem.

Somehow that also a feature of [ini] files to be categorized, so it made sense to have all in one place. If you look at the Pyramid framework the thing we put in the INI file is even wider.

I am not convinced that we should force people into a way of doing things. I wish we could handle INI config files for black, we don't even need to have a default file.

We could put black config in any INI files and add a black command argument such as --config file.ini to specify where the black config lies.

[altendky]

@zsol, just to clarify, are you only against multiple formats? Or also against multiple possible files? For example, would you consider checking for .black (configparser format) then setup.cfg to be one reasonable option? I can see arguments against both of them, though multiple files of the same format seems less troublesome since at least you don't have to deal with multiple possible structures for the config data.

@Natim, I haven't really followed the convergence. Is there anything formal in that regard I/we should read? It looks like coverage and flake8 anyways both offer setup.cfg, individual files, and tox.ini config options.

[leplatrem]

It looks like coverage and flake8 anyways both offer setup.cfg, individual files, and tox.ini config options.

I like this approach very much! It would allow black to fit in most existing projects without imposing too much beyond code formatting ;) Also since it is close to flake8 in terms of tooling, I believe it makes sense to follow their approach and allow both configs to be close to each other!

[zsol]

I'm against multiple formats as well as multiple possible files. My understanding is that flake8 isn't very happy with how their configuration story looks like today. See https://github.com/ambv/black/issues/65#issuecomment-377587386

[altendky]

https://github.com/altendky/tempexample/tree/f680fc866c6cf80ed3824b92926e0585e5e0143d

I updated the example failure I provided over in pypa/setuptools#1642. It no longer fails with pip==19.0.3 and setuptools==40.8.0. I don't know if this is a permanent state or not so someone might find it worth reading up on the pip/setuptools side to see what the path forward is.

[leplatrem]

I'm against multiple formats as well as multiple possible files.

Full switch to setup.cfg seems rather unlikely, and as said elsewhere it would couple black to one build system.

Since you are a major contributor to this project, I'm now wondering how big are the chances to see this issue being resolved :) Is there any exploration/study that we could do so that you change your mind?

[rixx]

I'm wondering, too – a setup.cfg section for black would be great, or alternatively (although much worse) a .black file or something like that. Are there any chances of that happening?

[tony]

Hi,

This ends up fracturing the config on a lot of projects, unfortunately. I'd like to keep python configs in one spot.

This is one area where setup.cfg makes sense. No other projects I'm aware of (flake8, pytest, isort) require pyproject.toml. In fact, out of all the python plugins I'm using (in open source projects and private ones) black is the only one that requires pyproject.toml.

P.S. I'm not against pyproject.toml (I'm 👍 for it!)

[tom-dalton-fanduel]

It's worth adding that as of pip 19.1, the mere existence of a pyproject.toml (even if it only contains black config and nothing else) causes a change in behaviour that causes issues (for me and some others it seems). https://github.com/pypa/pip/issues/6433 At least one other person has commented their pyproject.toml is used for black config. While I understand the reticence to support more than one file, it does feel like setup.cfg is a fairly standard place (if historic/legacy) for tooling config and if black supported this (in addition to the 'new standard' pyproject.toml) then it would certainly make my life easier.

I'd (and probably others would?) be willing to have a shot at writing the patch to add such support if it stood a chance of being included, but obv don't want to look at doing this if it's not welcome in the project in the first place.

[fchorney]

The mere existance of pyproject.toml is also breaking tox builds due to pip 19.1 and black configs needing to exist in pyproject.toml

I would very much look forward to being able to config black in a different file (I still think keeping the pyproject.toml implementation along side a new location would make sense for projects that specifically use pyptoject.toml for different build-backends, etc)

[altendky]

Ack, 19.0 broke everything and now 19.1 too? It seems like the build process agnostic choice is a separate file (black.toml or whatever). It would equally discriminate against those wanting it in setup.py setup.cfg and those wanting it in pyproject.toml while making sure it won't force people to break their builds when pip updates.

[Natim]

Did you mean setup.cfg? Because I don't think anyone suggested setup.py as an option as far as I know.

[altendky]

@Natim quite so, thanks

[fchorney]

Would adding a black.toml be any more difficult than just doing something like this (quick and dirty). I will admit that I did not look through the project more thoroughly to see if there were any other places that would need to change.

diff --git a/black.py b/black.py
index 0ea9b9a..303702c 100644
--- a/black.py
+++ b/black.py
@@ -205,10 +205,15 @@ def read_pyproject_toml(
     assert not isinstance(value, (int, bool)), "Invalid parameter type passed"
     if not value:
         root = find_project_root(ctx.params.get("src", ()))
-        path = root / "pyproject.toml"
-        if path.is_file():
-            value = str(path)
-        else:
+        # Attempt to find black.toml or pyproject.toml
+        paths = (root / "black.toml", root / "pyproject.toml")
+        found = False
+        for path in paths:
+            if path.is_file():
+                value = str(path)
+                found = True
+                break
+        if not found:
             return None

     try:
@@ -3239,6 +3244,9 @@ def find_project_root(srcs: Iterable[str]) -> Path:
         if (directory / ".hg").is_dir():
             return directory

+        if (directory / "black.toml").is_file():
+            return directory
+
         if (directory / "pyproject.toml").is_file():
             return directory

[zsol]

You can already run black --config black.toml today if this really is blocking you

[goerz]

Big +1 for using setup.cfg instead of pyproject.toml. Just creating a pyproject.toml file has a lot of unintended consequences (switching the build system, and thus breaking compatibility with editable installs). The setup.cfg file seems to have established itself as the standard place for tooling configuration, and would be the first choice from my point of view. Personally, I'd remove support for pyproject.toml completely (which would be a breaking change requiring a major version update).

[ambv]

pip 19.1.1 fixes this issue. We're keeping pyproject.toml.

[rth]

pip 19.1.1 fixes this issue. We're keeping pyproject.toml.

Does it? As far as I understood pip 19.1.1 only addresses the issue with develop mode https://github.com/pypa/pip/issues/6434 that was broken, it doesn't change the fact adding pyproject.toml is going to enable isolated builds PEP 517 for installs. I understand the problem with multiple redundant config formats, but that the auto-formater config has a major impact on the build system seems wrong.

It is maybe less problematic with pure python packages, but for scientific computing this appears to install a separate version numpy and scipy for the build when those used at runtime (e.g. installed with conda) which can be unacceptable¹. This in turns becomes a blocker for using black.

¹ For instance, in the case where no binary wheels exist on PyPi (e.g. ARM), I think it will try compile numpy & scipy from sources even when these packages are already installed from other sources.

[rth]

For more context here are some of unresolved issues in pip related to the use of pyproject.toml (https://github.com/pypa/pip/issues/6144, https://github.com/pypa/pip/issues/6411 ) and it is unfortunate to have to deal with those when all ones is interested in is a code formater (with one none default option).

[richardARPANET]

+1 for setup.cfg, way more widely used.

[leplatrem]

+1 for setup.cfg, way more widely used.

Among the projects that currently have a pyproject.toml file, how many of them had to use it only for black? How many of them also had a setup.cfg for other stuff?

[Natim]

Apparently in order to use pyproject.toml, you have to use poetry? Is that correct?
Here is an example of a project using pyproject.toml instead of setup.py/setup.cfg https://nuget.pkg.github.com/autopub/autopub

[astrojuanlu]

Apparently in order to use pyproject.toml, you have to use poetry?

If "not creating even more confusion about the Python packaging landscape" is not a good enough argument for black to switch to setup.cfg then I don't know what is.

[altendky]

Apparently in order to use pyproject.toml, you have to use poetry? Is that correct?
Here is an example of a project using pyproject.toml instead of setup.py/setup.cfg https://nuget.pkg.github.com/autopub/autopub

No. Black stores its config in pyproject.toml. Poetry stores its config in pyproject.toml. You can put info into pyproject.toml for pip to build your project instead of using setup.py. The issue was that Pip for a bit there was using the mere _existence_ of pyproject.toml to change how installation of projects worked. I think they may have ironed that out now.

[flying-sheep]

pyproject.toml is the standard for configuration. Every *.ini, *.cfg, *rc, and last but not least setup.py will go away at some point in the future.

Tool maintainers not wanting to add support for pyproject.toml are therefore making a bad decision, and black adding support for another config file would be pointless and increase maintenance costs.

[Natim]

As far as I know, today the default Python packaging tools don't support pyproject.toml except for a hack with Poetry.

Source:

• https://packaging.python.org/tutorials/packaging-projects/
• https://poetry.eustace.io/docs/basic-usage/
• https://www.python.org/dev/peps/pep-0518/
• https://packaging.python.org/specifications/distribution-formats/?highlight=pyproject.toml

[flying-sheep]

The links you quoted as sources aren’t saying anything about what supports PEP 518. Also why did you say “hack”? Poetry is a first-class packaging tool and no less or more a hack than any other. I made an overview here: https://github.com/flying-sheep/awesome-python-packaging#readme

Please note how 4/5 package development tools / package managers (exception: pipenv) support pyproject.toml: poetry, flit, pip, and dephell.

[rixx]

Poetry is the best* tool for package development out there. I made an overview here:

Please do not derail this into a discussion on the relative merits of Python packaging tools.

[flying-sheep]

Right, good point, I rephrased.

[cgthayer]

fwiw, until pytest and mypy support pyproject.toml, I'm stuck keeping around a setup.cfg, so I've got both. seems odd that pep518 has been out for 2 years and more things haven't added support.

[hugovk]

Here's a summary of which tools do support pyproject.toml:

• https://github.com/flying-sheep/awesome-python-packaging

[flying-sheep]

pylint added pyproject.toml support! So if one don’t need to configure pytest, they can have all their config in pyproject.toml, e.g. using tox, black, pylint, isort, and poetry

[benjaoming]

I'm a bit confused as to why this issue remains closed despite lots of reactions showing :+1:s for additionally supporting setup.cfg? The reasoning for closing was »_We're keeping pyproject.toml_« but no one suggested to remove this support. Is a PR introducing setup.cfg support unwelcome?

[ambv]

Yes, it's unwelcome. We are not adding setup.cfg as source of configuration, sorry.