Pip: Switch to TOML for configuration files

[pradyunsg]

3691 means pip will need to vendor some form of TOML library.

In view of consistency, is there any interest in shifting to the TOML format for configuration files?

[pradyunsg]

@dholth Could you label this config?

[RonnyPfannschmidt]

@pradyunsg this would be a massive compatibility breach - its not necessarily worth it

[dholth]

Which configuration files?

Uwsgi solves this problem by supporting all file formats

[agronholm]

Uwsgi is also notorious for its feature bloat.

[dholth]

And I use uwsgi instead of 15 separate programs, saving tons of maintenance work. Notorious!

[dholth]

Closed by mistake

[agronholm]

Not sure what you're trying to say here, but uWSGI's development style comes with a costly maintenance burden. We don't want pip to be another uWSGI.

[dholth]

The answer to the original question is "no" because this has been untouched for six months. The answer to "is there interest" might change to "yes" if @pradyunsg can show how toml improves pip's configuration from the implementation and end-user sides. But it might be difficult to write an implementation that elegantly supports both the old and new file formats. IIUC pip already supports three configuration formats: environment variables, command line arguments, and the file.

@agronholm You mean you don't think pip would be improved by hilarious documentation and sweary log messages? How about modes named after popular video games? I'm disappointed. We don't want pip to be hard to maintain but that is irrelevant to the end user. Without feature bloat there would be no features and no users. We wouldn't have to implement the features in the same style as uWSGI though.

[agronholm]

@dholth There is a middle ground between "no new features" and "features like in uWSGI". That's where we want to be. Given pip's importance to the Python community, we want to add features in a way that maximizes maintainability, and think hard about what to add and what not to.

[pradyunsg]

I'll respond to/randomly type about various comments:

@dholth said:
if @pradyunsg can show how toml improves pip's configuration

The way I look at this is that TOML is already a part of PEP-518 - it is going to be a part of pip. So, it makes sense to switch to TOML for what it is actually meant for: configuration files.

@RonnyPfannschmidt said:
this would be a massive compatibility breach - its not necessarily worth it

I don't think so. _massive compatibility breach_ is overstating the deal. The current pip.ini or pip.conf files will be _nearly_ what the final pip.toml would look like. Anyway, it'll obviously have a deprecation period and all that, so I think it'll be fine. To me, the minor inconvenience it'll cause seems to be worth it.

@dholth said:
Which configuration files?

There's only INI format files which have a OS-dependant (and environment-dependant) path.

@dholth said:
IIUC pip already supports three configuration formats: environment variables, command line arguments, and the file.

Yep! The overriding order is: cli -> environment -> subcommand config -> global config -> option defaults.

@agronholm said:
There is a middle ground between "no new features" and "features like in uWSGI".

I see this as the said middle ground. And then I like TOML enough to want to see it be the Python Packaging Ecosystem as the configuration file. :smiley:

@dholth said:
Without feature bloat there would be no features and no users.

Feature Bloat != Having features. Feature bloat is useless features that add no value to end user or even affect UX negatively. Anyway, let's not fight on terminology.

Also, I don't think uWSGI is a good example for this anyway. Let's not talk with that as a reference.

Could you label this config?

Yeah @dholth, please add the label config to this issue...

[dstufft]

FTR I think it makes sense to move our configuration over to using TOML, possibly deprecating the old configuration as well.

[pradyunsg]

@dstufft FWIW, #4192 is a start to getting there.

[pradyunsg]

Is this something that would be a nice-to-have in 10.0?

[dstufft]

It'd be nice to have, not sure I care about if it's in 10.0 or not though.

[agronholm]

Is somebody working on this?

[pradyunsg]

As of now, I'm don't think anyone is working on this.

It's on my personal roadmap but I don't think I would be getting to this in
the next 2 or even 3 months.

I guess anyone who wants to take this on can feel free to do so. :)

On Sat, Jul 8, 2017, 17:32 Alex Grönholm [e-mail] wrote:

Is somebody working on this?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pypa/pip/issues/3809#issuecomment-313851871, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ADH7SWojxApJ_r45D0tmYNL4XH0SKANAks5sL2-5gaJpZM4I8qm1
.

[cupen]

BTW: Would you add a sub-command config which use for set/get variables?
e.g.:

$ pip config set global.index-url   https://pypi.tuna.tsinghua.edu.cn/simple
Done.
$ pip config set global.format      columns
Done.

It means the variable global.index-url and global.format will be saved to the configuration file.

$ pip config get global.index-url
https://pypi.tuna.tsinghua.edu.cn/simple
$ pip config get global.format
columns

It is both on human and scripts friendly.

[RonnyPfannschmidt]

that would depend on if there is a vendor-able toml library that supports editing without throwing away meta-data (else a serialize/de-serialize would just throw away comments and other things the user might have added manually

[cupen]

Oh sorry, it's duplicated with #1736

[pradyunsg]

There is a pip config command ready to ship for pip 10. =)

And the serializability that Ronny Pfannschmidt points out isn't needed
(pip config doesn't have it now) but is definitely something I want to add.

On Thu, 8 Mar 2018, 15:53 Ronny Pfannschmidt, [e-mail]
wrote:

that would depend on if there is a vendor-able toml library that supports
editing without throwing away meta-data (else a serialize/de-serialize
would just throw away comments and other things the user might have added
manually

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pypa/pip/issues/3809#issuecomment-371445663, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ADH7SV2A9-k6G1Id25wHv_uMn8BAxSLNks5tcQalgaJpZM4I8qm1
.

[pradyunsg]

that would depend on if there is a vendor-able toml library that supports editing without throwing away meta-data (else a serialize/de-serialize would just throw away comments and other things the user might have added manually

This exists now: https://github.com/sdispater/tomlkit