Pip: PEP 518: Behavior when pyproject.toml doesn't have build-system.requires

[pradyunsg]

https://github.com/pypa/pip/blob/018f03aab448231462b6c013a65d60a35e06e65a/src/pip/_internal/req/req_install.py#L450

If other @pypa/pip-committers think it's a good idea, we should not isolate if there's no build-system.requires in the pyproject.toml file.

This isn't something I noticed earlier but maybe this would have eased transitions (prevented issues like the one with pandas). I still think it would be a good idea to do this.

[dstufft]

I think this came up in one of the discussions, and I think the decision was to isolate it anyways, but I don't remember why.

[pradyunsg]

@dstufft https://github.com/pypa/pip/pull/4144#issuecomment-293630823 (and the comments before that)

A quick skim doesn't show any mention/discussion on not isolating when build-system.requires is not specified but a pyproject.toml is present.

[pradyunsg]

@pypa/pip-committers Now that we've seen multiple bug reports about this behaviour, I think pip should be refusing to install packages if the pyproject.toml is not PEP 518 compliant (missing build-system.requires) _or_ to decide on using isolation it on the basis of the presence of build-system.requires, not merely the pyproject.toml file.

The former is sort of pushy, forcing people to use PEP 518 if there's a pyproject.toml file; something I'm fine with personally.

[pfmoore]

@pradyunsg I agree. I'm not 100% clear what use cases there are for a pyproject.toml that doesn't contain build-system.requires - the only one I'm aware of is that I think black uses pyproject.toml for configuration (which is allowed by PEP 518, but only with a compliant pyproject.toml).

If it's just people using the "you can put project config in here" part of PEP 518 without following the full spec, I'm +1 on rejecting pyproject.toml if it doesn't contain the mandatory build-system.requires key. If people want to be able to do this, they should get the PEP updated first.

And yes, it is pushy, but "please conform to the standards we took ages to define, when using a brand new file that never existed before the standard" seems like a perfectly OK point to be pushy on :smile:

[pradyunsg]

From https://github.com/pypa/pip/issues/5402#issuecomment-398215531:

Right now, pip opts-in packages to isolation, based on the presence of pyproject.toml but does not require the key to be specified. If it's not there, it just assumes setuptools + wheel are the dependencies and continues with the isolation.

This means that if you have a pyproject.toml, you need that run of pip to also able to install setuptools+wheel; with pip 10.0, it means including wheels for setuptools and wheel, if you're using --no-index.

---

The current behaviour just seems sort of weird to me: the package get opted into isolation because they included configuration for a tool, pip's still sometimes assuming what the build-time requirements are (setuptools+wheel) even in isolation and no one's following the PEP's language.

5512 makes pip compliant with the PEP's language.

I'm personally fine with making the build-system.requires key optional; updating the PEP and then updating pip as per that. In that case, it's presence should be used as a pointer on whether to enable PEP 518.

@RonnyPfannschmidt has correctly stated that making pip compliant with PEP 518's current language would break existing released versions of various packages invalid.

@asottile is of the opinion that PEP 518 should be updated to allow pyproject.toml files that only hold configuration and no build time requirements.

[RonnyPfannschmidt]

i would like to propose to simply fall back to non-isolated building if the pyproject.toml is incorrect, but warning about it

that way things keep "working" but people see it
additionally pip shoul pick a point in time, at which the warning will turn into an error to follow the pep to the letter

@asottile im wondering, could pyupgrade be thaught to match pyproject.toml and set up correct metadata if its missing, i'd prefer strict standards,

[pfmoore]

I agree with @RonnyPfannschmidt that the pragmatic solution is to warn that pyproject.toml is non-compliant and so we're falling back to the "legacy" (non-isolated) behaviour. But we should warn for one release only, and after that we should start rejecting non-compliant pyproject.toml. It's up to the people wanting to use pyproject.toml without specifying build requirements to get the PEP changed within that timescale (I also prefer that we keep the standards strict).

It really shouldn't be such a huge imposition to ask for the standard to be changed - it's not like the change is massively controversial. If no-one can be bothered to step up to champion that change, I don't see that we should be expected to act as if it had been made.

Personally, I'd prefer that we just assumed the spec was being followed, and triggered build isolation and the new code paths when we saw the new-style config file. But I can see that some projects like the idea of pyproject.toml being "somewhere central for tool config" and from what I can see, some tools (black is the one I found) don't support any other option than pyproject.toml. It would be unfortunate if it's not possible to configure such tools without buying into full PEP 517/518 processing. But on the other hand, being forced into a standards change (one that's quite significant in intent, if not in detail) because of a (probably unintended) grass-roots violation of the agreed spec makes me uncomfortable. The above approach seems like a fair compromise.

[pradyunsg]

My intent is to bring pip in line with the PEP. I like what @pfmoore proposes as a compromise since it seems a lot of tools have come to rely upon non-compliant pyproject.toml files now. It also gives users the opportunity to push to update PEP 518 if having the "build-system.requires" key is to be made optional.

As long as we come around to being in compliance with what the PEP says eventually (read by the end of this year) I'm happy. :)

[blueyed]

As stated in https://github.com/pypa/pip/pull/5512#issuecomment-398633297 already (but since discussion should happen here):

PEP 518 only states that requires is required in the build-system section, but not that the build-system section is required.
And using the provided JSON schema allows for an empty pyproject.toml (i.e. "{}" as JSON).

Therefore there should be no warning / later rejection of a pyproject.toml file missing this section.

That's not really what this issue is about, but it was brought up here and resulted in #5512.

[pradyunsg]

I'd read it differently. It does say "for illustrative purposes only" right above the JSON schema. And it even states "build tools are expected to use the example configuration file above as their default semantics when a pyproject.toml file is not present."; but not how to behave when an empty file exists.

To be clear, I'm not against making pyproject.toml files with only tool.* valid; I just want the PEP to be updated to state that clearly before making pip do that.

[pradyunsg]

Shot up a mail to distutils-sig: https://mail.python.org/mm3/archives/list/[email protected]/thread/7A4QHWCHR54TSIO2DQZUVNZHZS6ZPBLY/

[pradyunsg]

It seems to me that it's clear what we want to do here: pyproject.toml files without a build-system.requires are not valid per se; after a standard deprecation, we'll start erroring out in that scenario.

/cc @brettcannon

[pfmoore]

Trying to track back through this discussion.

@pradyunsg commented:

Now that we've seen multiple bug reports about this behaviour

Can you include links? The only two I can find are #5402 and #5511. They are both simple cases of users using --no-index and not including the build tools (setuptools and wheel) in the indexes they do provide. That's clearly a behaviour change, and one that the users need to address, but I'm not sure it's a big enough problem that we should be worrying about disabling isolation just for this case. It's pretty much expected behaviour when pyproject.toml is present, so I consider these issues as mainly user error (with a little bit of "could the documentation make this clearer?")

The discussion here has spread to a lot of places, and it certainly feels like there's a wider issue, but I'm failing to find the actual evidence - and I'm starting to feel that in actual fact, there really isn't a big problem here at all.

Before I say "never mind, pip's behaviour here is fine" can you confirm whether or not there are other issues that relate to this one? And provide links here, so that the chain of logic is clearer?

Regardless of the above, this discussion did pick up on an area where the PEP's wording was unclear, and it's good to fix that. But it's quite possible that all we need to do is to tidy up the wording in the PEP and there's nothing more to do here. My original position was that the PEP said that omitting build-system.requires was an error, and pip should reject with an explicit error in that case - the distutils-sig discussion seems to be reaching a point where the view is that it's not an error, in which case pip is fine on that score. The isolation question is mostly pip-specific, as re-reading PEP 518, there's nothing in there about how tools should set up the build environment based on pyproject.toml (someone please correct me if I missed something).

[brettcannon]

@pfmoore without build-system.requires is there any use for build-system? If not then should it be that not defining build-system represents falling back to the legacy semantics but if it is defined it's an error to not have requires? (I asked this on distutils-sig but never got a reply.)

[pfmoore]

@brettcannon That's certainly an option. As I say though, I'm unclear as to whether there's actually a significant enough issue here to warrant changing pip's behaviour at all. (Isolation is basically a pip implementation/PEP 517 issue, and I'm struggling to find evidence that we need to change what we currently have).

In terms of PEP 518 I have no problem with saying that "if you have [build-system] but it has no requires key, your pyproject.toml is malformed" (I also have no problem with not bothering to do so :smile:). If pip then chooses to stop triggering build isolation when pyproject.toml is present, it makes sense to trigger it based on whether [build-requires] is present instead (but that makes sense even now, as the only real world case which is under any debate is where a project has a pyproject.toml with nothing but a [tools] section).

[brettcannon]

@pfmoore yeah, I'm conflating PEP 518 details with pip details since this discussion has gotten a bit intertwined. 😉 If you want me to kick off a new thread on distutils-sig on this last idea then let me know and I can do that.

[pfmoore]

I've just commented on distutils-sig, I doubt it needs a new thread.

[pradyunsg]

I felt that the details of the implementation and the PEP wording are a bit intertwined since they're both going to influence each other both ways. I'm still calibrating where the line between standards/implementation is here.

[pradyunsg]

Closed by mistake.

[pradyunsg]

As for the scale of this issue, the two issues are from the user's end -- that isolation needs them to provide setuptools and wheel as well. I don't think there's anything we need to change for this.

Making the key mandatory would make it easier for us to deal with such issues since we can just point to the fact that the project's metadata says that you need to provide these packages to install it. That is cleaner than saying, the towncrier configuration the project added triggers a newer code path which means you need to provide wheels for packages (you already have installed those in your environment but we're isolating the builds now).

(edited for clarity + typos)

---

To summarize my understanding here:

• There's no established need to enable the legacy behavior when a pyproject.toml is present w/o the key.
• I see making the key mandatory as an improvement. Making existing projects specify this metadata costs little to us or them.
• Just keeping the warning pip 18.0 will be printing forever pushes users to specify the metadata -- which would need no updates to the PEP and prevents any breakage.

That leaves us with basically 2 reasonable ways we can go:

• Keep the pip 18.0 behavior
  
  
  
  • PEP changes: None
  
  
  • pip changes: None
  
  
  • pushes people to specify metadata but doesn't require it
  
  
  • no breakage
  
  
  • this edge case stays a grey area in the PEP -- we're just pushing users to avoid it
  
  
  • the explanation for behavior change to end users seems a little weird
  
  
  • The current response is like: project X added a pyproject.toml file for configuring a new tool they adopted in their development process, the file's presence changes in how pip builds project X so you now need to provide wheels for the setuptools and wheel when installing project X.
  
  
• mandatory build-system.requires
  
  
  
  • PEP changes: make the table mandatory
  
  
  • pip changes: after a transitory period, error out when the key is missing
  
  
  • requires people to specify metadata
  
  
  • makes end-users and packagers more familiar with it
  
  
  • gets a boost due to tools adopting pyproject.toml
  
  
  • some minor breakage, that is easy to work around
  
  
  • grey area is eliminated from the PEP
  
  
  • easier to explain the behavior change to users of --no-index
  
  
  • Response becomes: project X states that it has these build-time requirements and so you need to provide them in an install-able form to pip.
  
  
  • This is minor but good improvement for us. :P
  
  

---

PS: The discussion does seem to have blown a little out of scale to me now and I might have contributed to that. I now feel while mentioning disabling isolation as an option when I started the thread, I should have explained why we shouldn't do that or just not mentioned it at all since it's an implementation detail and not a PEP detail.

PPS: TOML 1.0 will have dotted keys, so build-system.requires = ["setuptools", "wheel"] is actually a thing you can write in TOML whenever that version gets released.

[pfmoore]

I have to say I'm still not clear what option you actually support here.

Regarding the PEP, I think this whole discussion has proved that a PEP change is needed, at a minimum to explicitly confirm that the [build-system] key is mandatory or not. General consensus over on distutils-sig is that it shouldn't be (although if it is present, @brettcannon is arguing that the requires element should be mandatory).

If you believe that PEP 518 should mandate the build-system section, then (a) you're saying that all the projects using pyproject.toml just for towncrier or black options are invalid and need changing, and (b) you'll need to persuade people over on distutils-sig.

For pip, we have to use the new PEP 518 code path (isolation) for projects that use pyproject.toml to specify [build-system].requires[1]. Not because the PEP says so, but because that's how our implementation works. And we do that. What we do for projects that don't specify that key is up to us - the options are:

1. Isolate always. This would be the ideal, and is the long-term plan, but we didn't do this for pip 10 because of backward compatibility concerns. But we should be clear that this is the intent.
2. Isolate if pyproject.toml is present, and not if it isn't. That's what we do at the moment.
3. Isolate only if the build-system key is present This is only a separate option if we don't implement Brett's proposal that the requires key is mandatory if the build-system section is specified. Otherwise it's the same as (4).
4. Only isolate if the full [build-system].requires key is present.

If (and only if) the consensus is that PEP 518 should allow for pyproject.toml without a build-system key, then we have to decide whether we retain or change the current behaviour. Personally, I'm coming to the conclusion that the current behaviour hasn't caused anywhere near enough problems to warrant changing anything (with the possible exception of trying to provide a clearer error message, which may not be possible).

As a separate point, with regard to ease of explaining the behaviour, I take your point that it's easier to explain if [build-system] is mandatory. Absolutes are always easier to explain. But I think the reason we get into such a mess is that we're tying isolation to PEP 518. If we simply say that "since pip 10, we set up an isolated build environment by default", with a proviso that for backward compatibility we currently retain the old non-isolated behaviour for projects that don't include pyproject.toml, then that's an easy explanation. Ideally, we'd include a deprecation warning for the old behaviour, but given that it makes essentially no difference in 99% of cases, that would likely be too intrusive. But maybe a deprecation note in the docs would be worthwhile. However, any such discussion is not relevant here, and we probably shouldn't even start such a discussion until this one is complete (the last thing we need is even more threads...)

[1] OK, so that's TOML 1.0. But it's a convenient shorthand, and I don't know any other concise way to say what I mean.

[pradyunsg]

(I had written a similar comment earlier today but it seems I'd gotten logged out of the network or something and it never got posted)

Okay. Right.

PEP wise, I'd prefer it be changed to make the table (and key) mandatory. If it's instead made optional, I do feel (more strongly than I realized earlier) that pip should keep doing what pip 18.0 would -- isolate on presence of pyproject.toml and printing a warning saying add build-system.requires when it's missing from the file. The reasoning is same as elaborated below.

---

I realize now that I've not really explained why I think the breakage it causes isn't a major issue and the advantage outweighs it (without going into pip's implementation details).

The breakage would be pretty simple to work around -- just pass --no-build-isolation (or using a per-project override as requested in . Yes, we can't avoid it but we can have a fairly smooth transition by staying with pip 18.0's behavior for a transitory period and then flipping the switch. To reiterate what I said on the mailing list about this:

if a project has been quick to adopt a new standard that's documented as implemented incompletely currently, it seems reasonable to make a 2 line change to comply with the standard as it matures.

The advantage is that by making people specify the key, it brings familiarity and more tools that use pyproject.toml, more projects would mention build-system.requires and the more visible it becomes. At a functional level, there's basically no change if people specify the key (pip would isolate with setuptools, wheel anyway). At a social level, it becomes a more prominent part of packaging and I expect more people would know/learn what it does/means if they have to explicitly use it.

I guess this still has some (indirect) reference to how pip has isolation tied-in with this. I guess I'll think about how to say this after you post your summary.

---

[1] OK, so that's TOML 1.0. But it's a convenient shorthand, and I don't know any other concise way to say what I mean.

I helped get it added to TOML (made the issue, engaged in discussion, made the spec change and all that) so I just mentioned this out of excitement.

It's nice to see that this is the way we ended up referring to this key organically. :)

[pfmoore]

PEP wise, I'd prefer it be changed to make the table (and key) mandatory.

OK. Please make that argument over on distutils-sig. It's not me you have to convince 😄

I do feel (more strongly than I realized earlier) that pip should keep doing what pip 18.0 would

We can defer that discussion till the final decision on the PEP is made clear.

[pradyunsg]

OK. Please make that argument over on distutils-sig.

Yep. Will do. :)

[pradyunsg]

It seems to me that distutils-sig has reached a consensus here.

If we decide that pip should keep status quo as its behavior going forward, we should go ahead and remove the warning that's currently on pip's master when this edge-case occurs.

---

I'm not keen on holding off 18.0 for this though, but am willing to do so it someone else from @pypa/pip-committers wants us to. Note that we might end up having to discuss a bit here and the current plan is to make the release on Monday.

[dstufft]

What's the delta between the current behavior and the distutils-sig behavior? Is it just the warning?

[pradyunsg]

Yes.

[pradyunsg]

Assuming we decide to isolate with (setuptools, wheel) when the build-system table is not specified.

[dstufft]

If the only change is the warning, then we should probably remove the warning, because it's not going to break in the future right? So the warning is just going to spread FUD.

[pradyunsg]

Whoops. I missed a detail - pip currently treats missing build-system table the same as missing requires key in a given build-system table but the discussion is inclined to rejecting the latter.

I'll make a PR to remove the warning though. It doesn't need to be there now. 18.0 will behave like 10.0 does.

I put in some minutes earlier drafting a PR for the 2b proposal (that's how I noticed this). I'll file it later -- let's not make that change in 18.0. We can discuss 2a vs 2b after 18.0 and make that change.

[pfmoore]

Sorry, now I'm confused.

Ignoring the warning, pip currently handles a missing [build-system].requires by isolating, and installing setuptools and wheel, doesn't it? That is the 2b behaviour. That's certainly what 10.0 did (hence incidents like #5511). So I'm not sure what you mean about "drafting a PR for the 2b proposal".

If we're now disabling isolation on master if pyproject.toml exists but has no [build-system] section, then that's a behaviour change from 10.0, and I don't recall it happening - can you point me to the PR where that was implemented?

I agree that while the PEP will expect tools to error if [build-system] is present but requires isn't, that's not a showstopper IMO and doesn't need to hold up 18.0.

[pradyunsg]

I messed up in the above comment (it got posted prematurely - I didn't realize that it was posted). I'll rectify in a bit.

[pradyunsg]

Coming back around to this finally...

About the "drafting a PR for the 2b proposal", I'd seen that pip stopped isolating in this edge case when #5512 was merged (by me); I subsequently realized that I had rectified that error in #5585. Sorry for the confusion.

---

More concretely:

5626 is what I expect will be a part of pip 18.0 -- it simply removes the warning that was added in #5512. Would appreciate reviews on that.

5627 implements the error when [build-system] is present but requires isn't. I don't think it is a showstopper either so it can come in after 18.0. We can bundle that with the "decision" to go ahead with "2b".

[pradyunsg]

Okay then. With #5626 and #5627, we're in compliance with PEP 518.

PEP 518 now allows packages to omit the build-system table entirely and pip is going to currently isolate when the table is missing -- treating it the same as ["setuptools", "wheel"] and building it in isolation.

[pradyunsg]

Closing since I think we're done here. If someone wants to discuss changing pip's behavior in this edge case, please file a new issue. :)

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.