Pyyaml: Linux wheels

[perrygeo]

When installing dependencies in a Linux production environment or container, it would help to have a binary wheel so that deployed artifacts don't require a full build toolchain. We've seen several of our build systems break since awscli>=1.11.19 added a pyyaml dependency.

Using the manylinux wheel approach should allow you to build wheels compatible with any modern 64-bit linux system.

[beraldoleal]

+1

[asottile]

The problem with manylinux is it would have to bundle a specifc version of libyaml.

Here's a more thorough writeup on why this is potentially problematic (especially considering libyaml has received security patches)

[sigmavirus24]

@asottile Not ever user has the same security concerns. Some people just want to get something set-up now that they can test the performance of inside a virtualenv before deploying something more serious to production that is configured a different way.

Yes, vendoring (bundling, whatever you want to call it) static objects of libraries that have had problems can be potentially problematic. That said, it works well enough for other libraries and I'm not terribly concerned about generating new releases for new versions of libyaml if necessary once we have the necessary machinery set up. Please don't spread FUD about manylinux wheels. I think most people understand the security concerns with manylinux wheels.

[asottile]

Unlike traditional FUD, I don't think I'm spreading any disinformation and I certainly don't believe most of the consumers of manylinux wheels understand the intricacies of the implementation and the security tradeoffs. That said, this is your project so do as you must :)

[sigmavirus24]

I certainly don't believe most of the consumers of manylinux wheels understand the intricacies of the implementation and the security tradeoffs.

That's certainly where we disagree then. I choose to trust in the competency of others. True, you're not spreading disinformation, but this is somewhat overly alarmist.

[bsolomon1124]

While I think that @asottile raises a valid concern, I feel like the debate over "are bundled wheels a bad security model?" is one that should take place on a forum that is more broadly scoped than within a single package that uses this model (PyYAML). So, with that said, I plan to push forward with https://github.com/yaml/pyyaml/pull/407, to get to a working POC if nothing else. In this case the question becomes, "how much do we trust the durability of libyaml from a security perspective." I am in no position to answer that myself*, but it looks like there is some crossover in the contributor lists between PyYAML and libyaml, so maybe those contributors would have an opinion one way or the other.

^{* It's also not lost on me that this is a very backwards-facing question akin to saying, "well, libssl is _currently_ secure prior to one of a handful of high severity vulns coming out of the woodwork.}

[nitzmahone]

IMO, bundled wheels are a necessary reality in 2020. Folks that are worried about that from a security perspective are free to install with --no-binary and sort through the extension builds themselves, but I think binary-bundled wheels are here to stay for the masses...

[nitzmahone]

addressed in #407 in 5.4b1