Aws-cli: PyYAML is vulnerable

[syabro]

Current version is

PyYAML>=3.10,<=3.13

Should be updated to 4.1b4 or higher

https://nvd.nist.gov/vuln/detail/CVE-2017-18342

[stealthycoin]

We do not use yaml.load with the default unsafe loader which is what the CVE is about.

[syabro]

@stealthycoin but we use and having aws-cli in requirements blocks update in our project :(

[gchiesa]

Are you planning to update anyway the pinning the the not vulnerable version of PyYAML? It would solve also for us the update of dependencies and security checks of our project.

[maciejstromich]

@stealthycoin we're waiting for an update as well. It's blocking our security checks as well.

[maciejstromich]

BTW one way to mitigate is to pin PyYAML version to 4.2b1 before installing awscli.
pip complains about a newer version but it shouldn't be an issue (just made few simple tests to see if my use cases work)

[kyleknap]

So another thing to keep in mind is that version of 4.2b4 is only a pre-release. According to PyYaml's release history, the latest stable release is considered 1.13, which does not default to the safe loader, as the stable release of4.1 was removed from PyPI. In general, the CLI should only be taking dependencies on stable versions of a library. Looking over the PyYaml GitHub repository, there is an issue tracking the release of a 4.2: https://github.com/yaml/pyyaml/issues/193, which would be considered stable.

Given we do not load with the unsafe loader, I think we will have to wait until 4.2 is released, in which we can then proceed in looking to upgrade. Otherwise, the only other option would be to switch out our dependencies to use the ruamel.yaml library. We use that library for v2 of the CLI, but it may be a difficult change to make without breaking any existing functionality for v1 of the CLI.

[maciejstromich]

@kyleknap awesome. thanks for the information.

Personally I'm good with the mitigation path outlined in my previous comment so my PR won't get blocked by our security scans.

[koleror]

BTW one way to mitigate is to pin PyYAML version to 4.2b1 before installing awscli.
pip complains about a newer version but it shouldn't be an issue (just made few simple tests to see if my use cases work)

The problem is it doesn't go through pip-compile, which has the purpose of avoiding exactly this...

[ViktorHaag]

The latest stable release of PyYAML (5.1) after 3.13 has dropped support for Python2.6, which I believe is a problem for this project (see https://github.com/aws/aws-cli/issues/3660), and will likely delay moving forward on this dependency. 8(

[jessedoyle]

Just a question - what is the concern (if any) about relaxing the the restriction on pyyaml to only >=3.10?

This would allow host applications to specify a non-vulnerable version of pyyaml if necessary.

I'm not very familiar with how pip resolves versions, but I'm assuming it'll install the latest possible version (introducing incompatibilities)?

EDIT: It looks like a similar approach is already being considered in https://github.com/aws/aws-cli/pull/4015. I should have dug a bit deeper!

[tpanza]

May I ask why is this still blocked?

It looks like #4015 has a provision for allowing Python 2.6 users to stay with a compatible version of PyYAML (3.13), while allowing Python 2.7+ users to finally move to a newer version of PyYAML (5.1).

Can the solution in #4015 be merged, and then track the issue of Python 2.6 users still having a vulnerable PyYAML dependency in a new issue?

[SeppPenner]

The currently used [email protected] contains a Arbitrary Code Execution issue. Check https://app.snyk.io/vuln/SNYK-PYTHON-PYYAML-42159 and https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-18342 for more information. (Just as reference for the duplicate issue I added here: https://github.com/aws/aws-cli/issues/4193)

[glyph]

Undoubtedly the real issue here is the lack of subtlety and manageability of vulnerability descriptions, but nevertheless, the pin here is creating lots of spurious github security advisories and the real solution to the noise is to just allow for upgrades.

(Edit: this was a mistaken belief generated by a CI workaround that was watching for this issue to be closed, not watching the pin itself...)

[dstufft]

Closing this as we are currently pinning to <5.2 now.