Pkg.jl: Add a Pkg.audit command to check dependencies against reported vulnerabilities

[SaschaMann]

TL;DR: A new command Pkg.audit() that checks installed dependencies/environment files for reported security vulnerabilities; inspired by npm audit.

This is meant to start a discussion first, rather than providing details on how to implement the feature.

---

npm has a command to check dependencies against security vulnerabilities known to the registry. Imo it would be useful to have a similar command in Julia’s package manager. This primarily benefits (deployed) Julia applications, not libraries.

Updating all dependencies, while recommended, often comes with some work to ensure that everything will still work after the update, and in more secure environments it also requires reviewing the new versions of the dependencies. It’s often common that dependencies of applications are only updated when absolutely necessary. The audit command could be run on a very frequent schedule, so that potential vulnerabilities can be found quicker than the usual update cycles.

npm also runs this automatically whenever you install or update any dependency. Further, they provide an npm audit fix command that bumps dependency versions if a later version of the package does not contain the vulnerability anymore.

Instead of repeating everything, take a look at https://docs.npmjs.com/auditing-package-dependencies-for-security-vulnerabilities to find out more about the details, what info is shown etc.

---

Do you think this would generally be a useful addition to Pkg?

---

A first step to implement this in Pkg would be a way for package authors to mark versions as potentially affected by a vulnerability, as well as attaching some information on the details of the vulnerability (even if it's just a link to an issue).

Ideally you’d also want to allow others to report them, but that would likely require human work to verify the reports. Publishing more info on the vulnerabilities, e.g. like the security advisories on npmjs.com, would also be nice in the future, but is not needed for basic functionality.

[KristofferC]

I think having it as a separate package makes more sense because then it isn't locked to the Julia release cycle like Pkg currently is. For something like this you want to be able to quickly make new releases, fixes etc.

Also, to me, it feels a bit too speculative to directly put into Pkg. I haven't seen a single security report for any Julia package ever, so it is unclear how much it will be used and how much it then makes sense to maintain it. In any case, it is probably best to develop something like this outside of Pkg.

[StefanKarpinski]

I do think it would be generally useful to be able to add commands to the Pkg REPL that can install, update and call external packages, so that when you do pkg> audit it starts an external Julia process with its own environment, installs/updates the PkgAudit package in that environment, and then calls PkgAudit.audit(). There are a few commands that people have wanted that could work like this. E.g. pkg> search would be handy but probably shouldn't be built in, and of course pkg> generate which could prompt you for what package generator to use, install that, record that choice and then use that going forward from the Pkg REPL.

[fredrikekre]

I believe it is already possible to hook into the Pkg REPL. I had a branch with that, but I got sniped by David that made the same changes so I think it should work. Not a nice interface for doing it, but IIRC it is doable.

[KristofferC]

A plugin system might be useful but we should really try to avoid exposing as much of the internals of the Pkg REPL as possible. Also, it isn't super clear to me that using PkgAudit; audit() is that much worse than using PkgAudit; ] audit.

[SaschaMann]

Also, to me, it feels a bit too speculative to directly put into Pkg. I haven't seen a single security report for any Julia package ever, so it is unclear how much it will be used and how much it then makes sense to maintain it. In any case, it is probably best to develop something like this outside of Pkg.

It's definitely more speculative/planning ahead. I think it's inevitable that these issues will show up eventually the more Julia is used in production applications.

---

Developing it as a separate package at first sounds like a good idea. However, there are many advantages to closely connecting it to the registries, which is the main reason why I think eventually it would make sense to have it in Pkg (assuming there will be enough demand for it). Automatically running it when doing other Pkg operations also seems quite useful.

[StefanKarpinski]

Note that there is already a mechanism for yanking package versions in the registry.

[SaschaMann]

Note that there is already a mechanism for yanking package versions in the registry.

Doesn't solve the same problem. If something is already deployed, yanking a version won't affect it. It's also fairly common that a vulnerability only affects a part of a package or a lot of different versions. Yanking would likely not be the optimal solution in those cases.