pub needs a dependency security audit feature

[kibantony]

pub needs a feature similar to npm audit that will scan a given project for known vulnerabilities and security issues by checking package versions against a vulnerability tracking database.

[jonasfj]

Wouldn't we also have to build such a database?

note. I'm not saying we shouldn't do this, only that it seems like we have lot of other things to do first.

[kibantony]

Yes. I'm not sure how pub works internally, but presumably you could add a table to the existing package database to track vulnerabilities per package. It would be nice to easily view and report vulnerabilities at pub.dartlang.org.

Alternatively you could rely on the something like the NVD, but that is problematic for a variety of other reasons.

In this age of constant security breaches, this is a critical feature. It's very unsettling to security minded engineers (all engineers?) to include packages from unknown sources with no means to report or audit for vulnerabilities. We can do this for nearly every other part of our stack, (Java, Node, etc.) but not Dart/Flutter. This will only get more critical as Flutter takes off and Dart becomes more mainstream, as I'm sure it will in the coming months.

[jonasfj]

I agree, this would be pretty neat to do. It's just not obvious to me how it would work.

Who can report a vulnerability? Who should review reported security issues? Who flags a package-version as containing a vulnerability?

I've been thinking a bit on how we could do a review system... I'm not sure what it would look like, maybe it's just starring favorit packages, or rating 1-5 stars or writing a text review in markdown.. inspired by how apps are rating the Android Play Store. Maybe, a vulnerability report is a just a particularly bad rating :)

Would vulnerability reports need to private? If so how would we warn users? How long we would keep vulnerabilities secret? Would they need independent reviews/verification?

It's not obvious to me how to do this. I would personally like to avoid supporting a pub audit thing that gives a false sense of impression, if the vulnerability reporting system isn't robust enough.

Note. I do think a pub audit command that could give insights into where the dependent packages originate from would be useful. But we likely need more metadata server side to see that two packages are published by the same author.

---

Note. You can report security issues to package uploaders, their emails are listed on the package page. It's not much, but you can contact the authors this way.

[kibantony]

These are good questions. I can give you specific answers (see below) which may or may not be entirely useful. I'm sure we can agree that once a platform reaches a certain threshold of maturity, explicit vulnerability reporting/tracking/auditing isn't just a neat feature, it's absolutely vital. I would argue that Dart/Flutter has already blown well past this point and will continue to accelerate.

I have no real insight into the organization, but the fact that you're asking questions of this nature leads me to believe there isn't much ancillary support coming from Google. In my answers below I cite npm as the standard example. It's not clear to me how well that applies since npm is backed by a single corporate entity that provides deeply dedicated support and resources. I would hope that Google would provide similar levels of support to Dart/Flutter given the direction it's headed. If that's the case then the short answer to all of your questions is: Escalate up to the larger team/management the need for first class security team and ancillary support to address the need for direct vulnerability reporting/tracking/auditing for Dart packages.

As for specific answers to your questions, this sort of thing is already being done by other language and dev tool teams, so one thing we can do is take a look around and see what's working for them. The closest example to pub/Dart would probably be npm. Looking at npm's policy we can consider some possible answers to your questions.

Who can report a vulnerability?

Anyone.

Who should review reported security issues? Who flags a package-version as containing a vulnerability?

Not the package maintainer. Presumably Google's security team. Or, if that's not feasible, a new security team formed as part of the Dart team. Are there any security engineers already working on Dart? (I hope so!)

Would vulnerability reports need to private? If so how would we warn users? How long we would keep vulnerabilities secret? Would they need independent reviews/verification?

Again npm's policy seems perfectly reasonable here. Reports are reviewed by the security team, independent of the package maintainer. They are kept private between the security team and the package maintainer for some reasonable amount of time (45 days in npm's case), after they are publicly disclosed (presumably at pub.dartlang.org).

[jonasfj]

...explicit vulnerability reporting/tracking/auditing isn't just a neat feature, it's absolutely vital. I would argue that Dart/Flutter has already blown well past this point and will continue to accelerate.

I think we're all in agreement :)

My point was that developing package moderation and vulnerability management policies for pub is a non-trivial process that involves many stakeholders. And it's not obvious that we should simply mirror what other package managers have done.

---

I guess you're suggesting that we simply have an email you can send vulnerabilities to, which we would review and forward to package owner. And if not fixed within 45 days we would publish the report.
(which perhaps seems easier than I initially thought)

I think that mostly involves publishing a policy, ensuring we have the right team receiving emails, and setting up some way of associating notices to specific package-versions. I filed https://github.com/dart-lang/pub-dartlang-dart/issues/2160, and closing this as it's not obvious what the CLI tool aspect will look like before the data is present serverside.

[kibantony]

Thank you @jonasfj. 👍

[bach942]

@jonasfj and @kibantony you might be able to use Snyk's database or have them create a plugin to scan the dependencies. They do this for other languages but is similar to npm audit, npm audit fix. Docs.