Dependencycheck: JenkinsPlugin: [DependencyCheck] Message: Unable to download the NVD CVE data.

Created on 19 Jan 2018 · 17Comments · Source: jeremylong/DependencyCheck

I am trying to run dependency check as a stage in jenkins file, but I have an error

[DependencyCheck] Message: Unable to download the NVD CVE data.
[DependencyCheck] org.owasp.dependencycheck.data.update.exception.UpdateException: Unable to download the NVD CVE data.
[DependencyCheck] at org.owasp.dependencycheck.data.update.NvdCveUpdater.update(NvdCveUpdater.java:130)

So how can I resolve this problem please !?

question

Source

ammy1999

Most helpful comment

Workaround, if you still have an older cached copy of the data:

Add -DcveValidForHours=168 to your mvn command, so that dependency-check will accept any data from the last 7 days and not attempt to download stuff again.

pioto on 22 Jan 2018

👍2

All 17 comments

Same here. The problem has appeared overnight, the nightly build before still worked.

I had some success tracking the problem to SSLSocketFactoryEx in dependency-check-utils. It seems if I remove TLSv1 from the preferred protocols, the download works again. Unfortunately, the property downloader.tls.protocols is not exposed in the Maven plugin settings, so I cannot test if it would help setting it to TLSv1.1,TLSv1.2,TLSv1.3.

I tried the Oracle JDK and the BouncyCastle provider, but nothing worked so far.

Some excerpt from the logs (maven -X):

15:05:22 [DEBUG] #download   threads: 12
15:05:22 [DEBUG] #processing threads: 8
15:05:22 [INFO] starting getUpdatesNeeded() ...
15:05:22 [DEBUG] Checking for updates from: https://nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2002.xml.gz
15:05:22 [DEBUG] Checking for updates from: https://nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2015.xml.gz
15:05:22 [DEBUG] Checking for updates from: https://nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2006.xml.gz
15:05:22 [DEBUG] Checking for updates from: https://nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2007.xml.gz
15:05:22 [DEBUG] Checking for updates from: https://nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2004.xml.gz
15:05:22 [DEBUG] Checking for updates from: https://nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2005.xml.gz
15:05:22 [DEBUG] Checking for updates from: https://nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2008.xml.gz
15:05:22 [DEBUG] Checking for updates from: https://nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2009.xml.gz
15:05:22 [DEBUG] Checking for updates from: https://nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-Modified.xml.gz
15:05:22 [DEBUG] Checking for updates from: https://nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2003.xml.gz
15:05:22 [DEBUG] Checking for updates from: https://nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2011.xml.gz
15:05:22 [DEBUG] Checking for updates from: https://nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2013.xml.gz
15:05:22 [DEBUG] Available Protocols:
15:05:22 [DEBUG] Available Protocols:
15:05:22 [DEBUG] SSLv2Hello
15:05:22 [DEBUG] SSLv3
15:05:22 [DEBUG] TLSv1
15:05:22 [DEBUG] TLSv1.1
15:05:22 [DEBUG] TLSv1.2
15:05:22 [DEBUG] SSLv2Hello
15:05:22 [DEBUG] SSLv3
15:05:22 [DEBUG] TLSv1
15:05:22 [DEBUG] TLSv1.1
15:05:22 [DEBUG] TLSv1.2
15:05:22 [DEBUG] Available Protocols:
15:05:22 [DEBUG] SSLv2Hello
15:05:22 [DEBUG] SSLv3
15:05:22 [DEBUG] TLSv1
15:05:22 [DEBUG] TLSv1.1
15:05:22 [DEBUG] TLSv1.2
15:05:22 [DEBUG] Available Protocols:
15:05:22 [DEBUG] SSLv2Hello
15:05:22 [DEBUG] SSLv3
15:05:22 [DEBUG] TLSv1
15:05:22 [DEBUG] TLSv1.1
15:05:22 [DEBUG] TLSv1.2
15:05:22 [DEBUG] Available Protocols:
15:05:22 [DEBUG] SSLv2Hello
15:05:22 [DEBUG] SSLv3
15:05:22 [DEBUG] TLSv1
15:05:22 [DEBUG] TLSv1.1
15:05:22 [DEBUG] TLSv1.2
15:05:22 [DEBUG] Available Protocols:
15:05:22 [DEBUG] SSLv2Hello
15:05:22 [DEBUG] SSLv3
15:05:22 [DEBUG] TLSv1
15:05:22 [DEBUG] TLSv1.1
15:05:22 [DEBUG] TLSv1.2
15:05:22 [DEBUG] Available Protocols:
15:05:22 [DEBUG] SSLv2Hello
15:05:22 [DEBUG] SSLv3
15:05:22 [DEBUG] TLSv1
15:05:22 [DEBUG] TLSv1.1
15:05:22 [DEBUG] TLSv1.2
15:05:22 [DEBUG] Available Protocols:
15:05:22 [DEBUG] SSLv2Hello
15:05:22 [DEBUG] SSLv3
15:05:22 [DEBUG] TLSv1
15:05:22 [DEBUG] TLSv1.1
15:05:22 [DEBUG] TLSv1.2
15:05:22 [DEBUG] Available Protocols:
15:05:22 [DEBUG] SSLv2Hello
15:05:22 [DEBUG] SSLv3
15:05:22 [DEBUG] TLSv1
15:05:22 [DEBUG] TLSv1.1
15:05:22 [DEBUG] TLSv1.2
15:05:22 [DEBUG] Available Protocols:
15:05:22 [DEBUG] SSLv2Hello
15:05:22 [DEBUG] SSLv3
15:05:22 [DEBUG] TLSv1
15:05:22 [DEBUG] TLSv1.1
15:05:22 [DEBUG] TLSv1.2
15:05:22 [DEBUG] Available Protocols:
15:05:22 [DEBUG] SSLv2Hello
15:05:22 [DEBUG] SSLv3
15:05:22 [DEBUG] TLSv1
15:05:22 [DEBUG] TLSv1.1
15:05:22 [DEBUG] TLSv1.2
15:05:22 [DEBUG] Available Protocols:
15:05:22 [DEBUG] SSLv2Hello
15:05:22 [DEBUG] SSLv3
15:05:22 [DEBUG] TLSv1
15:05:22 [DEBUG] TLSv1.1
15:05:22 [DEBUG] TLSv1.2
15:05:23 [DEBUG] Checking for updates from: https://nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2010.xml.gz
15:05:23 [DEBUG] Checking for updates from: https://nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2012.xml.gz
15:05:23 [DEBUG] Available Protocols:
15:05:23 [DEBUG] SSLv2Hello
15:05:23 [DEBUG] SSLv3
15:05:23 [DEBUG] TLSv1
15:05:23 [DEBUG] TLSv1.1
15:05:23 [DEBUG] TLSv1.2
15:05:23 [DEBUG] Available Protocols:
15:05:23 [DEBUG] SSLv2Hello
15:05:23 [DEBUG] SSLv3
15:05:23 [DEBUG] TLSv1
15:05:23 [DEBUG] TLSv1.1
15:05:23 [DEBUG] TLSv1.2
15:05:23 [DEBUG] Checking for updates from: https://nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2014.xml.gz
15:05:23 [DEBUG] Checking for updates from: https://nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2016.xml.gz
15:05:23 [DEBUG] Checking for updates from: https://nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2017.xml.gz
15:05:23 [DEBUG] Available Protocols:
15:05:23 [DEBUG] SSLv2Hello
15:05:23 [DEBUG] SSLv3
15:05:23 [DEBUG] TLSv1
15:05:23 [DEBUG] TLSv1.1
15:05:23 [DEBUG] TLSv1.2
15:05:23 [DEBUG] Available Protocols:
15:05:23 [DEBUG] SSLv2Hello
15:05:23 [DEBUG] SSLv3
15:05:23 [DEBUG] TLSv1
15:05:23 [DEBUG] Available Protocols:
15:05:23 [DEBUG] SSLv2Hello
15:05:23 [DEBUG] SSLv3
15:05:23 [DEBUG] TLSv1
15:05:23 [DEBUG] TLSv1.1
15:05:23 [DEBUG] TLSv1.2
15:05:23 [DEBUG] TLSv1.1
15:05:23 [DEBUG] TLSv1.2
15:05:23 [DEBUG] Checking for updates from: https://nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2018.xml.gz
15:05:23 [DEBUG] Available Protocols:
15:05:23 [DEBUG] SSLv2Hello
15:05:23 [DEBUG] SSLv3
15:05:23 [DEBUG] TLSv1
15:05:23 [DEBUG] TLSv1.1
15:05:23 [DEBUG] TLSv1.2
15:06:59 [WARNING] Unable to download the NVD CVE data; the results may not include the most recent CPE/CVEs from the NVD.
15:06:59 [WARNING] Unable to update Cached Web DataSource, using local data instead. Results may not include recent vulnerabilities.

The requests then run into a timeout.

Maven and Java versions:

15:04:15 Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T09:58:13+02:00)
15:04:15 Maven home: /var/lib/jenkins/.m2/wrapper/dists/apache-maven-3.5.2-bin/563ks5gsb98oggqkgnrf865tt2/apache-maven-3.5.2
15:04:15 Java version: 1.8.0_162, vendor: Oracle Corporation
15:04:15 Java home: /usr/lib/jvm/oracle-java8-jdk-amd64/jre
15:04:15 Default locale: en_US, platform encoding: UTF-8
15:04:15 OS name: "linux", version: "4.9.0-5-amd64", arch: "amd64", family: "unix"

MenschNestor on 19 Jan 2018

You can set downloader.tls.protocols as environment variable (i.e. mvn -Ddownloader.tls.protocols=<value> ...

jeremylong on 19 Jan 2018

I tried, but that didn't seem to take. Let me check again.

Yes, the logs above are from a build where I added -Ddownloader.tls.protocols=TLSv1.1,TLSv1.2,TLSv1.3

MenschNestor on 19 Jan 2018

And how can I modify datadir: 'dependency-check-data' by adding an environment variable !

ammy1999 on 19 Jan 2018

The properties from the Maven output:

15:05:17 [DEBUG] Properties updated via merge:
15:05:17 
15:05:17 data.driver_path=''
15:05:17 cve.url-2.0.modified='https://nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-Modified.xml.gz'
15:05:17 cve.check.validforhours='4'
15:05:17 cve.startyear='2002'
15:05:17 data.version='3.0'
15:05:17 analyzer.hint.enabled='true'
15:05:17 analyzer.cocoapods.enabled='true'
15:05:17 cve.url-2.0.original='https://nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-Modified.xml.gz'
15:05:17 analyzer.nvdcve.enabled='true'
15:05:17 cve.url-1.2.base='https://nvd.nist.gov/download/nvdcve-%d.xml.gz'
15:05:17 analyzer.falsepositive.enabled='true'
15:05:17 cve.url-2.0.base='https://nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-%d.xml.gz'
15:05:17 application.version='2.1.0'
15:05:17 data.connection_string='jdbc:h2:file:%s;FILE_LOCK=SERIALIZED;AUTOCOMMIT=ON;'
15:05:17 analyzer.archive.enabled='true'
15:05:17 analyzer.ruby.gemspec.enabled='true'
15:05:17 analyzer.assembly.enabled='true'
15:05:17 analyzer.jar.enabled='true'
15:05:17 analyzer.python.distribution.enabled='true'
15:05:17 analyzer.cmake.enabled='true'
15:05:17 analyzer.node.package.enabled='true'
15:05:17 data.password='*****'
15:05:17 cpe.url='https://static.nvd.nist.gov/feeds/xml/cpe/dictionary/official-cpe-dictionary_v2.3.xml.gz'
15:05:17 downloader.quick.query.timestamp='true'
15:05:17 application.name='Dependency-Check Core'
15:05:17 analyzer.cpesuppression.enabled='true'
15:05:17 engine.version.url='https://jeremylong.github.io/DependencyCheck/current.txt'
15:05:17 analyzer.central.url='https://search.maven.org/solrsearch/select'
15:05:17 data.user='dcuser'
15:05:17 analyzer.nsp.url='https://api.nodesecurity.io/check'
15:05:17 data.driver_name='org.h2.Driver'
15:05:17 analyzer.bundle.audit.enabled='true'
15:05:17 updater.nvdcve.enabled='true'
15:05:17 analyzer.dependencybundling.enabled='true'
15:05:17 analyzer.swift.package.manager.enabled='true'
15:05:17 analyzer.versionfilter.enabled='true'
15:05:17 cve.cpe.startswith.filter='cpe:/a:'
15:05:17 analyzer.vulnerabilitysuppression.enabled='true'
15:05:17 updater.versioncheck.enabled='true'
15:05:17 analyzer.filename.enabled='true'
15:05:17 data.file_name='dc.h2.db'
15:05:17 analyzer.experimental.enabled='false'
15:05:17 analyzer.nexus.proxy='true'
15:05:17 archive.scan.depth='3'
15:05:17 analyzer.openssl.enabled='true'
15:05:17 data.directory='[JAR]/../../dependency-check-data/3.0'
15:05:17 analyzer.central.enabled='true'
15:05:17 analyzer.composer.lock.enabled='true'
15:05:17 cve.url.modified.validfordays='7'
15:05:17 analyzer.nuspec.enabled='true'
15:05:17 analyzer.nexus.enabled='false'
15:05:17 analyzer.cpe.enabled='true'
15:05:17 proxy.disableSchemas='true'
15:05:17 cve.url-1.2.modified='https://nvd.nist.gov/download/nvdcve-Modified.xml.gz'
15:05:17 analyzer.nsp.package.enabled='true'
15:05:17 analyzer.autoconf.enabled='true'
15:05:17 downloader.tls.protocols='TLSv1,TLSv1.1,TLSv1.2,TLSv1.3'
15:05:17 analyzer.dependencymerging.enabled='true'
15:05:17 cpe.validfordays='30'
15:05:17 analyzer.nexus.url='https://repository.sonatype.org/service/local/'
15:05:17 autoupdate='true'
15:05:17 analyzer.python.package.enabled='true'

MenschNestor on 19 Jan 2018

please, I use dependency check as a plugin jenkins

ammy1999 on 19 Jan 2018

I'm not sure anymore that this is related to TLS. It started partially working again. Might be on our end after all.

MenschNestor on 19 Jan 2018

What do you mean by partially?

jeremylong on 22 Jan 2018

I suspect this is related to the budget issues with the Federal Government:

Because, http://nvd.nist.gov/ just plain doesn't load at all...

pioto on 22 Jan 2018

When I built the tool to mirror the NVD, gracefully operating Dependency-Check in the face of a government shutdown wasn't something I envisioned at the time, but those of you that are mirroring the NVD will likely not notice its offline. Dependency-Check will continue to work as expected, just without any updates until the shutdown lifts.

Its a little late now, but https://github.com/stevespringett/nist-data-mirror may be useful for these types of situations.

stevespringett on 22 Jan 2018

Workaround, if you still have an older cached copy of the data:

Add -DcveValidForHours=168 to your mvn command, so that dependency-check will accept any data from the last 7 days and not attempt to download stuff again.

pioto on 22 Jan 2018

👍2

You could also add this to your build.gradle:

dependencyCheck {
    autoUpdate = false
}

theoskolnik on 22 Jan 2018

👍1

@jeremylong Sorry, with "partially" I meant that the downloads started working again, but some of them still ran into timeouts. The problem has however fully resolved itself for me; our nightly builds have worked without a hitch since Friday night (local time).

MenschNestor on 23 Jan 2018

@pioto The Department of Commerce announcement at https://www.commerce.gov/news/blog/2018/01/united-states-department-commerce-plan-orderly-shutdown-due-lapse-congressional states the following:

In compliance with the restrictions of the Antideficiency Act, the Department of Commerce will maintain the following services and activities during a lapse in appropriations:
…

Operation of the national timing and synchronization infrastructure as well as the National Vulnerability Database
…

The downloads are working as of now, at least for our projects.

MenschNestor on 23 Jan 2018

@stevespringett Thanks for your response ,
please How can I implement nist data mirror ( I am using jenkins plugin ) because dependencyCheck not work !

ammy1999 on 23 Jan 2018

@MenschNestor yes, it appears to be working again.

@ammy1999 the details on how to mirror will vary slightly depending on what method of execution you're using, but refer to the tool and properties mentioned here: https://jeremylong.github.io/DependencyCheck/data/mirrornvd.html

stevespringett on 23 Jan 2018

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.