Dependencycheck: Bug in Maven Central Analyzer

[baderbuddy]

I'm not positive yet if this is an issue on my end or if it's a legitimate problem, the Maven Central analyzer has been throwing errors, it's getting a 400 back from Maven Central. I piped the results through Fiddler to take a look at what's going on, if I put the URL into Chrome it works, so I realized that the difference is that Chrome is URL encoding the quotation marks.

https://search.maven.org/solrsearch/select?q=1:"<sha1>"&wt=xml is what Dependency Check is trying to use.

https://search.maven.org/solrsearch/select?q=1:%22<sha1>%22&wt=xml works.

[AJShipowick]

I just started getting this error today, seems related.
"Could not connect to Central search. Analysis failed."

[stevespringett]

Same result here. Confirmed that search documentation still states searches have a literal double quote in them though.

http://search.maven.org/solrsearch/select?q=1:"35379fb6526fd019f331542b4e9ae2e566c57933"&wt=xml

Changing the \" in CentralSearch.java:111 to %%22 resolves this issue.

final URL url = new URL(String.format("%s?q=1:%%22%s%%22&wt=xml", rootURL, sha1));

[born2snipe]

@stevespringett when do you plan on doing the next release?

[livonthedge]

So glad to see this, was pulling my hair out since I had just done some other server updates and wasn't sure what was going on. Looking forward to the fix!

[THausherr]

But why did this work in the past? I have builds every day.

[jeremylong]

@THausherr great question - the obvious answer is Central has been updated. I know that there have been some issues with the Central Analyzer and I'm trying to come up with alternatives.

@born2snipe in terms of the next release - the team has not discussed a planned next release yet. I will try and cycle through the current list of FP and then cut the next release. Given my schedule I think that will happen next weekend (by Nov 19th).

[basilv]

This is affecting my organization as well. The workaround I found was to disable the Maven Central Analyzer in the global Manage Jenkins Configuration under OWASP Dependency-Check – Dependency-Check: Standard Analyzers and Dependency-Report: Standard Analyzers. (Many of our builds are using the Jenkins OWASP plugin instead of Maven's.)

[born2snipe]

@jeremylong thanks for the heads up on the release, sir.

[zerogvt]

I don't get this error with version 2.1.1 (but I do with 3.0.1).
Using the gradle plugin.

[jeremylong]

That is because in 2.1.1 the error was occurring- but it was silently being
ignored.

Jeremy

On Nov 13, 2017 5:35 PM, "V" [e-mail] wrote:

I don't get this error with version 2.1.1 (but I do with 3.0.1).

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
https://github.com/jeremylong/DependencyCheck/issues/978#issuecomment-344082204,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA0qwkTkXvCzKZZ4uJck4lf2EftUhy-eks5s2MQZgaJpZM4QaJ6B
.

[robertoschwald]

Could you please release 3.0.2 asap and add it to the gradle and jenkins plugins, so it is usable again? Thanks!

[ammy1999]

the version 3.0.2 is already installed in jenkins plugin !!

[robertoschwald]

Not related to this sub-project, but is jenkins plugin already released? I re-installed it and its still 3.0.1

[danshome]

https://github.com/jeremylong/DependencyCheck/releases shows that 3.0.2 isn't released yet, do you have an estimate when this will happen?

[burberius]

I'm desperately waiting for a release, too.

[robertoschwald]

3.0.2 seems to be tagged now. Thanks Jeremy. When can we see it in maven central repo?

[jeremylong]

3.0.2 has been mostly released - gradle, maven, ant, and CLI have been published - it just takes time to cycle through Central. Jenkins and brew will be released hopefully later today... Sorry for the inconvenience - but this project isn't corporate sponsored and I do not believe any of the core contributors work on dependency-check as part of their day job.

[robertoschwald]

np. Thanks for your efforts.

[stevespringett]

Jenkins plugin has been pushed as well. It usually takes a half-day for it to show up in the update site, but it can be downloaded directly for those wanting to get it earlier.

https://repo.jenkins-ci.org/releases/org/jenkins-ci/plugins/dependency-check-jenkins-plugin/3.0.2/

[london-coder]

Hi,

I guess I'm having the same problem. It was working earlier, (last week) and now I get the same symptoms whether using version 3.0.1 or 3.0.2. Here is a maven output.

GDS5714:pay-publicapi alanlewis$ mvn compile
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building pay-publicapi 0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
Downloading: https://repo.maven.apache.org/maven2/org/owasp/dependencey-check-maven/3.0.2/dependencey-check-maven-3.0.2.pom
[WARNING] The POM for org.owasp:dependencey-check-maven:jar:3.0.2 is missing, no dependency information available
Downloading: https://repo.maven.apache.org/maven2/org/owasp/dependencey-check-maven/3.0.2/dependencey-check-maven-3.0.2.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.775 s
[INFO] Finished at: 2017-11-14T17:10:55Z
[INFO] Final Memory: 9M/245M
[INFO] ------------------------------------------------------------------------
[ERROR] Plugin org.owasp:dependencey-check-maven:3.0.2 or one of its dependencies could not be resolved: Could not find artifact org.owasp:dependencey-check-maven:jar:3.0.2 in central (https://repo.maven.apache.org/maven2) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException

and for 3.0.1

GDS5714:pay-publicapi alanlewis$ mvn compile
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building pay-publicapi 0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
Downloading: https://repo.maven.apache.org/maven2/org/owasp/dependencey-check-maven/3.0.1/dependencey-check-maven-3.0.1.pom
[WARNING] The POM for org.owasp:dependencey-check-maven:jar:3.0.1 is missing, no dependency information available
Downloading: https://repo.maven.apache.org/maven2/org/owasp/dependencey-check-maven/3.0.1/dependencey-check-maven-3.0.1.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.966 s
[INFO] Finished at: 2017-11-14T17:05:36Z
[INFO] Final Memory: 9M/245M
[INFO] ------------------------------------------------------------------------
[ERROR] Plugin org.owasp:dependencey-check-maven:3.0.1 or one of its dependencies could not be resolved: Could not find artifact org.owasp:dependencey-check-maven:jar:3.0.1 in central (https://repo.maven.apache.org/maven2) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException

Not sure how helpful this is. I'm trying to make an argument to use dependency-check to replace another tool performing a similar job, and issues like this damage confidence. I realise this is not a fully sponsored effort, and is reliant on the time of individuals, so please keep this terrific effort going.

Regards,
Alan

[life4u2c]

I've been able to successfully use 3.0.2 this morning resolving my issue here. Thank you for attention to this issue.

[livonthedge]

Everything is working correctly for me now with 3.0.2
Thank you!

[robertoschwald]

Great work! It's fixed.

[ammy1999]

the version 3.0.2 resolve my issue also , thank you !!

[AJShipowick]

Solved for me too, thanks for the super fast turn-around!

[johnou]

[ERROR] Failed to execute goal org.owasp:dependency-check-maven:3.0.2:aggregate (default-cli) on project x-server: One or more exceptions occurred during dependency-check analysis: One or more exceptions occurred during dependency-check analysis
[ERROR] Could not connect to Central search. Analysis failed.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]

Still an issue with 3.0.2, I will try and reproduce with the -X switch.

[samuelfernando]

Yes it seems the error is recurring with 3.0.2

[ERROR] Failed to execute goal org.owasp:dependency-check-maven:3.0.2:check (default) on project **: One or more exceptions occurred during dependency-check analysis: One or more exceptions occurred during dependency-check analysis
[ERROR] Could not connect to Central search. Analysis failed.
[ERROR] Could not connect to Central search. Analysis failed.

[dannil]

@johnou @samuelfernando very odd since the 3.0.2 version works for almost everyone else (including me). Is it maybe network related on your end? Try rerunning with -X switch.

[samuelfernando]

Hi @dannil, it does seem like a temporary network issue (don't know whether on our end or the Maven Central side). I tried again a few minutes later and it worked fine. But I have noticed this issue several times before and it is a bit of a problem for us as we need reliable builds.

[johnou]

@dannil still hasn't failed after adding the -X switch, I will comment back here when it happens.

[jeremylong]

We are working on a replacement/change for the central analyzer. I would highly recommend just disabling it if you are using the Maven or Gradle plugins as you get very little benefit in using it.

[tstibbs]

@jeremylong can you expand on your comment? It sounds like you're saying that this maven plugin is not much use anymore - how so?

[jeremylong]

No - the maven plugin is one of the best ways to run dependency-check. However, within dependency-check there is a centralAnalyzer that can add a lot to the execution time and provide little to no benefit (it does add benefit to the ant, command line, and Jenkins executions though).

[johnou]

@jeremylong so by disabling centralAnalyzer the meta data will be pulled in by the Sonar plugin?

[jeremylong]

The meta data is contained within the mavens execution context. No need to
reach out to central to search on the sha1.

Jeremy

On Dec 12, 2017 8:47 AM, "Johno Crawford" [e-mail] wrote:

@jeremylong https://github.com/jeremylong so by disabling
centralAnalyzer the meta data will be pulled in by the Sonar plugin?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/jeremylong/DependencyCheck/issues/978#issuecomment-351055948,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA0qwsyw4HYl_zzsAPvY9EBm4rwZCL0Fks5s_oQEgaJpZM4QaJ6B
.

[johnou]

@jeremylong I mean the metadata for the CVEs etc. that's independent from the centralAnalyzer?

[jeremylong]

The CVE data is separate from central. The CVE data comes from NIST.

Jeremy

On Dec 13, 2017 6:53 AM, "Johno Crawford" [e-mail] wrote:

@jeremylong https://github.com/jeremylong I mean the metadata for the
CVEs etc. that's independent from the centralAnalyzer?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/jeremylong/DependencyCheck/issues/978#issuecomment-351369792,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA0qwh34MnVTEC5DktkeUujnVVNOKJZ5ks5s_7qzgaJpZM4QaJ6B
.

[rompic]

@jeremylong I guess the same is true for the gradle plugin, right? (i.e. central analyzer). Which features depend on it?

[jeremylong]

Yes. Gradle and Maven will add the important coordinates of the dependency to the evidence being collected automatically. Disabling the CentralAnalyzer in these plugins will really not affect much. Whereas the other was to perform a dependency-check analysis are generally more accurate with the additional data retrieved from Central.

[synergiator]

@jeremylong again in 3.1.1 https://travis-ci.org/dropwizard/dropwizard/jobs/349920719 for https://github.com/dropwizard/dropwizard/pull/2296

[jeremylong]

@1605200517 Consider disabling the central analyzer for the integration into a Maven build. Very little is gained from using the Central analyzer when using the Maven or Gradle plugin.

[Mortagne]

@jeremylong , how to "disabling" the central analyzer ... ? I use the "dependencyCheckAnalyzer" in a Declarative Pipeline.

[stevespringett]

@Mortagne if you're using the Jenkins plugin to perform the analysis, the enabling/disabling of analyzers is a Jenkins global config option.

[Mortagne]

@stevespringett , thank you for information.
so do you have a idea ... I have 2 slaves ... node A ... all work fine ... so on node B ... I have this error.

a specific reason ?

[stevespringett]

If both nodes have the same master and the master is configured with Maven Central disabled, then both nodes should work the same. There isn't node-specific or job-specific configuration for the analyzers. If you're seeing inconsistent behavior, it's out of scope for the plugin and may be an issue with Java, Jenkins, configuration, etc outside of the plugin.

Now, if one of the nodes is executing a Maven job and Dependency-Check is being run as part of Maven instead of being analyzed with the Jenkins plugin, that may explain the difference.

Each Dependency-Check implementation (Ant, Maven, Gradle, Jenkins, CLI, etc) all have their own settings and if you want Central disabled, it may be necessary to disable it multiple places depending on what implementations are used.

[Mortagne]

hum ... I use the "dependencyCheckAnalyzer" in a Declarative Pipeline. All slave are controlled by the same Master.

I call this, in a stage of my pipeline:

dependencyCheckAnalyzer datadir: '', hintsFile: '', includeCsvReports: false, includeHtmlReports: true, includeJsonReports: false, includeVulnReports: true, isAutoupdateDisabled: true, outdir: '', scanpath: '', skipOnScmChange: false, skipOnUpstreamChange: false, suppressionFile: '', zipExtensions: ''

dependencyCheckPublisher canComputeNew: false, defaultEncoding: '', healthy: '', pattern: '', unHealthy: ''

Same configuration / user on each two nodes.

[vulnerability-h8er]

I'm still getting the [ERROR] Could not connect to Central search. Analysis failed. intermittently. I'm using version 3.1.2 (latest version as of the moment) and sometimes it works, sometimes not. I'm not changing anything in my config and still getting the error randomly.

Has anyone experienced this? Is disabling centralAnalyzer the only option I have? Thanks.

[nico-amsterdam]

I also get this error, but it looks more like an maven repository issue.
First I get:
Caused by: java.io.IOException: Could not connect to MavenCentral (504): GATEWAY_TIMEOUT
at org.owasp.dependencycheck.data.central.CentralSearch.searchSha1(CentralSearch.java:194)

Later I get:
Caused by: java.io.IOException: Could not connect to MavenCentral (503): Service Unavailable: Back-end server is at capacity
at org.owasp.dependencycheck.data.central.CentralSearch.searchSha1(CentralSearch.java:194)
at org.owasp.dependencycheck.analyzer.CentralAnalyzer.fetchMavenArtifacts(CentralAnalyzer.java:266)

[Mortagne]

@vulnerability-h8er and @nico-amsterdam , are you sure where your run the analysis have access to internet ... try wget http://www.google.com for a example ...

when i open the port to the internet, so the same configuration for my two nodes ... all work fine!

[johnou]

@Mortagne http status code 503 has nothing to do whether or not the build machine has internet access, it's an issue with Maven Central.

[eskamuc]

@nico-amsterdam, @Mortagne: We get the error since today. Our server have connection to the internet and the whole plugin worked before for us.

[vulnerability-h8er]

I am also connected to the internet and the issue is intermittent. Are there other workarounds other than disabling the centralAnalyzer?

[NeilGuce]

Hi @jeremylong do we have any update on this? If you're suggesting to just disable CentralAnalyzer when using Maven because it may not help or of a little benefit only, may I know what is the reason?

[eskamuc]

Has someone already checked, if the behavior still occurs with the latest release (3.1.2.1, published May 2nd)?

[jeremylong]

The version with the Maven and Gradle plugins disabling the central analyzer by default have not been published yet. Hopefully this weekend.

[eskamuc]

@jeremylong Thanks for the update

[PeterFromWiesbaden]

all of a sudden we have got the same problem. Our builds ran fine until today. We haven't changed anything. We have configured Maven Central in a Nexus repository and run Dependency-Check 3.1.2 commandline.
[DependencyCheck] Exception Caught: org.owasp.dependencycheck.analyzer.exception.AnalysisException
[DependencyCheck] Cause: Finally failed connecting to Central search. Giving up after 5 tries.
[DependencyCheck] Message: Could not connect to Central search. Analysis failed.
[DependencyCheck] org.owasp.dependencycheck.analyzer.exception.AnalysisException: Could not connect to Central search. Analysis failed.
[DependencyCheck] at org.owasp.dependencycheck.analyzer.CentralAnalyzer.analyzeDependency(CentralAnalyzer.java:244)
[DependencyCheck] at org.owasp.dependencycheck.analyzer.AbstractAnalyzer.analyze(AbstractAnalyzer.java:137)
[DependencyCheck] at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:88)
[DependencyCheck] at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:37)
[DependencyCheck] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[DependencyCheck] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[DependencyCheck] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[DependencyCheck] at java.lang.Thread.run(Thread.java:748)

Any suggestions what we can do.

Peter

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.