Azure-pipelines-tasks: [Question] Is there a way to have SonarQube break build on PR when quality gate fails?

Created on 19 Sep 2016 · 10Comments · Source: microsoft/azure-pipelines-tasks

The code looks pretty straight forward: https://github.com/Microsoft/vsts-tasks/blob/master/Tasks/SonarQubePostTest/SonarQubeBuildBreaker.ps1#L13

But still I wanted to ask if maybe there is some configuration or some other way to ensure build breaks when quality gate fails. If not, can you explain why? For me, and I believe many others, this is the single most important feature of SonarQube and VSTS integration.

Source

sinapis

All 10 comments

Hi @sinapis - this is in fact a SonarQube limitation - i.e. the quality gate is not calculated during PR builds. Please contact SonarSource, the company behind SonarQube with a feature request.

In a PR build we run the analysis in "issues mode", i.e. the analysis is not recorded in the SQ database and more importantly, the quality gate is NOT computed. The issues are written in a file which we process and then we inject them into the code review associated with the PR . Not running a full analysis during a PR is important because it would create too much noise in the tech debt trends and it has been suggested by the SonarSource team.

bgavrilMS on 19 Sep 2016

👍1

Thanks for the quick response!
In this case I'd like to request a feature - add something like approvalFeatureEnabled for https://github.com/mibexsoftware/sonar-bitbucket-plugin where build will break if blocking or critical issues exist.

sinapis on 19 Sep 2016

Interesting. Please confirm I understand the scenarion - SonarQube pushed comments to the PR but the PR cannot be merged if messages arising from critical issues are still active ?

bgavrilMS on 19 Sep 2016

Yes, exactly (Just note that blocker issues are more severe than critical issues, so obviously they will block the merge as well). I actually think it would be best if the blocking severity could be configured by the user, but I can definitely live with blocking the merge for critical and blocker issues only.

sinapis on 19 Sep 2016

I'll send this to the team that handles Pull Requests. Currently PR comments do not have priority and there is no API that I know of that allows to block a PR based on the comments in it. That being said we already have various branch policies, so maybe we could leverage one in this case.

bgavrilMS on 20 Sep 2016

Oh, and feel free to use the "smiley face" to send a smile / frown with your suggestions directly from VSTS. :)

bgavrilMS on 20 Sep 2016

Done!
How about breaking the build by the build task "SonarQube for MSBuild - End Analysis" when severe issues are found?

sinapis on 20 Sep 2016

That's a good idea, I will add it to the backlog.

bgavrilMS on 20 Sep 2016

👍1

Hope I'm not resurrecting a dead post, though I might have a similar use-case where I would like to prevent PR merges if some blocker issues or a quality gate failures exist.

Since SonarQube 6.7, with sonar-scanner 3.0 (integrated with MSBuild), we were able to break builds on a quality gate failure, with the exception of builds triggered by a PR. (I'm aware that sonar-scanner runs on 'issues mode' in PR builds.) This allowed us to pinpoint what caused the issue and quickly address it, should a build break on a quality gate violation.

However we have recently updated our Sonar server to 7.0, as well as updating the SonarQube analysis build tasks to utilize sonar-scanner 4.0, and it seems that once again the ability to break builds on quality gate failures is removed - unfortunately, this was the single most utilized feature on our side that allowed us to prevent PR merges that would introduce new issues on our codebase.

Is there any way to achieve such functionality with SonarQube 7.0 and sonar-scanner 4.0, if so I would like to know. Any help or suggestion would be appreciated.