Screwdriver: Sonarqube Pull Request flow is broken

[jithine]

What happened:

Screwdriver creates a new Sonar Project for each pull request. This is not ideal and does not enable a project to make use of Pull Request Analysis. Uses looses out on a unified Quality Gate Status check against Pipeline branch and Pull Requests. This also prevents Screwdriver from offering Pull Request Decorations

What you expected to happen:

1. When a Pull Request Analysis runs, Screwdriver Sonar Coverage book end must make use of Pull Request Analysis.
2. Screwdriver should not create a new Sonar Project for each Pull Request. We should rather use Pull Request Analysis](https://docs.sonarqube.org/latest/analysis/pull-request/) feature and use analysis data from Pull Request to populate coverage data on UI.

How to reproduce it:

Create a Pull Request and observer it's Sonar Project. It's a separate one from the Sonar Project for main repository.

References

1. https://docs.sonarqube.org/latest/analysis/pull-request/
2. https://docs.sonarqube.org/latest/analysis/pr-decoration/
3. Implementation references
   
   
   
   1. https://github.com/screwdriver-cd/coverage-sonar/blob/master/commands.txt#L11
   
   
   2. https://github.com/screwdriver-cd/coverage-sonar/blob/master/index.js#L215-L226
   
   
   3. https://github.com/screwdriver-cd/screwdriver/blob/master/plugins/coverage/token.js#L24-L28
   
   

[jithine]

This would also need https://github.com/screwdriver-cd/screwdriver/issues/2153 to provide Pipeline branch name in PR build

[renaud-vmg]

My opinion on the matter is that

• Almost all of the information needed for normal Sonarqube integration is available in the environment (pointed out by @jithine)
• Looking at the arguments/parameters required to invoke SonarQube integration in any way (manual, command line, gradle, fastlane, etc) are only missing the parent branch name (see #2153 as above)
• If not using the automated sonar step, then only #2153 is of any importance to the user

Stretch goal consideration:

• A PR build could have an annotation forcing Sonarqube project creation, but by default shall not create a new Sonarqube project.
  
  
  
  • The default shall be engine-level configurable
  
  
  • This should be made available since SD deployments may be using older variants of Sonarqube and/or non-enterprise deployments that may not have the PR feature available
  
  

[renaud-vmg]

Adding: we're also missing the "source of truth" project key, which is to say, the Parent Job's project key, which is likely known by the engine. We have a local workaround in that we set a certain meta value to tell Screwdriver which project to get the required key from, but it would be easier for integration to have something such as PR_PARENT_SQ_PROJECT_ID.

[jithine]

Tasks

1. Add SD_SONAR_ENTERPRISE=false (by default)
2. coverage-sonar command to check this variable, and if present and equals true, do the following
   
   
   
   1. pass projectKey=SD_PR_PARENT_JOB_ID in this API call https://github.com/screwdriver-cd/coverage-sonar/blob/master/commands.txt#L6
   
   
   2. Change project key here https://github.com/screwdriver-cd/coverage-sonar/blob/master/commands.txt#L11 to SD_PR_PARENT_JOB_ID
   
   
3. Pass these sonar params on PR SD_SONAR_OPTS="-Dsonar.pullrequest.key=$SD_PULL_REQUEST -Dsonar.pullrequest.branch=${PR_BRANCH_NAME#*/} -Dsonar.pullrequest.base=${PR_BASE_BRANCH_NAME#*/}"
4. Update build coverage fetch logic to fetch from PR analysis for PR jobs.

[jithine]

Pending design consideration is to determine whether we should use pipelineID as Sonar Project Key. Concerns are

1. Can a single pipeline contain multiple kinds of analysis, mono-repo ?
2. How can a job's coverage be fetched if pipelineId is project key ?
   
   
   
   1. Lets say there are 3 jobs for project which runs in different containers, Today UI uses build start/end times to pickup an analysis https://github.com/screwdriver-cd/coverage-sonar/blob/2ad56393a948b66176d350a13602390f33d12f9f/index.js#L128
   
   

Downside of not associating with project key is that Sonar Enterprise will count number of lines 3 times for above scenario.

@kumada626 thoughts ?

---

We can workaround concerns by providing users a hook (an env vairable to still create separate project key for each job. Eg: pipeline:id:job:id can be the Sonar projectKey if user sets an env variable SD_SONAR_JOB_PROJECT=true for a Job

[renaud-vmg]

Actually I've been thinking about this myself, and in this case I have to introduce a new concept: an SQ_Truth _map_.

There are use-cases where multiple projects, covering single parts of a given repository, have their own jobs, both for PR and Commit events (possibly separate jobs at certain points).

• In the case of multi-event jobs (~commit, ~pr) it becomes relatively easy and can be considered an "automatic" process where using an sd-cmd with auto-filled values can be used, and the source of truth is thus relayed
• In the case of single-event jobs (only ~commit or ~pr, not both at once ), we need to be able to specify that a given job is the source of truth for a given "string", which technically we can consider to be the job name. This is essentially what has been done in some projects under my control, as we specify an "SQ_SOURCE_OF_TRUTH" target and then use an sd-cmd to get the sonarqube project ID out of it. Streamlining this into a directly-invoked command without any other arguments than the target job name would simplify things.

The concept of "map" comes in as, essentially, the jobID would be (as an illustration) _keyed_ to the job_name (by default), and using a command with only a target job name as input would be akin to fetching a value from a map.

[jithine]

We will support both pipeline and job level granularity

[jithine]

Updated Proposal

1. Add SD_SONAR_ENTERPRISE=false or some config which is false by default to determine whether Sonar serve is enterprise or not.
2. coverage-sonar command to check this variable, and if present and equals true (Enterprise Sonar), do the following
   
   
   
   1. pass projectKey=SD_PIPELINE_ID in this API call https://github.com/screwdriver-cd/coverage-sonar/blob/master/commands.txt#L6 when generating Sonar Token
   
   
   2. Change project key here https://github.com/screwdriver-cd/coverage-sonar/blob/master/commands.txt#L11 to pipeline:SD_PIPELINE_ID
   
   
   3. Update build coverage fetch logic to fetch from PR analysis for PR jobs when under Enterprise mode.
   
   
   4. Expose job level annotation screwdriver.cd/coverageGranularity: pipeline/job with pipeline as default. This controls whether project key should be job or pipeline.
   
   
3. Job level Project Key to be of format. pipeline:ID:job:ID for Sonar Enterprise & job:ID for Sonar Community Edition.
4. If Sonar is not Enterprise Edition existing logic of separate Sonar Projects for Jobs to continue
5. Pass these sonar params on PR SD_SONAR_OPTS="-Dsonar.pullrequest.key=$SD_PULL_REQUEST -Dsonar.pullrequest.branch=${PR_BRANCH_NAME#*/} -Dsonar.pullrequest.base=${PR_BASE_BRANCH_NAME#*/}"

Changes in current implementation when using with Sonar Community Edition
~1. Project key changes from job:ID to pipeline:ID:job:ID~

[kumada626]

I think that the proposal of support both pipeline and job level granularity is reasonable.
However, why job level project key includes pipeline:ID?
I think it is enough only includes job:ID as current.
If job level project key changes to pipeline:ID:job:ID, there is no continuity with the past data event if using Sonar Community Edition?

[jithine]

@kumada626 makes sense. updated Point 3. to Keep existing behavior for Sonar Community Edition

[tkyi]

See blog post: https://blog.screwdriver.cd/post/626730108375465985/sonarqube-enterprise-edition-support