SonarQube Code Analysis issues integration into Pull Requests
See also SonarQube documentation available from Analyzing with SonarQube Extension for VSTS/TFS
Goal: Let developers fix issues early
Team leads and managers spend time drilling into the SonarQube dashboard, setting up quality gates and monitoring technical debt. Have a look at the publicly available SonarQube dashboard for the Roslyn project to get an idea of the insights available.
However, as a developer, I don’t like to spend time looking at dashboards. I also don’t like it when my boss sends me an email about a “quality gate failure” and a list of static analysis issues I should resolve. I like to get feedback early and I like to get feedback as close as possible to where I work – in the IDE or source control.
For C# projects, you can use SonarLint for Visual Studio, which will configure your IDE to use the same analyzers as SonarQube uses. This is based on the work done in collaboration with SonarSource to fully integrate with any Roslyn based code analyzer.
When it comes to other languages, code analysis solutions are diverse, and not all analyzers have IDE integrations. The next best place to see analysis issues is in the code review. And SonarQube is good at abstracting away the technical details of the myriad of analyzers available – it just deals with rules and quality profiles. So we have worked on a feature that will inject code analysis comments identified by SonarQube directly into a Visual Studio Team Services pull request.
Code Analysis issues shown directly in Pull Requests (PR)
This paragraph assumes you are using the SonarQube build tasks for MSBuild. To find out more about these tasks, see this blog post.
When you configure a build definition to run as part of a Pull Request (PR), and the build definition uses the SonarQube tasks, we will now run SonarQube in incremental mode. As documented on the SonarQube website, an incremental analysis means that an analysis is performed, but the results are not stored in the SonarQube database. The analysis is performed only on changed code. The SonarQube build tasks will then add comments to the pull request for new static analysis issues, with respect to the Continuous Integration (CI) build, as another member of the team would have done.
After I fix the problem at line 326 I push a commit to the same branch. This will trigger another PR build which will mark the issue as resolved:
How to get all this working
We recommend that you use the same build definition with SonarQube analysis tasks for both CI and PR builds. When a CI build occurs, a full SonarQube analysis is triggered, the results are uploaded to the SonarQube database and the dashboard is updated. When a PR build occurs, SonarQube uses the last full analysis for the project as a baseline to identify issues that are new.
Other than that, you don’t need to do anything to enable it. It just works. If you’d like to know more about CI and PR builds, keep on reading. Otherwise, we hope this helps you, and please post feedback by raising issues on the VSTS task repo.
Continuous Integration (CI) and Pull Request (PR) Builds
Pull requests (PRs) can be used to create code reviews. Team members comment on the changes made, then the author can push further commits to their feature branch until the reviewers are happy and the pull requests can be completed.
Team Services allows you define extra policies around PRs – at the time of writing there are 3 policies available: run a build the PR changes, require that an work item is associated with the changes, and require that people actually review the changes.
Now, every time a PR is submitted, Team Services will perform a build, run tests, and run an incremental SonarQube analysis that will push code analysis issues to the Pull Request.
Feedback and Troubleshooting
Currently this feature works for the SonarQube build tasks for MSBuild, when using Git as version control. Integration with the Java build tasks is on our backlog.
It is also not available on TFS 2015 on-premise and will not ship with any of the updates, including TFS 2015 Update 3, since this update is focused on bug fixes. We do not recommend you run a SonarQube analysis in PR builds on TFS 2015 on-premise, because the build will perform a full analysis which leaves a mark in the project’s history, even if the PR is abandoned.
If you do find that something is not working for you, you can disable this feature by adding a build variable named “SQPullRequestBot” with the value “false”. In this case, the SonarQube analysis tasks will not run in PR builds.
We’d like to hear from you, and if you have a problem it helps to include detailed logs. To create these logs, set the build variable named “system.debug” to “true”. Raise issues and suggestions on the issues tab of the VSTS task repo.
For general SonarQube functionality, please raise questions on StackOverflow and add the “SonarQube” tag.