Running the Sonar Build Wrapper and Scanner on different hosts in Azure Pipelines

Photo by James Harrison on Unsplash

When using SonarQube or SonarCloud to analyze a C/C++/Objective-C project, the Build Wrapper is a required component that is executed as a prefix to the build command. After the build has finished, a build-wrapper-dump.json file is generated in the Build Wrapper output directory. This file contains absolute file paths (source files, standard headers, libraries, etc…) which SonarScanner needs access to while running. On account of this, both the Build Wrapper and SonarScanner binaries must run on the same host.

In Azure Pipelines, there are some special scenarios where we may want to run them on different hosts:

• The project demands a powerful machine for building which is expensive to use for anything other than building. It’s better to use a less powerful machine to run lightweight tasks such as static code analysis.
• The pipeline is broken into multiple stages, for example: Build Stage and Static Analysis Stage. The Build Wrapper binary is run in the Build Stage and the SonarScanner binary is run in the Static Analysis Stage each on a different machine.

In the next section, we’ll have a look at the azure-storage-cpp-sas-sample project to learn about how SonarCloud is implemented in a multi-stage pipeline.

Build Pipeline Diagram

• Invoke Build Wrapper (Build Stage): Generates the build-wrapper-dump.json file.
• Upload build-wrapper-dump.json (Build Stage): Publishes the build-wrapper-dump.json file as an artifact.
• Download build-wrapper-dump.json (Static Analysis Stage): Downloads the build-wrapper-dump.json file.
• Preprocess build-wrapper-dump.json (Static Analysis Stage): Updates the absolute paths in the json file. This is an important step and without it SonarScanner won’t be able to locate the source files and will throw the following exception:
  java.lang.IllegalStateException: The “build-wrapper-dump.json” file was found but 0 C/C++/Objective-C files were analyzed.
  We’ll discuss the workaround in the next section.
• Invoke SonarScanner (Static Analysis Stage): Finally, specifies the path to the build-wrapper-dump.json file and invokes SonarScanner to analyze the project.

Preprocess the build-wrapper-dump.json file

There are two places in the file that need to be modified:

1. The absolute paths of the source files containing the old workspace: In the picture above, the source files were checked out to …/agent1/_work/1/s in the Build Stage. That info needs to be changed to the new location of the source files being checked out in the Static Analysis Stage, for example …/agent2/_work/2/s.
2. cwd (current working directory): These are the build directories where the compiler was invoked. They’re created by the build command so they don’t exist in the new workspace (Static Analysis Stage). When SonarScanner runs, it looks for those directories so we need to create them.

Here is an example bash script that does all replacements:

# Replace source files path
wrapperFile=build-wrapper-dump.json
workspaceRegex="\/.*\/_work\/[0-9]*"
currentWorkspace=$(echo "$(Pipeline.Workspace)" | sed 's/\//\\\//g')
sed -i "s@$workspaceRegex@$currentWorkspace@g" $wrapperFile
# Create current working dirs
for cwd in `grep -oP '\"cwd\"\:\".*\"' $wrapperFile | sort -u`; do
  cwd=${cwd:7:-1}
  mkdir -p $cwd
done

At the time of writing, this workaround seems to be fine with Build Wrapper v6.17 and SonarScanner v4.4.0.2170. It may break in the future if either the format of the build-wrapper-dump.json file or SonarScanner’s behavior changes. Please keep this in mind if you’re going to try it in your production environment.