Blogpost

09.05.2019

EcoSystem

Continuous Delivery with Sonatype Nexus, Jenkins and the Cloudogu EcoSystem

In order to meet the growing demand to release new features at an increasingly faster pace, these features must be implemented faster and faster. But that's just one side of the coin. After all, these features have to be put into production as well. Often, deployments are made manually and prone to errors. They tie up resources, and they may take a long time. The solution is complete automation, which is called Continuous Delivery.

Johannes Schnatterer

Technical Lead

In order to meet the growing demand to release new features at an increasingly faster pace, these features must be implemented faster and faster. But that's just one side of the coin. After all, these features have to be put into production as well. Often, deployments are made manually and prone to errors. They tie up resources, and they may take a long time. The solution is complete automation, which is called Continuous Delivery. Sometimes a distinction is made between continuous delivery and Continuous Deployment, but this is a complex debate that is beside the main point of the present article.

Continuous delivery can have different meanings depending on the artifact type and your own security and privacy needs. Similarly, you can use different tools to achieve continuous delivery. The advantage of Jenkins Pipelines is that they enable very simple and modular implementation. This article demonstrates two practical examples of how to continuously deliver Java artifacts to Nexus repositories with a few lines of Groovy code.

  • The first example shows how an open source Java Library is first built and tested when pushed to a GitHub repository by Jenkins, analyzed using SonarCloud and then deployed to Maven Central (see Figure 1).
  • The second example comes from the enterprise context and shows how the Cloudogu EcoSystem can be used to automatically build and test a Java web application through Jenkins when pushed to a Git repository in SCM-Manager It is then analyzed by SonarQube before being finally deployed to a private Nexus repository (see Figure 2).

Although the infrastructures of the examples are different, in both cases nearly the same Jenkins pipeline code can be used.

Cloud Services Workflow

Continuous delivery to Maven Central

Below is an example of an open source Java library that shows how continuous delivery to Maven Central can be implemented. In less than 100 lines of Jenkins Pipeline Groovy DSL (scripted Syntax), a sophisticated continuous delivery pipeline (with Build, Unit and Integration Tests, static code analysis, and deployment) can be implemented in a Jenkinsfile. Listing 1 shows an example based on the Jenkinsfile of the command-bus library. For the sake of clarity, it has been shortened slightly for this article.

@Library('github.com/cloudogu/ces-build-lib@24c4f03')
import com.cloudogu.ces.cesbuildlib.*

node {

  Maven mvn = new MavenWrapper(this)

  catchError {

    stage('Checkout') {
      checkout scm
    }

    initMaven(mvn)

    stage('Build') {
      mvn 'clean install -DskipTests'
    }

    stage('Unit Test') {
      mvn 'test'
    }

    stage('Integration Test') {
      mvn 'verify -DskipUnitTests'
    }

    stage('Static Code Analysis') {
      def sonarQube = new SonarCloud(this, [sonarQubeEnv: 'sonarcloud.io-cloudogu'])

      sonarQube.analyzeWith(mvn)

      if (!sonarQube.waitForQualityGateWebhookToBeCalled()) {
        currentBuild.result ='UNSTABLE'
      }
    }

    stage('Deploy') {
      if (preconditionsForDeploymentFulfilled()) {

        mvn.useDeploymentRepository([id: 'ossrh', url: 'https://oss.sonatype.org/',
                                     credentialsId: 'mavenCentral-acccessToken', type: 'Nexus2'])

        mvn.setSignatureCredentials('mavenCentral-secretKey-asc-file', 'mavenCentral-secretKey-Passphrase')

        mvn.deployToNexusRepositoryWithStaging()
      }
    }
  }

  junit allowEmptyResults: true, testResults: '**/target/surefire-reports/TEST-*.xml, **/target/failsafe-reports/*.xml'

  mailIfStatusChanged(new Git(this).commitAuthorEmail)
}

boolean preconditionsForDeploymentFulfilled() {
  if (isBuildSuccessful() &&
      !isPullRequest() &&
      shouldBranchBeDeployed()) {
    return true
  } else {
    echo "Skipping deployment because of branch or build result: currentResult=${currentBuild.currentResult}, " +
      "result=${currentBuild.result}, branch=${env.BRANCH_NAME}."
    return false
  }
}

private boolean shouldBranchBeDeployed() {
  return env.BRANCH_NAME == 'master' || env.BRANCH_NAME == 'develop'
}

private boolean isBuildSuccessful() {
  currentBuild.currentResult == 'SUCCESS' &&
    // Build result == SUCCESS seems not to set be during pipeline execution.
    (currentBuild.result == null || currentBuild.result == 'SUCCESS')
}

void initMaven(Maven mvn) {

  if ("master".equals(env.BRANCH_NAME)) {

    echo "Building master branch"
    mvn.additionalArgs = "-DperformRelease"
    currentBuild.description = mvn.getVersion()
  }
}

Listing 1

The Jenkinsfile consists of five so-called stages:

  • Checkout – retrieves the code from the Git repository,
  • Build, Unit Test and Integration Test – builds the code using Maven and then performs Unit and Integration Tests,
  • Static Code Analysis – performs a static code analysis using SonarQube (here: the public instance SonarCloud) and checks the collected metrics against a defined quality goal
  • Deploy – deploys the artifacts to the Nexus repository (here: the "Maven Central” instance that is publicly provided by Sonatype) if no errors have occurred.

The pipeline uses the Jenkins Shared ces-build-lib. It contains reusable building blocks for Maven (as well as the Nexus repository), Git and SonarQube. Jenkins Shared Libraries provide the same advantages as you get from integrating libraries in regular application development. In particular, ces-build-lib handles the signing and staging at the time of deployment to Maven Central. It also initiates commenting on pull requests through SonarCloud. Thus, it handles most of the complexity.

All branches (including feature branches and pull requests) undergo automated quality control, but the deployments are limited to the more stable branches master and develop (based on Git Flow). Deployments are triggered by a push to them.

In order for the pipeline presented to run in Jenkins, the following must be present in Jenkins:

  • The "Pipeline: GitHub Groovy Libraries plugin. It loads the shared library on the fly from GitHub.
  • A configured SonarQube instance with the sonarcloud.io-cloudogu ID, where the URL, Authentication Token, and organization are configured for SonarCloud.In order for Jenkins to obtain SonarQube's asynchronously calculated Quality Gate status, a Webhook must be created in SonarCloud on the Jenkins instance under /sonarqube-webhook/. If SonarCloud is to be configured to comment on GitHub Issues, the SonarCloud Application for GitHub must be installed in the GitHub Organization.
  • A User Access Token for oss.sonatype.org (which is used to publish releases from the Central Staging Repository to Maven Central) as a Secret Text Credential mavenCentral-acccessToken.
  • The Private Key in the form of an ASC file (as the Secret File Credential mavenCentral-secretKey-asc-file). This will be used to sign the artifacts for Maven Central.
  • The passphrase for the Private Key (in the form of the Secret Text Credential mavenCentral-secretKey-Passphrase). The details are available in the Maven Central documentation.

Continuous Delivery in an On-Premises Nexus repository

Cloudogu EcoSystem Workflow

The same workflow used for the open source library can also be deployed in the enterprise environment on your own infrastructure. Figure 2 shows a possible tool selection: SCM-Manager instead of GitHub, your own Nexus repository instead of Maven Central, and SonarQube instead of SonarCloud. These tools and many more can be obtained, for example, fully automated and preconfigured from the Cloudogu EcoSystem at the push of a button. They can be accessed with a single sign-on.

In this environment, virtually the same Jenkins pipeline can be used as in the first example. This is demonstrated by the Jenkinsfile from the second example cloudogu/spring-petclinic (See Listing 2). This is a prominent example of a Java web application that has been extended to include continuous delivery by a Jenkinsfile.

@Library('github.com/cloudogu/ces-build-lib@24c4f03')
import com.cloudogu.ces.cesbuildlib.*

properties([
        disableConcurrentBuilds()
])

node {

    String cesFqdn = findHostName()
    String cesUrl = "https://${cesFqdn}"

    Maven mvn = new MavenWrapper(this)

    catchError {

        stage('Checkout') {
            checkout scm
        }

        stage('Build') {
            mvn 'clean package -DskipTests'
        }

        String jacoco = "org.jacoco:jacoco-maven-plugin:0.8.1"
        parallel(
                test: {
                    stage('Unit Test') {
                        mvn "${jacoco}:prepare-agent test ${jacoco}:report"
                    }
                },
                integrationTest: {
                    stage('Integration Test') {
                        mvn "${jacoco}:prepare-agent-integration failsafe:integration-test failsafe:verify ${jacoco}:report-integration"
                    }
                }
        )

        stage('Static Code Analysis') {
            def sonarQube = new SonarQube(this, [usernamePassword: 'jenkins-sonar', sonarHostUrl: "${cesUrl}/sonar"])

            sonarQube.analyzeWith(mvn)
        }

        stage('Deploy') {
            mvn.useDeploymentRepository([id: cesFqdn, url:  "${cesUrl}/nexus", credentialsId: 'jenkins-nexus', type: 'Nexus3'])

            mvn.deployToNexusRepository()
        }
    }

    junit allowEmptyResults: true, testResults: '**/target/failsafe-reports/TEST-*.xml,**/target/surefire-reports/TEST-*.xml'
}

Listing 2

Each project is different, so the pipelines also demonstrate smaller differences. Compared to the first example, there are the following relevant differences:

  • The URLs to SonarQube and Nexus are relative to Jenkins (since they are easier to maintain thanks to the Cloudogu EcoSystem).
  • Unit and Integration Tests run concurrently for faster feedback.
  • The Deploy Stage, unlike Maven Central, does not use a staging repository.

In general, this example is a bit simpler, so not as much is required to be present in Jenkins to run the pipeline:

  • The "Pipeline: GitHub Groovy Libraries" plugin, just like in the first example.
  • The Username With Password Credential jenkins-sonar for a technical user in SonarQube.
  • The Username With Password Credential jenkins-nexus for a technical user in Nexus.

Conclusion

This article shows how easy continuous delivery of Java artifacts to Nexus repositories can be. In each case, the desired tools can be used in different applications, such as Open Source and Enterprise. Of course, quality assurance can also be extended, for example by running a vulnerability analysis with Sonatype Lifecycle or load tests – even DevSecOps. Regardless of the specific tool selected, Jenkins Pipelines provide elegant and clear continuous delivery.

Tags