Jenkins + SonarQube 代码省察

2023-10-18

字数统计: 2.8k字 | 阅读时长≈ 16分

安装 SonarQube 9.5

根据Sonar官方要求，Sonar 9.5 安装前需要做如下准备：

安装JDK，版本为 11
数据库只支持 PostgreSQL，Oracle 以及 MSSQ
Linux平台上，要求 vm.max_map_count 大于或等于 262144，fs.file-max 大于或等于 65536。

安装 JDK

创建 JDK 安装目录
1
mkdir -p /usr/local/java

解压 JDK

1	tar xf jdk-11.0.14_linux-x64_bin.tar.gz -C /usr/local/java/

创建 JAVA 环境变量配置

cat > /etc/profile.d/java.sh <<EOF 
#!/bin/bash
export JAVA_HOME=/usr/local/java/jdk-11.0.14
export CLASSPATH=\$CLASSPATH:\$JAVA_HOME/lib:\$JAVA_HOME/jre/lib
export PATH=\$JAVA_HOME/bin:\$JAVA_HOME/jre/bin:\$PATH:\$HOME/bin
EOF

# 刷新环境变量
source /etc/profile.d/java.sh

检查安装
1
java -version

安装 PostgreSQL

请参考文章 CentOS7 安装 PostgreSQL 13

安装 SonarQube

设置JVM线程数和文件句柄数

# 临时设置，只对当前会话生效
sysctl -w vm.max_map_count=262144
sysctl -w fs.file-max=65536
ulimit -n 65536
ulimit -u 65536

# 永久设置
cat > /etc/sysctl.d/99-sonarqube.conf <<EOF
vm.max_map_count=262144
fs.file-max=65536
EOF

sysctl -p

cat > /etc/security/limits.d/99-sonarqube.conf <<EOF
sonarqube   -   nofile   65536
sonarqube   -   nproc    65536
EOF

下载 SonarQube

1	wget https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-9.5.0.56709.zip

解压安装包

1
2
3

unzip sonarqube-9.5.0.56709.zip

mv sonarqube-9.5.0.56709 /usr/local/sonarqube

创建 sonarqube 用户，并修改 sonarqube 权限

1 2	useradd sonarqube chown -R sonarqube:sonarqube /usr/local/sonarqube/

创建 systemd 管理 sonarqube 服务文件

cat > /usr/lib/systemd/system/sonarqube.service <<EOF
[Unit]
Description=SonarQube service
After=syslog.target network.target

[Service]
Type=simple
User=sonarqube
Group=sonarqube
PermissionsStartOnly=true
ExecStart=/bin/nohup \\
  /usr/local/java/jdk-11.0.14/bin/java \\
  -Xms32m \\
  -Xmx32m \\
  -Djava.net.preferIPv4Stack=true \\
  -jar /usr/local/sonarqube/lib/sonar-application-9.5.0.56709.jar
StandardOutput=syslog
LimitNOFILE=131072
LimitNPROC=8192
TimeoutStartSec=5
Restart=always
SuccessExitStatus=143

[Install]
WantedBy=multi-user.target
EOF

启动并配置开机启动
1
systemctl enable --now sonarqube

查看启动日志

# tailf /usr/local/sonarqube/logs/sonar.log 
2022.07.20 09:54:20 INFO  app[][o.s.a.p.ProcessLauncherImpl] Launch process[[key='es', ipcIndex=1, logFilenamePrefix=es]] from [/usr/local/sonarqube/elasticsearch]: /usr/local/sonarqube/elasticsearch/bin/elasticsearch -Epath.conf=/usr/local/sonarqube/temp/conf/es
2022.07.20 09:54:20 INFO  app[][o.s.a.SchedulerImpl] Waiting for Elasticsearch to be up and running
2022.07.20 09:54:20 INFO  app[][o.e.p.PluginsService] no modules loaded
2022.07.20 09:54:20 INFO  app[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.transport.Netty4Plugin]
2022.07.20 09:54:26 INFO  app[][o.s.a.SchedulerImpl] Process[es] is up
2022.07.20 09:54:26 INFO  app[][o.s.a.p.ProcessLauncherImpl] Launch process[[key='web', ipcIndex=2, logFilenamePrefix=web]] from [/usr/local/sonarqube]: /usr/local/java/jdk1.8.0_202/jre/bin/java -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/usr/local/sonarqube/temp -Xmx512m -Xms128m -XX:+HeapDumpOnOutOfMemoryError -cp ./lib/common/*:/usr/local/sonarqube/lib/jdbc/h2/h2-1.3.176.jar org.sonar.server.app.WebServer /usr/local/sonarqube/temp/sq-process8125589691281982031properties
2022.07.20 09:54:43 INFO  app[][o.s.a.SchedulerImpl] Process[web] is up
2022.07.20 09:54:43 INFO  app[][o.s.a.p.ProcessLauncherImpl] Launch process[[key='ce', ipcIndex=3, logFilenamePrefix=ce]] from [/usr/local/sonarqube]: /usr/local/java/jdk1.8.0_202/jre/bin/java -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/usr/local/sonarqube/temp -Xmx512m -Xms128m -XX:+HeapDumpOnOutOfMemoryError -cp ./lib/common/*:/usr/local/sonarqube/lib/jdbc/h2/h2-1.3.176.jar org.sonar.ce.app.CeServer /usr/local/sonarqube/temp/sq-process4174613729094792747properties
2022.07.20 09:54:46 INFO  app[][o.s.a.SchedulerImpl] Process[ce] is up
2022.07.20 09:54:46 INFO  app[][o.s.a.SchedulerImpl] SonarQube is up

打开浏览器，访问 http://10.1.40.51:9000/ 端口

配置 PostgreSQL 数据库

su - postgres 
$ psql -U postgres
Password for user postgres: 

// 执行以下命令
CREATE DATABASE sonar;
CREATE USER sonar WITH  PASSWORD 'Sonar@123';
GRANT all ON sonar TO sonar;

编辑 sonarqube 配置文件 /usr/local/sonarqube/conf/sonar.properties

# User credentials.
# Permissions to create tables, indices and triggers must be granted to JDBC user.
# The schema must be created first.
sonar.jdbc.username=sonar
sonar.jdbc.password=Sonar@123

......

#----- PostgreSQL 9.6 or greater
# By default the schema named "public" is used. It can be overridden with the parameter "currentSchema".
#sonar.jdbc.url=jdbc:postgresql://localhost/sonarqube?currentSchema=my_schema
sonar.jdbc.url=jdbc:postgresql://127.0.0.1:5432/sonar?currentSchema=public

重启 SonarQube
1
systemctl restart sonarqube.service
查看服务启动日志
1
journalctl -xe -u sonarqube.service -f

配置 Jenkins 集成 SonarQube

配置 Jenkins

Jenkins 安装插件 SonarQube Scanner for Jenkins
生成 SonarQube Token，SonarQube 菜单：My Account -> Security 或者访问：http://10.1.40.51:9000/account/security/

生成之后记得复制并保存Token，不然页面刷新或者关闭后就无法查询到Token了。
添加 Jenkins 凭据，菜单：凭据 —> 系统 -> 全局凭据 -> 添加凭据，或者直接访问：http://10.1.30.91:8080/credentials/store/system/domain/_/newCredentials 添加凭据
- 凭据类型: Secret text
增加 SonarQube Server，菜单：管理Jenkins -> 系统设置，或者直接访问：http://10.1.30.91:8080/configure，找到 SonarQube servers 配置项

配置项说明：

配置项说明

Name Sonar服务名，按照自己习惯来即可

Server URL SonarQube Server 的主页地址

Sonar authentication token Sonar Token，选择已添加的凭据即可

配置项	说明
Name	Sonar服务名，按照自己习惯来即可
Server URL	SonarQube Server 的主页地址
Sonar authentication token	Sonar Token，选择已添加的凭据即可

配置 Job 使用 Scanner 扫描代码

这里的架构是基于 kubernetes 启动 Jenkins Agent 进行 CICD 的动作，完整的 Pipeline 代码如下:

pipeline {
  agent {
    kubernetes {
      cloud 'kubernetes-dev'
      slaveConnectTimeout 1200
      workspaceVolume hostPathWorkspaceVolume(hostPath: "/opt/jenkins/workspace", readOnly: false)
      yaml '''
apiVersion: v1
kind: Pod
spec:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: node-role.kubernetes.io/edge
            operator: DoesNotExist
          - key: build
            operator: In
            values:
            - "true"
  tolerations:
  - key: "groups"
    operator: "Equal"
    effect: "NoExecute"
    value: "vbaas"
  containers:
  - name: jnlp
    image: \'10.1.40.69/citools/jnlp:alpine\'
    imagePullPolicy: IfNotPresent
    args: [\'$(JENKINS_SECRET)\', \'$(JENKINS_NAME)\']
    volumeMounts:
    - mountPath: "/etc/localtime"
      name: "localtime"
      readOnly: false
  - name: "sonar-scanner"
    image: "10.1.40.69/citools/sonar-scanner-cli:4.7"
    imagePullPolicy: "IfNotPresent"
    command:
    - "cat"
    env:
    - name: "LANGUAGE"
      value: "en_US:en"
    - name: "LC_ALL"
      value: "en_US.UTF-8"
    - name: "LANG"
      value: "en_US.UTF-8"
    - name: "SONAR_HOST_URL"
      value: "http://10.1.40.51:9000"
    tty: true
    volumeMounts:
    - mountPath: "/etc/localtime"
      name: "localtime"
  - name: "build"
    image: "10.1.40.69/citools/maven:3.6.3-jdk-11"
    imagePullPolicy: "IfNotPresent"
    command:
    - "cat"
    tty: true
    volumeMounts:
    - mountPath: "/etc/localtime"
      name: "localtime"
    - mountPath: "/root/.m2/"
      name: "cachedir"
      readOnly: false
  - name: "kubectl"
    image: "10.1.40.69/citools/kubectl:self-1.17"
    imagePullPolicy: "IfNotPresent"
    command:
    - "cat"
    env:
    - name: "LANGUAGE"
      value: "en_US:en"
    - name: "LC_ALL"
      value: "en_US.UTF-8"
    - name: "LANG"
      value: "en_US.UTF-8"
    tty: true
    volumeMounts:
    - mountPath: "/etc/localtime"
      name: "localtime"
      readOnly: false
  - name: "docker"
    image: "10.1.40.69/citools/docker:19.03.9-git"
    imagePullPolicy: "IfNotPresent"
    command:
    - "cat"
    env:
    - name: "LANGUAGE"
      value: "en_US:en"
    - name: "LC_ALL"
      value: "en_US.UTF-8"
    - name: "LANG"
      value: "en_US.UTF-8"
    tty: true
    volumeMounts:
    - mountPath: "/etc/localtime"
      name: "localtime"
      readOnly: false
    - mountPath: "/var/run/docker.sock"
      name: "dockersock"
      readOnly: false
    - mountPath: /opt/jenkins/workspace
      name: "jenkins-workspace"
      readOnly: false
  - name: "alpine-curl"
    image: "10.1.40.69/publics/alpine-curl:latest"
    imagePullPolicy: "IfNotPresent"
    command:
    - "/bin/sh"
    - "-c"
    - "sleep 1200"
    env:
    - name: "LANGUAGE"
      value: "en_US:en"
    - name: "LC_ALL"
      value: "en_US.UTF-8"
    - name: "LANG"
      value: "en_US.UTF-8"
    volumeMounts:
    - mountPath: /opt/jenkins/workspace
      name: "jenkins-workspace"
      readOnly: false
    - mountPath: "/etc/localtime"
      name: "localtime"
      readOnly: true
  restartPolicy: "Never"
  volumes:
  - hostPath:
      path: "/var/run/docker.sock"
    name: "dockersock"
  - hostPath:
      path: "/usr/share/zoneinfo/Asia/Shanghai"
    name: "localtime"
  - name: "cachedir" 
    hostPath:
      path: "/opt/m2"
  - name: "jenkins-workspace"
    hostPath:
      path: "/opt/jenkins/workspace"
'''
    }
  }

  environment {
    TAG = ''
    COMMIT_ID = ''
    HARBOR_ADDRESS = '10.1.40.69'
    HARBOR_USER = credentials('HARBOR_ACCOUNT')
    REGISTRY_DIR = 'spring-boot-project'
    IMAGE_NAME = 'spring-boot-project'
    NAMESPACE = 'kubernetes-test'
    MY_KUBECONFIG = credentials('kubernetes-dev')
    REPO = 'ssh://git@gitlab.59izt.com/kubernetes/spring-boot-project.git'
    REPO_HTTP = 'http://gitlab.59izt.com/kubernetes/spring-boot-project.git'
    PROJECT_DIR = 'spring-boot-project'
    GIT_AUTH = 'gitlab-key'
    PROJECT_NAME = 'spring-boot-project'
    SOURCES_DIR = './src/main/java'
    LANGUAGE = 'java'
  }
  
  parameters {
    gitParameter(
      branch: '',
      branchFilter: 'origin/(.*)', 
      defaultValue: 'dev', 
      description: 'Branch for build and deploy', 
      name: 'BRANCH', 
      quickFilterEnabled: false, 
      selectedValue: 'NONE', 
      sortMode: 'NONE', 
      tagFilter: '*', 
      type: 'PT_BRANCH'
    )
    imageTag(
      name: "IMAGE_TAG", 
      description: '需要提测的镜像版本，构建发版开发环境时请忽略该参数',
      image: "publics/spring-boot-project", 
      filter: '.*', 
      defaultTag: '',
      registry: 'http://10.1.40.69', 
      credentialId: 'HARBOR_ACCOUNT', 
      tagOrder: 'DSC_VERSION'
    )
    choice(
      name: 'ACTION',
      description: '执行动作: \nbuild: 只构建镜像，不发版\ndeploy: 构建镜像并发布更新到开发环境\nsummit_the_test: 提交测试申请',
      choices: ['deploy', 'build', 'summit_the_test']
    )
    booleanParam(
      name:'SCANNER',
      defaultValue: false, 
      description: '是否执行代码扫描'
    )
  }

  stages {
    stage('Pulling Code') {
      when {
        expression { params.ACTION ==~ /(build|deploy)/ }
      }
      parallel {
        stage('Pulling Code by Jenkins') {
          when {
            expression {
              env.gitlabBranch == null
            }
          }
          steps {
            git(
              url: "${env.REPO}", 
              changelog: true, 
              poll: true, 
              branch: "${BRANCH}", 
              credentialsId: "${env.GIT_AUTH}"
            )
            script {
              COMMIT_ID = sh(
                returnStdout: true, 
                script: "git log -n 1 --pretty=format:'%h'"
              ).trim()
              TAG = BUILD_TAG + '-' + COMMIT_ID
              println "Current branch is ${BRANCH}, Commit ID is ${COMMIT_ID}, Image TAG is ${TAG}"
            }
          }
        }

        stage('Pulling Code by trigger') {
          when {
            expression {
              env.gitlabBranch != null
            }
          }
          steps {
            git(
              url: "${env.REPO}", 
              branch: env.gitlabBranch, 
              changelog: true, 
              poll: true, 
              credentialsId: "${env.GIT_AUTH}"
            )
            script {
              COMMIT_ID = sh(
                returnStdout: true,
                script: "git log -n 1 --pretty=format:'%h'"
              ).trim()
              TAG = BUILD_TAG + '-' + COMMIT_ID
              println "Current branch is ${env.gitlabBranch}, Commit ID is ${COMMIT_ID}, Image TAG is ${TAG}"
            }
          }
        }

      }
    }

    stage('Building') {
      when {
        expression { params.ACTION ==~ /(build|deploy)/ }
      }
      steps {
        container(name: 'build') {
          sh '''
            mvn -f ./vchain-gen/pom.xml clean package -U -B -DskipTests=true
          '''
        }

      }
    }

    stage('SonarQube analysis') {
      environment {
        SONAR_TOKEN = credentials('sonar-token')
      }
      when {
        expression { return params.SCANNER }
      }

      steps {
        container("sonar-scanner") {
          sh '''
            sonar-scanner \
              -Dsonar.login=${SONAR_TOKEN} \
              -Dsonar.projectKey=${PROJECT_NAME} \
              -Dsonar.projectName=${PROJECT_NAME} \
              -Dsonar.sourceEncoding=UTF-8 \
              -Dsonar.language=${LANGUAGE} \
              -Dsonar.sources=${SOURCES_DIR} \
              -Dsonar.java.binaries=.
          '''
        }
      }
    }

    stage('Docker build for creating image') {
      when {
        expression { params.ACTION ==~ /(build|deploy)/ }
      }
      steps {
        container(name: 'docker') {
          sh """
            cd ${PROJECT_DIR}
            echo ${HARBOR_USER_USR} ${HARBOR_USER_PSW} ${TAG}
            docker build -t ${HARBOR_ADDRESS}/${REGISTRY_DIR}/${IMAGE_NAME}:${TAG} .
            docker login -u ${HARBOR_USER_USR} -p ${HARBOR_USER_PSW} ${HARBOR_ADDRESS}
            docker push ${HARBOR_ADDRESS}/${REGISTRY_DIR}/${IMAGE_NAME}:${TAG}
          """
        }
      }
    }

    stage('Deploying to K8s') {
      when {
        expression { params.ACTION == 'deploy' }
      }
      steps {
        container(name: 'kubectl') {
          sh """
            /usr/local/bin/kubectl --kubeconfig ${MY_KUBECONFIG} \
              set image deploy -l \
              app=${IMAGE_NAME} \
              ${IMAGE_NAME}=${HARBOR_ADDRESS}/${REGISTRY_DIR}/${IMAGE_NAME}:${TAG} \
              -n $NAMESPACE
            /usr/local/bin/kubectl --kubeconfig=${MY_KUBECONFIG} \
              get pod -l app=${IMAGE_NAME} \
              -n ${NAMESPACE} -w
          """
        }
      }
    }

    stage('提交到测试') {
      when {
        expression { params.ACTION == 'summit_the_test' }
      }

      steps {
        container(name: 'alpine-curl') {
          wrap([$class: 'BuildUser']) {
            script {
              BUILD_USER = "${env.BUILD_USER}"
              IMAGE_TAG = "${params.IMAGE_TAG}"
              BUILD_TIMESTAMP = "${env.BUILD_TIMESTAMP}"
              sh """
                echo "发送提测消息到企业微信."
                curl 'https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' -H 'Content-Type: application/json' -d '
                {
                  "msgtype": "markdown",
                  "markdown": {
                    "content": "`项目 <font color=\'info\'>${IMAGE_NAME}</font> 已经提测，请相关同事注意。`\n>
                              > 提测项目名称: <font color=\'info\'>${IMAGE_NAME}</font>
                              > 提测代码仓库: [${IMAGE_NAME}](${env.REPO_HTTP})
                              > 提测代码分支: <font color=\'warning\'>${BRANCH}</font>
                              > 提测镜像版本: ${IMAGE_TAG}
                              > 提测镜像标签: <font color=\'warning\'>${IMAGE_TAG_TAG}</font>
                              > 提测镜像名称: <font color=\'warning\'>${IMAGE_TAG_IMAGE}</font>
                              > 提测申请人: <font color=\'info\'>${BUILD_USER}</font>
                              > 提测申请时间: <font color=\'comment\'>${BUILD_TIMESTAMP}</font>
                              > 测试环境部署: 确保nacos配置已经同步，请点击：[发布测试环境](http://10.1.30.91:8080/)"
                  }
                }'
              """
            }
          }
        }
      }
    }
  }
}

SonarQube 创建 Project，菜单: SonarQube 首页，Projects –> Create Project –> Manually

输入 Project Name 以及 Project Key，这里的 Project 名称和 key 要对应上面 Pipeline 中环境变量配置中的 PROJECT_NAME
创建 Jenkins 流水线，此处略；
测试运行流水线，然后查看 SonarQube 状态