tekton 发布 kubernetes 应用

tekton 发布 kubernetes 应用

基于Kubernetes 服务部署 Tekton Pipeline 实例，部署完成后使用tekton来完成源码拉取、应用打包、镜像推送和应用部署。

本文实现一个 golang-helloworld 项目 CI/CD 的完整流程，具体包括以下步骤：

• 从 gitee 仓库拉取代码，将源码构建成二进制文件
• 根据 Dockerfile 构建镜像并推送到阿里云ACR镜像仓库
• 使用sed命令替换yaml文件中的镜像地址为上一步构建的镜像
• 使用 kubectl apply -f 命令部署yaml文件到kubernetes集群

示例git仓库：https://gitee.com/willzhangee/tekton-golang-demo

创建serviceaccount

镜推送到外部镜像仓库需要进行认证，创建登录阿里云ACR仓库的secret

kubectl create secret docker-registry aliyun-acr \
--docker-server=https://registry.cn-shenzhen.aliyuncs.com \
--docker-username=<your-username> \
--docker-password=<your-password> \
--dry-run=client -o json | jq -r '.data.".dockerconfigjson"' | base64 -d > /tmp/config.json

kubectl create secret generic docker-config --from-file=/tmp/config.json

创建kubernetes secret

kubectl create secret generic kubernetes-config --from-file=/root/.kube/config

创建serviceAccount

$ cat serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: build-bot
secrets:
  - name: docker-config
  - name: kubernetes-config
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: tekton-kubectl-role
rules:
- apiGroups:
  - "*"
  resources:
  - pods
  - deployments
  - deployments/scale
  - deployments/status
  verbs:
  - get
  - list
  - watch
  - create
  - delete
  - patch
  - update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: tekton-kubectl-binding
subjects:
- kind: ServiceAccount
  name: build-bot
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: tekton-kubectl-role

应用yaml文件

kubectl apply -f serviceaccount.yaml

创建 git-clone task

在执行镜像构建前Dockerfile存放在git仓库中，需要将代码克隆到本地，需要安装git-clone task，这里使用官方task。

kubectl apply -f \
https://raw.githubusercontent.com/tektoncd/catalog/main/task/git-clone/0.9/git-clone.yaml

创建kaniko-build task

创建kaniko-build task，用于构建dokcer镜像，基于官方kaniko-task改造。

$ cat kaniko-build-task.yaml
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: kaniko-build
spec:
  params:
    - name: IMAGE_URL
      description: Name (reference) of the image to build.
    - name: IMAGE_TAG
      description: Tag to apply to the built image
      default: latest
    - name: DOCKERFILE
      description: Path to the Dockerfile to build.
      default: ./Dockerfile
    - name: CONTEXT
      description: The build context used by Kaniko.
      default: ./
    - name: EXTRA_ARGS
      type: array
      default: []
    - name: BUILDER_IMAGE
      description: The image on which builds will run (default is v1.5.1)
      default: gcr.io/kaniko-project/executor:v1.5.1@sha256:c6166717f7fe0b7da44908c986137ecfeab21f31ec3992f6e128fff8a94be8a5
  workspaces:
    - name: source
      description: Holds the context and Dockerfile
    - name: dockerconfig
      description: Includes a docker `config.json`
      optional: true
      mountPath: /kaniko/.docker
  results:
    - name: IMAGE_DIGEST
      description: Digest of the image just built.
    - name: IMAGE_URL
      description: URL of the image just built.
  steps:
    - name: build-and-push
      workingDir: $(workspaces.source.path)
      image: $(params.BUILDER_IMAGE)
      args:
        - $(params.EXTRA_ARGS)
        - --dockerfile=$(params.DOCKERFILE)
        - --context=$(workspaces.source.path)/$(params.CONTEXT)
        - --destination=$(params.IMAGE_URL):$(params.IMAGE_TAG)
        - --digest-file=$(results.IMAGE_DIGEST.path)
      securityContext:
        runAsUser: 0

应用yaml文件

kubectl apply -f kaniko-build-task.yaml

创建kubernetes-deploy task

创建kubernetes-deploy task，用于部署yaml文件到kubernetes集群。

$ cat kubernetes-deploy-task.yaml
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: kubernetes-deploy
spec:
  workspaces:
    - name: manifest-dir
    - name: kubeconfig-dir
      mountPath: /root/.kube
  params:
    - name: pathToYamlFile
      description: The path to the yaml file to deploy within the git source
      default: deployment.yaml
    - name: IMAGE_URL
    - name: IMAGE_TAG
    - name: KUBECTL_IMAGE
      default: docker.io/bitnami/kubectl:latest
  steps:
    - name: run-kubectl
      image: $(params.KUBECTL_IMAGE)
      workingDir: $(workspaces.manifest-dir.path)
      script: |
        sed -i s#IMAGE#$(params.IMAGE_URL)#g $(params.pathToYamlFile)
        sed -i s#TAG#$(params.IMAGE_TAG)#g $(params.pathToYamlFile)
        kubectl apply -f $(params.pathToYamlFile)

应用yaml文件

kubectl apply -f kubernetes-deploy-task.yaml

创建pipeline和pipelinerun

$ cat pipeline-run.yaml
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: devops-hello-world-pipeline
spec:
  workspaces:
    - name: shared-data
    - name: docker-config
    - name: kubernetes-config
  params:
    # git-clone
    - name: git_url
      type: string
    - name: revision
      type: string
    - name: gitInitImage
      type: string
    # kaniko-build
    - name: dockerfile
      type: string
      description: reference of the image to build
    - name: builder_image
      type: string
      description: reference of the image to build
    - name: image_url
      description: Url of image repository
    - name: image_tag
      description: Tag to apply to the built image
      default: latest
    # kubernetes-deploy
    - name: kubectl_image
      type: string
  tasks:
    - name: clone
      taskRef:
        name: git-clone
      workspaces:
        - name: output
          workspace: shared-data
      params:
        - name: url
          value: $(params.git_url)
        - name: revision
          value: $(params.revision)
        - name: gitInitImage
          value: $(params.gitInitImage)
    - name: build-push-image
      params:
      - name: DOCKERFILE
        value: $(params.dockerfile)
      - name: IMAGE_URL
        value: $(params.image_url)
      - name: IMAGE_TAG
        value: $(tasks.clone.results.commit)
      - name: BUILDER_IMAGE
        value: $(params.builder_image)
      taskRef:
        name: kaniko
      runAfter:
        - clone
      workspaces:
        - name: source
          workspace: shared-data
        - name: dockerconfig
          workspace: docker-config
    - name: deploy-to-k8s
      taskRef:
        name: kubernetes-deploy
      params:
        - name: KUBECTL_IMAGE
          value: $(params.kubectl_image)
        - name: IMAGE_URL
          value: $(params.image_url)
        - name: IMAGE_TAG
          value: $(tasks.clone.results.commit)
        - name: pathToYamlFile
          value: deployment.yaml
      workspaces:
        - name: manifest-dir
          workspace: shared-data
        - name: kubeconfig-dir
          workspace: kubernetes-config
      runAfter:
        - build-push-image
---
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  generateName: devops-hello-world-pipeline-run-
spec:
  serviceAccountName: build-bot
  pipelineRef:
    name: devops-hello-world-pipeline
  params:
    # git-clone
    - name: git_url
      value: https://gitee.com/willzhangee/tekton-golang-demo.git
    - name: revision
      value: master
    - name: gitInitImage
      #value: gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git-init:latest
      value: dyrnq/tektoncd-pipeline-cmd-git-init:latest
    # kaniko
    - name: dockerfile
      value: ./Dockerfile
    - name: builder_image
      # value: gcr.io/kaniko-project/executor:v1.5.1@sha256:c6166717f7fe0b7da44908c986137ecfeab21f31ec3992f6e128fff8a94be8a5
      value: docker.io/bitnami/kaniko:latest
    - name: image_url
      value: registry.cn-shenzhen.aliyuncs.com/cnmirror/devops-hello-world
    - name: image_tag
      value: latest
    # kubernetes-deploy
    - name: kubectl_image
      value: 'docker.io/bitnami/kubectl:latest'
  workspaces:
    - name: shared-data
      volumeClaimTemplate:
        spec:
          accessModes:
          - ReadWriteOnce
          storageClassName: openebs-hostpath
          resources:
            requests:
              storage: 1Gi
    - name: docker-config
      secret:
        secretName: docker-config
    - name: kubernetes-config
      secret:
        secretName: kubernetes-config

参数说明：

• gitInitImage：执行git clone任务的镜像，官方镜像无法访问，推荐在docekrhub中查找替代镜像
• builder_image：执行kaniko 构建任务的镜像，官方镜像无法访问，推荐在docekrhub中查找替代镜像
• image_url：最终构建的应用镜像
• serviceAccountName：指定serviceAccountName用于认证
• shared-data workspace：用于在不同任务之间共享数据，PipelineRun中定义了volumeClaimTemplate类型的workspaces，能够动态申请所需的持久卷，使用kubectl get storageclass命令，确认k8s集群有默认可用的storageclass资源可用，本示例输出为openebs-hostpath (default)
• docker-config workspace：用于镜像仓库认证的secret卷，将secret中的config.json挂载到/kaniko/.docker下
• kubernetes-config：用于访问kubernetes，挂载到/root/.kube目录下

应用yaml文件

kubectl create -f pipeline-run.yaml

查看pipelinerun执行结果

连接到kubernetes 确认部署的应用

root@kube001:~# kubectl get pods -l run=go-web-app
NAME                          READY   STATUS    RESTARTS   AGE
go-web-app-79454cfdd7-dcz7p   1/1     Running   0          64s

查看镜像信息

root@kube001:~# kubectl get pods go-web-app-79454cfdd7-dcz7p -o jsonpath='{.spec.containers[0].image}'
registry.cn-shenzhen.aliyuncs.com/cnmirror/devops-hello-world:927ec5cc665690ad798ffbbd02a8db520692951e

参考：https://juejin.cn/post/7073347226772340749