argo workflows

苏品

2023-12-01

一.Argo简介

官网

参考

1. Argo子项目

Argo Workflows：基于容器的任务编排工具
Argo CD：基于GitOps声明的持续交付工具
Argo Events：事件驱动工具
Argo Rollouts：支持金丝雀以及蓝绿发布的应用渐进式发布工具

2. argo CRD

Argo Workflows是一个开源容器原生工作流引擎，用于在 Kubernetes 上编排并行作业。Argo 工作流作为 Kubernetes CRD（自定义资源定义）实现
Argo 定义的 CRD，先从其中的三级定义入手。概念上的从大到小分别为 WorkflowTemplate、Workflow、template，这些资源的命名有些相似，注意分辨。

参考1

参考2

Argo是Argo项目集合中的一个，Argo项目集合还包括Argo Workflow、Argo Rollout、Argo Event等项目, 是CNCF孵化级别的托管项目

二. Argo Workflows安装

1. 控制器

kubectl create ns argo
kubectl apply -n argo -f https://raw.githubusercontent.com/argoproj/argo-workflows/master/manifests/quick-start-postgres.yaml

# kubectl get po -n argo
NAME                                   READY   STATUS      RESTARTS   AGE
argo-server-574ddc66b-62rjc            1/1     Running     4          4h25m
minio                                  1/1     Running     0          4h25m
postgres-56fd897cf4-k8fwd              1/1     Running     0          4h25m
workflow-controller-77658c77cc-p25ll   1/1     Running     4          4h25m

##
argo-server是argo服务端
mino是进行制品仓库
postgres是数据库
workflow-controller是流程控制器

2. client端

Argo Workflows提供Argo CLI，其安装方式也非常简单

# Download the binary
curl -sLO https://github.com/argoproj/argo/releases/download/v3.0.0-rc4/argo-linux-amd64.gz

# Unzip
gunzip argo-linux-amd64.gz

# Make binary executable
chmod +x argo-linux-amd64

# Move binary to path
mv ./argo-linux-amd64 /usr/local/bin/argo

# version
argo version

# 主要命令
list      列出工作流
logs      查看工作流的日志
submit    创建工作流
watch     实时监听工作流
get       现实详细信息
delete    删除工作流
stop      停止工作流

3. 简单workflows

# workflows配置清单
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: hello-world-
  labels:
    workflows.argoproj.io/archive-strategy: "false"
spec:
  entrypoint: whalesay
  templates:
  - name: whalesay
    container:
      image: docker/whalesay:latest
      command: [cowsay]
      args: ["hello world"]

# 创建并观察
argo submit -n argo helloworld.yaml --watch

Name:                hello-world-9pw7v
Namespace:           argo
ServiceAccount:      default
Status:              Succeeded
Conditions:          
 Completed           True
Created:             Mon Mar 08 14:51:35 +0800 (10 seconds ago)
Started:             Mon Mar 08 14:51:35 +0800 (10 seconds ago)
Finished:            Mon Mar 08 14:51:45 +0800 (now)
Duration:            10 seconds
Progress:            1/1
ResourcesDuration:   4s*(1 cpu),4s*(100Mi memory)

STEP                  TEMPLATE  PODNAME            DURATION  MESSAGE
 ✔ hello-world-9pw7v  whalesay  hello-world-9pw7v  5s
 
# 查看状态
# argo list -n argo
NAME                STATUS      AGE   DURATION   PRIORITY
hello-world-9pw7v   Succeeded   1m    10s        0

# 查看日志
argo logs -n argo hello-world-9pw7v

三. Workflow

Workflow是Argo中最重要的资源，其主要有两个重要功能：
- 它定义要执行的工作流
- 它存储工作流程的状态
要执行的工作流定义在Workflow.spec字段中，其主要包括templates和entrypoint
在一个 Workflow 中，其 spec 中有一个名为 templates 的字段，在其中至少需要一个 template 作为其组成的任务

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: hello-world-  # Workflow的配置名称
spec:
  entrypoint: whalesay        # 解析whalesay templates
  templates:
  - name: whalesay            # 定义whalesay templates，和entrypoint保持一致
    container:                # 定义一个容器，输出"helloworld"
      image: docker/whalesay
      command: [cowsay]
      args: ["hello world"]

四. templates

templates是列表结构，主要分为两类：
- 定义具体的工作流, 定义具体的工作流有4种类别为:
  - Container
  - Script
  - Resource
  - Suspend
- 调用其他模板提供并行控制, 分为两类:
  - Steps
  - Dag

1. 定义工作量

1.1 Container

container是最常用的模板类型，它将调度一个container，其模板规范和K8S的容器规范相同 (如上面Workflow案例就是调用一个容器)

  - name: whalesay            
    container:                
      image: docker/whalesay
      command: [cowsay]
      args: ["hello world"]

1.2 Script

Script是Container的另一种包装实现，其定义方式和Container相同，只是增加了source字段用于自定义脚本

  - name: gen-random-int
    script:
      image: python:alpine3.6
      command: [python]
      source: |
        import random
        i = random.randint(1, 100)
        print(i)

脚本的输出结果会根据调用方式自动导出到{{tasks.<NAME>.outputs.result}}或{{steps.<NAME>.outputs.result}}中

1.3 Resource

Resource主要用于直接在K8S集群上执行集群资源操作，可以 get, create, apply, delete, replace, patch集群资源。如下在集群中创建一个ConfigMap类型资源：

  - name: k8s-owner-reference
    resource:
      action: create
      manifest: |
        apiVersion: v1
        kind: ConfigMap
        metadata:
          generateName: owned-eg-
        data:
          some: value

1.4 Suspend

Suspend主要用于暂停，可以暂停一段时间，也可以手动恢复，命令使用argo resume进行恢复。定义格式如下:

  - name: delay
    suspend:
      duration: "20s"

2. 调用其他模板

2.1 steps

在 Steps 中，--代表顺序执行，- 代表并行执行
Steps主要是通过定义一系列步骤来定义任务，其结构是"list of lists"，外部列表将顺序执行，内部列表将并行执行。如下：

  - name: hello-hello-hello
    steps:
    - - name: step1
        template: prepare-data
    - - name: step2a
        template: run-data-first-half
      - name: step2b
        template: run-data-second-half

其中step1和step2a是顺序执行，而step2a和step2b是并行执行

2.2 Dag

Dag主要用于定义任务的依赖关系，可以设置开始特定任务之前必须完成其他任务，没有任何依赖关系的任务将立即执行。如下：

  - name: diamond
    dag:
      tasks:
      - name: A
        template: echo
      - name: B
        dependencies: [A]
        template: echo
      - name: C
        dependencies: [A]
        template: echo
      - name: D
        dependencies: [B, C]
        template: echo

其中A会立即执行，B和C会依赖A，D依赖B和C

Dag案例

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: dag-diamond-
spec:
  entrypoint: diamond
  templates:
  - name: diamond
    dag:
      tasks:
      - name: A
        template: echo
        arguments:
          parameters: [{name: message, value: A}]
      - name: B
        dependencies: [A]
        template: echo
        arguments:
          parameters: [{name: message, value: B}]
      - name: C
        dependencies: [A]
        template: echo
        arguments:
          parameters: [{name: message, value: C}]
      - name: D
        dependencies: [B, C]
        template: echo
        arguments:
          parameters: [{name: message, value: D}]

  - name: echo
    inputs:
      parameters:
      - name: message
    container:
      image: alpine:3.7
      command: [echo, "{{inputs.parameters.message}}"]

3. 其他功能

3.1 判断

when用于判断

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: coinflip-
spec:
  entrypoint: coinflip
  templates:
  - name: coinflip
    steps:
    - - name: flip-coin
        template: flip-coin
    - - name: heads
        template: heads
        when: "{{steps.flip-coin.outputs.result}} == heads"
      - name: tails
        template: tails
        when: "{{steps.flip-coin.outputs.result}} == tails"

  - name: flip-coin
    script:
      image: python:alpine3.6
      command: [python]
      source: |
        import random
        result = "heads" if random.randint(0,1) == 0 else "tails"
        print(result)

  - name: heads
    container:
      image: alpine:3.6
      command: [sh, -c]
      args: ["echo \"it was heads\""]

  - name: tails
    container:
      image: alpine:3.6
      command: [sh, -c]
      args: ["echo \"it was tails\""]

3.2 循环

withItems用于循环

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: loops-
spec:
  entrypoint: loop-example
  templates:
  - name: loop-example
    steps:
    - - name: print-message
        template: whalesay
        arguments:
          parameters:
          - name: message
            value: "{{item}}"
        withItems:
        - hello world
        - goodbye world

  - name: whalesay
    inputs:
      parameters:
      - name: message
    container:
      image: docker/whalesay:latest
      command: [cowsay]
      args: ["{{inputs.parameters.message}}"]

3.3 input与output

通过input、output实现数据交互，即一个task的output可以作为另一个task的input
如下generate-parameter通过whalesay输出hello world到/tmp/hello_world.txt上并作为outputs输出。而print-message直接读取了generate-parameter outputs作为参数当作inputs

templates:
- name: output-parameter
  steps:
  - - name: generate-parameter
      template: whalesay
  - - name: consume-parameter
      template: print-message
      arguments:
        parameters:
        - name: message
          value: "{{steps.generate-parameter.outputs.parameters.hello-param}}"

- name: whalesay
  container:
    image: docker/whalesay:latest
    command: [sh, -c]
    args: ["echo -n hello world > /tmp/hello_world.txt"]
  outputs:
    parameters:
    - name: hello-param
      valueFrom:
        path: /tmp/hello_world.txt

- name: print-message
  inputs:
    parameters:
    - name: message
  container:
    image: docker/whalesay:latest
    command: [cowsay]
    args: ["{{inputs.parameters.message}}"]

3.4 Artifacts

通过input和output实现数据交互，对于数据比较大的，比如二进制文件，则可以通过Artifacts制品进行共享，这些制品可以是提前准备好的，比如已经存储在git仓库或者s3中，也可以通过任务生成制品供其他任务读取
如下generate-artifact任务完成后output输出一个名为hello-art的制品，这个制品会把/tmp/hello_world.txt这个文件打包后上传到制品库中，默认制品库可以通过configmap配置，通常是放在S3上

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: artifact-example-
spec:
  entrypoint: main
  templates:
  - name: main
    steps:
    - - name: generate-artifact
        template: whalesay
    - - name: consume-artifact
        template: print-message
        arguments:
          artifacts:
          - name: message
            from: "{{steps.generate-artifact.outputs.artifacts.hello-art}}"
  - name: whalesay
    container:
      image: docker/whalesay:latest
      command: [sh, -c]
      args: ["sleep 1; cowsay hello world | tee /tmp/hello_world.txt"]
    outputs:
      artifacts:
      - name: hello-art
        path: /tmp/hello_world.txt
  - name: print-message
    inputs:
      artifacts:
      - name: message
        path: /tmp/message
    container:
      image: alpine:latest
      command: [sh, -c]
      args: ["cat /tmp/message"]

4. 变量使用

首先在spec字段定义arguments，定义变量名message，其值是hello world，然后在templates字段中需要先定义一个inputs字段，用于templates的输入参数，然后在使用"{{}}"形式引用变量

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: hello-world-parameters-
spec:
  entrypoint: whalesay
  arguments:
    parameters:
      - name: message
        value: hello world
  templates:
    - name: whalesay
      inputs:
        parameters:
          - name: message
      container:
        image: docker/whalesay
        command: [ cowsay ]
        args: [ "{{inputs.parameters.message}}" ]

变量还可以进行一些函数运算
- filter：过滤
- asInt：转换为Int
- asFloat：转换为Float
- string：转换为String
- toJson：转换为Json

参考

filter([1, 2], { # > 1})
asInt(inputs.parameters["my-int-param"])
asFloat(inputs.parameters["my-float-param"])
string(1)
toJson([1, 2])

五. WorkflowTemplate

WorkflowTemplate相当于 Workflow 的模板库，和 Workflow 一样，也由template 组成。用户在创建完WorkflowTemplate 后，可以通过直接提交它们来执行 Workflow
WorkflowTemplate和template的区别：
- template只是Workflow中templates下的一个任务，当我们定义一个Workflow时，至少需要定义一个template
- WorkflowTemplate是驻留在集群中的Workflow的定义，它是Workflow的定义，因为它包含模板，可以从WorkflowTemplate内部或者集群上其他Workflow和WorkflowTemplate引用它们
在2.7版本后，WorkflowTemplate的定义和Workflow的定义一样，可以简单的将kind:Workflow改成kind:WorkflowTemplate

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: workflow-template-1
spec:
  entrypoint: whalesay-template
  arguments:
    parameters:
      - name: message
        value: hello world
  templates:
    - name: whalesay-template
      inputs:
        parameters:
          - name: message
      container:
        image: docker/whalesay
        command: [cowsay]
        args: ["{{inputs.parameters.message}}"]

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: workflow-template-hello-world-
spec:
  entrypoint: whalesay
  templates:
  - name: whalesay
    steps:                              # 引用模板必须在steps/dag/template下
      - - name: call-whalesay-template
          templateRef:                  # 应用模板字段
            name: workflow-template-1   # 引用的WorkflowTemplate名
            template: whalesay-template # WorkflowTemplate中具体的template名
          arguments:                    # 参数
            parameters:
            - name: message
              value: "hello world"

六. ClusterWorkflowTemplate

ClusterWorkflowTemplate创建的是一个集群范围内的WorkflowTemplate，其他workflow可以引用它,其他和WorkflowTemplate是一样的

apiVersion: argoproj.io/v1alpha1
kind: ClusterWorkflowTemplate
metadata:
  name: cluster-workflow-template-whalesay-template
spec:
  templates:
  - name: whalesay-template
    inputs:
      parameters:
      - name: message
    container:
      image: docker/whalesay
      command: [cowsay]
      args: ["{{inputs.parameters.message}}"]

引用

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: workflow-template-hello-world-
spec:
  entrypoint: whalesay
  templates:
  - name: whalesay
    steps:          
      - - name: call-whalesay-template
          templateRef:                  # 引用模板
            name: cluster-workflow-template-whalesay-template   # ClusterWorkflow名
            template: whalesay-template # 具体的模板名
            clusterScope: true          # 表示是ClusterWorkflow
          arguments:                    #  参数
            parameters:
            - name: message
              value: "hello world"

七. 实践案例

和jenkins流水线差不多, 拉代码->编译->构建镜像->上传镜像->部署

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  annotations:
    workflows.argoproj.io/description: |
      Checkout out from Git, build and deploy application.
    workflows.argoproj.io/maintainer: '@joker'
    workflows.argoproj.io/tags: java, git
    workflows.argoproj.io/version: '>= 2.9.0'
  name: devops-java 
spec:
  entrypoint: main
  arguments:
    parameters:
      - name: repo
        value: gitlab-test.coolops.cn:32080/root/springboot-helloworld.git
      - name: branch
        value: master
      - name: image
        value: registry.cn-hangzhou.aliyuncs.com/rookieops/myapp:202103101613
      - name: cache-image
        value: registry.cn-hangzhou.aliyuncs.com/rookieops/myapp
      - name: dockerfile
        value: Dockerfile
      - name: devops-cd-repo
        value: gitlab-test.coolops.cn:32080/root/devops-cd.git
      - name: gitlabUsername
        value: devops-argo
      - name: gitlabPassword
        value: devops123456
  templates:
    - name: main
      steps:
        - - name: Checkout
            template: Checkout
        - - name: Build
            template: Build
        - - name: BuildImage
            template: BuildImage
        - - name: Deploy
            template: Deploy
    # 拉取代码
    - name: Checkout
      script:
        image: registry.cn-hangzhou.aliyuncs.com/rookieops/maven:3.5.0-alpine
        workingDir: /work
        command:
        - sh
        source: |
          git clone --branch {{workflow.parameters.branch}} http://{{workflow.parameters.gitlabUsername}}:{{workflow.parameters.gitlabPassword}}@{{workflow.parameters.repo}} .
        volumeMounts:
          - mountPath: /work
            name: work
    # 编译打包  
    - name: Build
      script:
        image: registry.cn-hangzhou.aliyuncs.com/rookieops/maven:3.5.0-alpine
        workingDir: /work
        command:
        - sh
        source: mvn -B clean package -Dmaven.test.skip=true -Dautoconfig.skip
        volumeMounts:
          - mountPath: /work
            name: work
    # 构建镜像  
    - name: BuildImage
      volumes:
      - name: docker-config
        secret:
          secretName: docker-config
      container:
        image: registry.cn-hangzhou.aliyuncs.com/rookieops/kaniko-executor:v1.5.0
        workingDir: /work
        args:
          - --context=.
          - --dockerfile={{workflow.parameters.dockerfile}}
          - --destination={{workflow.parameters.image}}
          - --skip-tls-verify
          - --reproducible
          - --cache=true
          - --cache-repo={{workflow.parameters.cache-image}}
        volumeMounts:
          - mountPath: /work
            name: work
          - name: docker-config
            mountPath: /kaniko/.docker/
    # 部署  
    - name: Deploy
      script:
        image: registry.cn-hangzhou.aliyuncs.com/rookieops/kustomize:v3.8.1
        workingDir: /work
        command:
        - sh
        source: |
           git remote set-url origin http://{{workflow.parameters.gitlabUsername}}:{{workflow.parameters.gitlabPassword}}@{{workflow.parameters.devops-cd-repo}}
           git config --global user.name "Administrator"
           git config --global user.email "coolops@163.com"
           git clone http://{{workflow.parameters.gitlabUsername}}:{{workflow.parameters.gitlabPassword}}@{{workflow.parameters.devops-cd-repo}} /work/devops-cd
           cd /work/devops-cd
           git pull
           cd /work/devops-cd/devops-simple-java
           kustomize edit set image {{workflow.parameters.image}}
           git commit -am 'image update'
           git push origin master
        volumeMounts:
          - mountPath: /work
            name: work
  volumeClaimTemplates:
    - name: work
      metadata:
        name: work
      spec:
        storageClassName: nfs-client-storageclass
        accessModes: [ "ReadWriteOnce" ]
        resources:
          requests:
            storage: 1Gi

argo workflows

一.Argo简介

1. Argo子项目

2. argo CRD

二. Argo Workflows安装

1. 控制器

2. client端

3. 简单workflows

三. Workflow

四. templates

1. 定义工作量

1.1 Container

1.2 Script

1.3 Resource

1.4 Suspend

2. 调用其他模板

2.1 steps

2.2 Dag

3. 其他功能

3.1 判断

3.2 循环

3.3 input与output

3.4 Artifacts

4. 变量使用

五. WorkflowTemplate

六. ClusterWorkflowTemplate

七. 实践案例

相关阅读

相关文章

相关问答