Gitlab运行程序在阶段完成后缓存未命中文件

Question

摘要

我的gitlab-ci.yml有3个阶段用于将应用程序部署到okd pod应用程序在tomcat上运行Spring启动： 8有时，cache.zip在阶段完成后没有更新，因此下一步无法正确运行

复制的步骤

我的gitlab ci运行以下阶段

阶段1：运行测试编译---

阶段2：打包war文件作为部署的输出---

第3阶段：将war文件部署到pod---

gitlab ci。yml

image: openshift/origin-cli

stages:
  - build
  - test
  - staging

cache:
  paths:
    - .m2/repository
    - target
    - artifact

validate:jdk8:
  stage: build
  script:
    - 'mvn test-compile'
  only:
    - master
  image: maven:3.3.9-jdk-8

verify:jdk8:
  stage: test
  script:
    - 'mvn verify'
    - 'mvn package' # =====> this command generate war file
  only:
    - master
  image: maven:3.3.9-jdk-8

staging:
  script:
    - "mkdir -p artifact"
    - "cp ./target/*.war ./artifact/" # ======> Sometimes error at this line because of previous step not add war file into cache
    - "oc start-build $APP"
    - "rm -rf ./target/* && rm -rf ./artifact/*" # Remove war & class file, only cache m2 lib
  stage: staging
  variables:
    APP: $CI_PROJECT_NAME
  environment:
    name: staging
    url: http://$CI_PROJECT_NAME-staging.$OPENSHIFT_DOMAIN
  only:
    - master

实际行为有时缓存在测试阶段完成后没有war文件（这取决于war文件大小吗？）

在测试阶段之后，升级阶段部署的预期行为War文件更新到缓存中

相关日志和/或截图

截图

工作日志

Running with gitlab-runner 13.7.0 (943fc252)
  on gitlab-runner-node1 y6awygsj
Preparing the "docker" executor
00:01
Using Docker executor with image openshift/origin-cli ...
Using locally found image version due to if-not-present pull policy
Using docker image sha256:7ebb6be01117a50344d63f77c385a13302afecd33480b97c36a518d4f5ebc25a for openshift/origin-cli with digest docker.io/openshift/origin-cli@sha256:509e052d0f2d531b666b7da9fa49c5558c76ce5d286456f0859c0a49b16d6bf2 ...
Preparing environment
00:00
Running on runner-y6awygsj-project-489-concurrent-0 via gitlab.runner.node1...
Getting source from Git repository
00:01
Fetching changes...
Reinitialized existing Git repository in /builds/my-project/.git/
Checking out b4c97428 as master...
Removing .m2/
Removing artifact/
Removing target/
Skipping Git submodules setup
Restoring cache
00:05
Checking cache for default-23...
No URL provided, cache will not be downloaded from shared cache server. Instead a local version of cache will be extracted. 
Successfully extracted cache
Executing "step_script" stage of the job script
00:01
$ mkdir -p artifact
$ cp ./target/*.war ./artifact/
cp: cannot stat './target/*.war': No such file or directory
Cleaning up file based variables
00:00
ERROR: Job failed: exit code 1

环境描述

配置。汤姆

concurrent = 1
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "gitlab-runner-node1"
  url = "https://gitlab.mycompany.vn/"
  token = "y6awygsj9zks18nU6PDt"
  executor = "docker"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
    [runners.cache.azure]
  [runners.docker]
    dns = ["192.168.100.1"]
    tls_verify = false
    image = "alpine:latest"
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/mnt/nfs/nfsshare-gitlab/cache:/cache"]
    shm_size = 0
    pull_policy = "if-not-present"

使用GitLab Runner版本

版本：13.7.0

Git版本：943fc252

Git分支：13-7-稳定

GO版本：go1。13.8

建成：2020-12-21T13:47:06 0000

OS/Arch:linux/amd64

可能的解决办法

重新运行测试阶段，直到缓存中有war文件

Answer 1

让我们一步一步走。

首先，关于如何管理阶段之间的文件。

的确，如果两个作业和阶段都在同一个环境中运行，您可以直接访问作业和阶段之间的文件，但情况并非总是如此（即使两个运行程序都使用相同的nfs共享目录），因此您应该使用工件。

在作业中定义工件时，您指定的是作业成功、失败或始终附加到作业的文件列表，具体取决于您的配置。

默认情况下，来自前几个阶段的所有工件都会传递给每个作业，但在任何情况下，您都可以使用依赖项来定义要从哪些作业中提取工件。

所以基本上你应该使用下面的。gitlab ci。yml

image: openshift/origin-cli

stages:
  - build
  - test
  - staging

cache:
  paths:
    - .m2/repository

validate:jdk8:
  stage: build
  script:
    - 'mvn test-compile'
  only:
    - master
  image: maven:3.3.9-jdk-8

verify:jdk8:
  stage: test
  script:
    - 'mvn verify' # =====> verify already includes: validate, compile, test and package
  artifacts:
    paths:
      - target/[YOUR_APP_NAME].war
  only:
    - master
  image: maven:3.3.9-jdk-8

staging:
  dependencies:
    - verify:jdk8
  script:
    - "mkdir -p artifact"
    - "cp ./target/[YOUR_APP_NAME].war ./artifact/"
    - "oc start-build $APP"
  stage: staging
  variables:
    APP: $CI_PROJECT_NAME
  environment:
    name: staging
    url: http://$CI_PROJECT_NAME-staging.$OPENSHIFT_DOMAIN
  only:
    - master

另外，请注意，我删除了mvn包指令。我建议您了解一下Maven的构建生命周期基础知识。