同时工作时如何在gitlab-runner上共享缓存
我有一个使用gitlab-ci测试的代码。 以前,我正在使用共享的gitlab-runner,一切都很好。
我最近迁移到了我公司托管的GitLab运营商,并且完全相同的代码不再通过Gitlab-Ci管道有效。 提出的错误是:
Executing "step_script" stage of the job script 00:05
Using docker image sha256:xxx for registry.gitlab.com/me/myproject/myimage with digest registry.gitlab.com/me/myproject/myimage@sha256:xxx ...
$ cd build
/bin/bash: line 125: cd: build: No such file or directory
我调查并发现有关共享缓存的问题。 的确,我的缓存是在同一阶段同时工作的不同作业之间共享的,但是在共享的跑步者中,这不是问题。 因此,我要切换Gitlab-Runner配置,以便没有并发作业(即并发= 1
在文件中/etc/gitlab-runner/config.toml
) ,导致没有缓存问题。 我得出的结论是,问题更多地是关于Gitlab-runner配置。 我当前的配置文件/etc/gitlab-runner/config.toml
is:
concurrent = 8
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "MY-RUNNER-NAME"
url = "https://gitlab.com/"
token = "MY-TOKEN"
executor = "docker"
[runners.custom_build_dir]
[runners.cache]
Shared = true
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
tls_verify = false
image = "IMAGE:latest"
privileged = true
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/cache"]
shm_size = 0
那么如何配置我的跑步者以允许并发作业共享其caches(或解决此冲突)?
在这里编辑
是我的gitlab-ci.yml
文件:
# GitLab-CI configuration file.
stages:
- prepare
- build
- test
.allConfigs:
parallel:
matrix:
- DISTRIBUTION: ['alpine']
COMPILER: ['gcc']
image: ${CI_REGISTRY_IMAGE}/${DISTRIBUTION}-${COMPILER}
cache:
key: ${DISTRIBUTION}-${COMPILER}-key
paths:
- build/
- external/
prepare:
stage: prepare
script:
- git reset --hard HEAD
- git clean -d -f -x
- mkdir build
- cd build
- cmake ../
- make update
extends: .allConfigs
build-exe:
stage: build
script:
- cd build
- cmake ../ #-DTESTING=OFF
- make main
- make install
extends: .allConfigs
test-integration:
stage: test
script:
- cd build
- cmake ../ -DTESTING=ON
- make integrationTest
extends: .allConfigs
test-unit:
stage: test
script:
- cd build
- cmake ../ -DTESTING=ON
- make unitTest
extends: .allConfigs
当2个测试作业同时执行时,提到的错误出现。
I have a code that I test using gitlab-ci.
Previously, I was using a shared gitlab-runner, and everything was fine.
I recently migrated to a gitlab-runner hosted by my company, and the exact same code is no longer valid through the gitlab-ci pipeline.
The error raised was:
Executing "step_script" stage of the job script 00:05
Using docker image sha256:xxx for registry.gitlab.com/me/myproject/myimage with digest registry.gitlab.com/me/myproject/myimage@sha256:xxx ...
$ cd build
/bin/bash: line 125: cd: build: No such file or directory
I investigate and found a problem about the cache shared.
Indeed, my cache is shared between different jobs working concurrently at the same stage, but on shared runner freely available it was not an issue.
So, I switch my gitlab-runner configuration in order to have no concurrent jobs (ie. concurrent = 1
in the file /etc/gitlab-runner/config.toml
), resulting no cache problem anymore.
I conclude that the issue is more about the gitlab-runner configuration.
My current configuration file /etc/gitlab-runner/config.toml
is:
concurrent = 8
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "MY-RUNNER-NAME"
url = "https://gitlab.com/"
token = "MY-TOKEN"
executor = "docker"
[runners.custom_build_dir]
[runners.cache]
Shared = true
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
tls_verify = false
image = "IMAGE:latest"
privileged = true
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/cache"]
shm_size = 0
So how to configure my runner to allow concurrent jobs to share their caches (or solve this conflict) ?
EDIT
Here is my gitlab-ci.yml
file:
# GitLab-CI configuration file.
stages:
- prepare
- build
- test
.allConfigs:
parallel:
matrix:
- DISTRIBUTION: ['alpine']
COMPILER: ['gcc']
image: ${CI_REGISTRY_IMAGE}/${DISTRIBUTION}-${COMPILER}
cache:
key: ${DISTRIBUTION}-${COMPILER}-key
paths:
- build/
- external/
prepare:
stage: prepare
script:
- git reset --hard HEAD
- git clean -d -f -x
- mkdir build
- cd build
- cmake ../
- make update
extends: .allConfigs
build-exe:
stage: build
script:
- cd build
- cmake ../ #-DTESTING=OFF
- make main
- make install
extends: .allConfigs
test-integration:
stage: test
script:
- cd build
- cmake ../ -DTESTING=ON
- make integrationTest
extends: .allConfigs
test-unit:
stage: test
script:
- cd build
- cmake ../ -DTESTING=ON
- make unitTest
extends: .allConfigs
The error mentioned appears when the 2 test jobs are executing concurrently.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论