Gitlab CI规则不使用扩展和个人规则

发布于 2025-02-06 05:11:52 字数 7702 浏览 2 评论 0原文

以下是建筑阶段的两个工作。 默认情况下,设置了一些常见条件,并且使用扩展关键字为此,ifawsdeploy。 由于其中只有一个应该运行,如果提供了变量$ admin_server_ip,则应运行Connect_Admin_Server,以这种方式工作。 如果没有向$ admin_server_ip提供的值,则create_admin_server不应运行,但不运行。

.ifawsdeploy:
  rules:
    - if: '$TEST_CREATE_ADMIN && $REGION && $ROLE_ARN && $PACKAGEURL && $TEST_CREATE_ADMIN == "aws" && $SUB_PLATFORM == "aws" && $ROLE_ARN != "" && $PACKAGEURL != "" && $REGION != ""'

variables:
 TEST_CREATE_ADMIN:
   #value: aws
   description: "Platform, currently aws only"
 SUB_PLATFORM:
   value: aws
   description: "Platform, currently aws only"
 REGION:
   value: "us-west-2"
   description: "region where to deploy company"
 PACKAGEURL:
   value: "http://somerpmurl.x86_64.rpm"
   description: "company rpm file url"
 ACCOUNT_NAME:
   value: "testsubaccount"
   description: "Account name of sub account to refer in the deployment, no need to match in AWS"
 ROLE_ARN:
   value: "arn:aws:iam::491483064167:role/uat"
   description: "ROLE ARN of the user account assuming: aws sts get-caller-identity"
 tfenv_version: "1.1.9"
 DEV_PUB_KEY:
   description: "Optional public key file to add access to admin server" 
 ADMIN_SERVER_IP:
   description: "Existing Admin Server IP Address"
 ADMIN_SERVER_SSH_KEY:
   description: "Existing Admin Server SSH_KEY PEM content"
  
#export variables below will cause the terraform to use the root account instead of the one specified in tfvars file
.configure_aws_cli: &configure_aws_cli
    - aws configure set region $REGION
    - aws configure set aws_access_key_id $AWS_FULL_STS_ACCESS_KEY_ID
    - aws configure set aws_secret_access_key $AWS_FULL_STS_ACCESS_KEY_SECRET
    - aws sts get-caller-identity
    - aws configure set source_profile default --profile $ACCOUNT_NAME
    - aws configure set role_arn $ROLE_ARN --profile $ACCOUNT_NAME
    - aws sts get-caller-identity --profile $ACCOUNT_NAME
    - aws configure set region $REGION --profile $ACCOUNT_NAME

.copy_remote_log: &copy_remote_log
- if [ -e outfile ]; then rm outfile; fi
- copy_command="$(cat $CI_PROJECT_DIR/scp_command.txt)"
- new_copy_command=${copy_command/"%s"/"outfile"}
- new_copy_command=${new_copy_command/"~"/"/home/ec2-user/outfile"}
- echo $new_copy_command
- new_copy_command=$(echo "$new_copy_command" | sed s'/\([^.]*\.[^ ]*\) \([^ ]*\) \(.*\)/\1 \3 \2/')
- echo $new_copy_command
- sleep 10
- eval $new_copy_command

.check_remote_log: &check_remote_log
- sleep 10
- grep Error outfile || true
- sleep 10
- returnCode=$(grep -c Error outfile) || true
- echo "Return code received $returnCode"
- if [ $returnCode -ge 1 ]; then exit 1; fi
- echo "No errors"

.prepare_ssh_key: &prepare_ssh_key
- echo $ADMIN_SERVER_SSH_KEY > $CI_PROJECT_DIR/ssh_key.pem
- cat ssh_key.pem
- sed -i -e 's/-----BEGIN RSA PRIVATE KEY-----/-bk-/g' ssh_key.pem
- sed -i -e 's/-----END RSA PRIVATE KEY-----/-ek-/g' ssh_key.pem
- perl -p -i -e 's/\s/\n/g' ssh_key.pem
- sed -i -e 's/-bk-/-----BEGIN RSA PRIVATE KEY-----/g' ssh_key.pem
- sed -i -e 's/-ek-/-----END RSA PRIVATE KEY-----/g' ssh_key.pem
- cat ssh_key.pem
- chmod 400 ssh_key.pem

connect-admin-server:
  stage: build
  allow_failure: true
  image:
    name: amazon/aws-cli:latest
    entrypoint: [ "" ]
  rules:
    - if: '$ADMIN_SERVER_IP && $ADMIN_SERVER_IP != "" && $ADMIN_SERVER_SSH_KEY && $ADMIN_SERVER_SSH_KEY != ""'
  extends:
    - .ifawsdeploy
  script:
    - TF_IN_AUTOMATION=true
    - yum update -y
    - yum install git unzip gettext jq -y
    - echo "Your admin server key and info are added as artifacts"
    # Copy the important terraform outputs to files for artifacts to pass into other jobs
    - *prepare_ssh_key
    - echo "ssh -i ssh_key.pem ec2-user@${ADMIN_SERVER_IP}" > $CI_PROJECT_DIR/ssh_command.txt
    - echo "scp -q -i ssh_key.pem %s ec2-user@${ADMIN_SERVER_IP}:~" > $CI_PROJECT_DIR/scp_command.txt
    - test_pre_command="$(cat "$CI_PROJECT_DIR/ssh_command.txt") -o StrictHostKeyChecking=no"
    - echo $test_pre_command
    - test_command="$(echo $test_pre_command | sed -r 's/(ssh )(.*)/\1-tt \2/')"
    - echo $test_command
    - echo "sudo yum install -yq $PACKAGEURL 2>&1 | tee outfile ; exit 0" | $test_command
    - *copy_remote_log
    - echo "Now checking log file for returnCode"
    - *check_remote_log
  artifacts:
    untracked: true
    when: always
    paths:
      - "$CI_PROJECT_DIR/ssh_key.pem"
      - "$CI_PROJECT_DIR/ssh_command.txt"
      - "$CI_PROJECT_DIR/scp_command.txt"
  after_script:
    - cat $CI_PROJECT_DIR/ssh_key.pem
    - cat $CI_PROJECT_DIR/ssh_command.txt
    - cat $CI_PROJECT_DIR/scp_command.txt

create-admin-server:
  stage: build
  allow_failure: false
  image:
    name: amazon/aws-cli:latest
    entrypoint: [ "" ]
  rules:
    - if: '$ADMIN_SERVER_IP != ""'
      when: never
  extends:
    - .ifawsdeploy
  script:
    - echo "admin server $ADMIN_SERVER_IP"
    - TF_IN_AUTOMATION=true
    - yum update -y
    - yum install git unzip gettext jq -y
    - *configure_aws_cli
    - aws sts get-caller-identity --profile $ACCOUNT_NAME #to check whether updated correctly or not
    - git clone "https://project-n-setup:$(echo $PERSONAL_GITLAB_TOKEN)@gitlab.com/company-oss/project-n-setup.git"
    # Install tfenv
    - git clone https://github.com/tfutils/tfenv.git ~/.tfenv
    - ln -s ~/.tfenv /root/.tfenv
    - ln -s ~/.tfenv/bin/* /usr/local/bin
    # Install terraform 1.1.9 through tfenv
    - tfenv install $tfenv_version
    - tfenv use $tfenv_version
    # Copy the tfvars temp file to the terraform setup directory
    - cp .gitlab/admin_server.temp_tfvars project-n-setup/$SUB_PLATFORM/
    - cd project-n-setup/$SUB_PLATFORM/
    - envsubst < admin_server.temp_tfvars > admin_server.tfvars
    - rm -rf .terraform || exit 0
    - cat ~/.aws/config
    - terraform init -input=false
    - terraform apply -var-file=admin_server.tfvars -input=false -auto-approve
    - echo "Your admin server key and info are added as artifacts"
    # Copy the important terraform outputs to files for artifacts to pass into other jobs
    - terraform output -raw ssh_key > $CI_PROJECT_DIR/ssh_key.pem
    - terraform output -raw ssh_command > $CI_PROJECT_DIR/ssh_command.txt
    - terraform output -raw scp_command > $CI_PROJECT_DIR/scp_command.txt
    - cp $CI_PROJECT_DIR/project-n-setup/$SUB_PLATFORM/terraform.tfstate $CI_PROJECT_DIR
    - cp $CI_PROJECT_DIR/project-n-setup/$SUB_PLATFORM/admin_server.tfvars $CI_PROJECT_DIR
  artifacts:
    untracked: true
    paths:
      - "$CI_PROJECT_DIR/ssh_key.pem"
      - "$CI_PROJECT_DIR/ssh_command.txt"
      - "$CI_PROJECT_DIR/scp_command.txt"
      - "$CI_PROJECT_DIR/terraform.tfstate"
      - "$CI_PROJECT_DIR/admin_server.tfvars"

如何解决?

我尝试了以下评论部分的建议。

.generalgrabclustertrigger:
  rules:
    - if: '$TEST_CREATE_ADMIN && $REGION && $ROLE_ARN && $PACKAGEURL && $TEST_CREATE_ADMIN == "aws" && $SUB_PLATFORM == "aws" && $ROLE_ARN != "" && $PACKAGEURL != "" && $REGION != ""'

.ifteardownordestroy:  # Automatic if triggered from gitlab api AND destroy variable is set
  rules:
    - !reference [.generalgrabclustertrigger, rules]
    - if: 'CI_PIPELINE_SOURCE == "triggered"'
      when: never

并包括上述工作。

destroy-admin-server:
  stage: cleanup
  extends:
    - .ifteardownordestroy
  allow_failure: true
  interruptible: false

但是我在.fifeardownOddestroy部分中遇到语法错误。

作业:Destroy-Admin-Server:规则:规则如果无效的表达语法

Below are two jobs in the build stage.
Default, there is set some common condition, and using extends keyword for that, ifawsdeploy.
As only one of them should run, if variable $ADMIN_SERVER_IP provided then connect_admin_server should run, working that way.
If no value provided to $ADMIN_SERVER_IP then create_admin_server should run, but it is not running.

.ifawsdeploy:
  rules:
    - if: '$TEST_CREATE_ADMIN && $REGION && $ROLE_ARN && $PACKAGEURL && $TEST_CREATE_ADMIN == "aws" && $SUB_PLATFORM == "aws" && $ROLE_ARN != "" && $PACKAGEURL != "" && $REGION != ""'

variables:
 TEST_CREATE_ADMIN:
   #value: aws
   description: "Platform, currently aws only"
 SUB_PLATFORM:
   value: aws
   description: "Platform, currently aws only"
 REGION:
   value: "us-west-2"
   description: "region where to deploy company"
 PACKAGEURL:
   value: "http://somerpmurl.x86_64.rpm"
   description: "company rpm file url"
 ACCOUNT_NAME:
   value: "testsubaccount"
   description: "Account name of sub account to refer in the deployment, no need to match in AWS"
 ROLE_ARN:
   value: "arn:aws:iam::491483064167:role/uat"
   description: "ROLE ARN of the user account assuming: aws sts get-caller-identity"
 tfenv_version: "1.1.9"
 DEV_PUB_KEY:
   description: "Optional public key file to add access to admin server" 
 ADMIN_SERVER_IP:
   description: "Existing Admin Server IP Address"
 ADMIN_SERVER_SSH_KEY:
   description: "Existing Admin Server SSH_KEY PEM content"
  
#export variables below will cause the terraform to use the root account instead of the one specified in tfvars file
.configure_aws_cli: &configure_aws_cli
    - aws configure set region $REGION
    - aws configure set aws_access_key_id $AWS_FULL_STS_ACCESS_KEY_ID
    - aws configure set aws_secret_access_key $AWS_FULL_STS_ACCESS_KEY_SECRET
    - aws sts get-caller-identity
    - aws configure set source_profile default --profile $ACCOUNT_NAME
    - aws configure set role_arn $ROLE_ARN --profile $ACCOUNT_NAME
    - aws sts get-caller-identity --profile $ACCOUNT_NAME
    - aws configure set region $REGION --profile $ACCOUNT_NAME

.copy_remote_log: ©_remote_log
- if [ -e outfile ]; then rm outfile; fi
- copy_command="$(cat $CI_PROJECT_DIR/scp_command.txt)"
- new_copy_command=${copy_command/"%s"/"outfile"}
- new_copy_command=${new_copy_command/"~"/"/home/ec2-user/outfile"}
- echo $new_copy_command
- new_copy_command=$(echo "$new_copy_command" | sed s'/\([^.]*\.[^ ]*\) \([^ ]*\) \(.*\)/\1 \3 \2/')
- echo $new_copy_command
- sleep 10
- eval $new_copy_command

.check_remote_log: &check_remote_log
- sleep 10
- grep Error outfile || true
- sleep 10
- returnCode=$(grep -c Error outfile) || true
- echo "Return code received $returnCode"
- if [ $returnCode -ge 1 ]; then exit 1; fi
- echo "No errors"

.prepare_ssh_key: &prepare_ssh_key
- echo $ADMIN_SERVER_SSH_KEY > $CI_PROJECT_DIR/ssh_key.pem
- cat ssh_key.pem
- sed -i -e 's/-----BEGIN RSA PRIVATE KEY-----/-bk-/g' ssh_key.pem
- sed -i -e 's/-----END RSA PRIVATE KEY-----/-ek-/g' ssh_key.pem
- perl -p -i -e 's/\s/\n/g' ssh_key.pem
- sed -i -e 's/-bk-/-----BEGIN RSA PRIVATE KEY-----/g' ssh_key.pem
- sed -i -e 's/-ek-/-----END RSA PRIVATE KEY-----/g' ssh_key.pem
- cat ssh_key.pem
- chmod 400 ssh_key.pem

connect-admin-server:
  stage: build
  allow_failure: true
  image:
    name: amazon/aws-cli:latest
    entrypoint: [ "" ]
  rules:
    - if: '$ADMIN_SERVER_IP && $ADMIN_SERVER_IP != "" && $ADMIN_SERVER_SSH_KEY && $ADMIN_SERVER_SSH_KEY != ""'
  extends:
    - .ifawsdeploy
  script:
    - TF_IN_AUTOMATION=true
    - yum update -y
    - yum install git unzip gettext jq -y
    - echo "Your admin server key and info are added as artifacts"
    # Copy the important terraform outputs to files for artifacts to pass into other jobs
    - *prepare_ssh_key
    - echo "ssh -i ssh_key.pem ec2-user@${ADMIN_SERVER_IP}" > $CI_PROJECT_DIR/ssh_command.txt
    - echo "scp -q -i ssh_key.pem %s ec2-user@${ADMIN_SERVER_IP}:~" > $CI_PROJECT_DIR/scp_command.txt
    - test_pre_command="$(cat "$CI_PROJECT_DIR/ssh_command.txt") -o StrictHostKeyChecking=no"
    - echo $test_pre_command
    - test_command="$(echo $test_pre_command | sed -r 's/(ssh )(.*)/\1-tt \2/')"
    - echo $test_command
    - echo "sudo yum install -yq $PACKAGEURL 2>&1 | tee outfile ; exit 0" | $test_command
    - *copy_remote_log
    - echo "Now checking log file for returnCode"
    - *check_remote_log
  artifacts:
    untracked: true
    when: always
    paths:
      - "$CI_PROJECT_DIR/ssh_key.pem"
      - "$CI_PROJECT_DIR/ssh_command.txt"
      - "$CI_PROJECT_DIR/scp_command.txt"
  after_script:
    - cat $CI_PROJECT_DIR/ssh_key.pem
    - cat $CI_PROJECT_DIR/ssh_command.txt
    - cat $CI_PROJECT_DIR/scp_command.txt

create-admin-server:
  stage: build
  allow_failure: false
  image:
    name: amazon/aws-cli:latest
    entrypoint: [ "" ]
  rules:
    - if: '$ADMIN_SERVER_IP != ""'
      when: never
  extends:
    - .ifawsdeploy
  script:
    - echo "admin server $ADMIN_SERVER_IP"
    - TF_IN_AUTOMATION=true
    - yum update -y
    - yum install git unzip gettext jq -y
    - *configure_aws_cli
    - aws sts get-caller-identity --profile $ACCOUNT_NAME #to check whether updated correctly or not
    - git clone "https://project-n-setup:$(echo $PERSONAL_GITLAB_TOKEN)@gitlab.com/company-oss/project-n-setup.git"
    # Install tfenv
    - git clone https://github.com/tfutils/tfenv.git ~/.tfenv
    - ln -s ~/.tfenv /root/.tfenv
    - ln -s ~/.tfenv/bin/* /usr/local/bin
    # Install terraform 1.1.9 through tfenv
    - tfenv install $tfenv_version
    - tfenv use $tfenv_version
    # Copy the tfvars temp file to the terraform setup directory
    - cp .gitlab/admin_server.temp_tfvars project-n-setup/$SUB_PLATFORM/
    - cd project-n-setup/$SUB_PLATFORM/
    - envsubst < admin_server.temp_tfvars > admin_server.tfvars
    - rm -rf .terraform || exit 0
    - cat ~/.aws/config
    - terraform init -input=false
    - terraform apply -var-file=admin_server.tfvars -input=false -auto-approve
    - echo "Your admin server key and info are added as artifacts"
    # Copy the important terraform outputs to files for artifacts to pass into other jobs
    - terraform output -raw ssh_key > $CI_PROJECT_DIR/ssh_key.pem
    - terraform output -raw ssh_command > $CI_PROJECT_DIR/ssh_command.txt
    - terraform output -raw scp_command > $CI_PROJECT_DIR/scp_command.txt
    - cp $CI_PROJECT_DIR/project-n-setup/$SUB_PLATFORM/terraform.tfstate $CI_PROJECT_DIR
    - cp $CI_PROJECT_DIR/project-n-setup/$SUB_PLATFORM/admin_server.tfvars $CI_PROJECT_DIR
  artifacts:
    untracked: true
    paths:
      - "$CI_PROJECT_DIR/ssh_key.pem"
      - "$CI_PROJECT_DIR/ssh_command.txt"
      - "$CI_PROJECT_DIR/scp_command.txt"
      - "$CI_PROJECT_DIR/terraform.tfstate"
      - "$CI_PROJECT_DIR/admin_server.tfvars"

How to fix that?

I tried the below step from suggestions on comments section.

.generalgrabclustertrigger:
  rules:
    - if: '$TEST_CREATE_ADMIN && $REGION && $ROLE_ARN && $PACKAGEURL && $TEST_CREATE_ADMIN == "aws" && $SUB_PLATFORM == "aws" && $ROLE_ARN != "" && $PACKAGEURL != "" && $REGION != ""'

.ifteardownordestroy:  # Automatic if triggered from gitlab api AND destroy variable is set
  rules:
    - !reference [.generalgrabclustertrigger, rules]
    - if: 'CI_PIPELINE_SOURCE == "triggered"'
      when: never

And included the above in extends of a job.

destroy-admin-server:
  stage: cleanup
  extends:
    - .ifteardownordestroy
  allow_failure: true
  interruptible: false

But I am getting syntax error in the .ifteardownordestroy part.

jobs:destroy-admin-server:rules:rule if invalid expression syntax

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

等数载,海棠开 2025-02-13 05:11:53

您正在覆盖规则:在工作中扩展了.ifawsdeploy规则:在这种情况下未合并 - 规则的定义:在作业中完全优先。

以以下配置为例:

.template:
  rules:
    - one
    - two

myjob:
  extends: .template
  rules:
    - a
    - b

在上面的示例中,myjob作业只有规则ab生效。规则<代码>一个和两个被完全忽略,因为它们在作业配置中被覆盖。

您可以使用而不是uinsg 扩展:可以使用!参考来保存和结合规则。如果需要,也可以使用YAML锚。

create-admin-server:
  rules:
    - !reference [.ifawsdeploy, rules]
    - ... # your additional rules

如果没有向$ admin_server_ip提供的值,则create_admin_server应该运行

最后运行,请特别注意您的规则:

  rules:
    - if: '$ADMIN_SERVER_IP != ""'
      when: never

在这种情况下,没有规则可以使作业运行。您要么需要一个可以评估作业运行的案例,要么具有默认情况(如果没有:条件),以便运行该作业。

为了获得您期望的行为,您可能希望您的默认情况为on_success

  rules:
    - if: '$ADMIN_SERVER_IP != ""'
      when: never
    - when: on_success

You are overriding rules: in your job that extends .ifawsdeploy. rules: are not combined in this case -- the definition of rules: in the job takes complete precedence.

Take for example the following configuration:

.template:
  rules:
    - one
    - two

myjob:
  extends: .template
  rules:
    - a
    - b

In the above example, the myjob job only has rules a and b in effect. Rules one and two are completely ignored because they are overridden in the job configuration.

Instead of uinsg extends:, you can use !reference to preserve and combine rules. You can also use YAML anchors if you want.

create-admin-server:
  rules:
    - !reference [.ifawsdeploy, rules]
    - ... # your additional rules

If no value provided to $ADMIN_SERVER_IP then create_admin_server should run

Lastly, pay special attention to your rules:

  rules:
    - if: '$ADMIN_SERVER_IP != ""'
      when: never

In this case, there are no rules that allow the job to run ever. You either need a case that will evaluate true for the job to run, or to have a default case (an item with no if: condition) in order for the job to run.

To get the behavior you expect, you probably want your default case to be on_success:

  rules:
    - if: '$ADMIN_SERVER_IP != ""'
      when: never
    - when: on_success
痴情换悲伤 2025-02-13 05:11:53

您可以将规则更改为:

  rules:
    - if: '$ADMIN_SERVER_IP != ""'
      when: never
    - when: always

或者

  rules:
    - if: '$ADMIN_SERVER_IP == ""'
      when: always

我在这里有一个示例: try-rules-stackoverflow-72545625-Gitlab-gitlab 和管道记录 pipeline no value-gitlab 管道具有值 - gitlab

you can change your rules to :

  rules:
    - if: '$ADMIN_SERVER_IP != ""'
      when: never
    - when: always

or

  rules:
    - if: '$ADMIN_SERVER_IP == ""'
      when: always

I have a sample in here: try-rules-stackoverflow-72545625 - GitLab and the pipeline record Pipeline no value - GitLab, Pipeline has value - GitLab

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文