使用JQ动态构建JSON,以与GitHub-Actions策略矩阵一起使用

发布于 2025-02-10 23:16:09 字数 2893 浏览 0 评论 0原文

我有许多与AWS和GitHub动作一起使用的变量。 源不是JSON,而是TerraForm.TFVARS文件存储为秘密,但需要在运行时动态转换为AWS CLI V2-Parameters和-parameter-overrides JSON,以用于云形式。

秘密值参数中的变量的示例(存储为base64来保留LF):

unique_prefix = "a-unique-value"
base_region   = "ap-southeast-2"

变量也用于外部云形式调用(例如,在假定角色中使用base_region),因此github操作需要构建其JSON格式的矩阵,

我也希望这些变量用于这些变量使用纯JQ的GitHub动作,将动态生成各种JSON格式,并使用纯JQ进行操作,并将最小的bash字符串操作成AWS Cli Cliformation命令所需的不一致格式和GitHub Action Matrix。

因此,带有以下JSON输入/输出的GitHub矩阵,

{
  "include": [
    {
      "unique_prefix": "a-unique-value",
      "base_region": "ap-southeast-2",
    }
  ]
}

带有GitHub作业,引用此构建Matrix作业,可以产生两个可以

          echo ${{ matrix.unique_prefix }}
          echo ${{ matrix.base_region }}

使用此策略矩阵进行引用的变量,应该可以构建云形式形式:

用于输入。对于CloudFormation stack(-parameter-overrides file://param.json),结果是:

[
  {
    "ParameterKey": "unique_prefix",
    "ParameterValue": "a-unique-value"
  },
  {
    "ParameterKey": "base_region",
    "ParameterValue": "ap-southeast-2"
  }
]

对于CloudFormation stacksets( - Parameters),结果是:

[
  {
    "Key": "unique_prefix",
    "Value": "a-unique-value"
  },
  {
    "Key": "base_region",
    "Value": "ap-southeast-2"
  }
]

因此,理想情况下,我需要一个GitHub Action作业,只是 动态创建这两种或三种JSON格式,以进一步在其他作业中使用,或者只是在策略矩阵中,然后将其转换为进一步的作业。

通过 策略矩阵(并通过调试检查)

name: Test
on:
  workflow_dispatch:

jobs:
  build-matrix:
    name: Build JSON from PARAMS
    runs-on: ubuntu-latest
    outputs:
      matrix: ${{ steps.set-matrix.outputs.matrix }}
    steps:
      - name: Create Mapping JSON
        id: set-matrix
        run: |
          echo '${{ secrets.PARAMS }}' | base64 --decode > "${{ github.workspace }}/terraform.fvars"
          # I dont know how to pull in the file here for jq
          JSON=$(jq -c -n --argjson v '{"unique_prefix":"a-unique-value","base_region":"ap-southeast-2"}' '{"include": [$v]}')
          echo "::set-output name=matrix::${JSON}"

和我用来显示矩阵的YAML输出的调试工作:

  check-matrix:
    name: Check Parameters
    runs-on: ubuntu-latest
    needs: build-matrix
    strategy:
      matrix: ${{fromJson(needs.build-matrix.outputs.matrix)}}
    steps:
      - name: Install json2yaml
        run: |
          sudo npm install -g json2yaml

      - name: Check matrix definition
        run: |
          matrix='${{ needs.build-matrix.outputs.matrix }}'
          echo $matrix
          echo $matrix | jq .
          echo $matrix | json2yaml
          echo ${{ matrix.unique_prefix }} # Will show a-unique-value
          echo ${{ matrix.base_region }}   # Will show ap-southeast-2
          # jq to create CFN stacksets json params
          # jq to create CFN stack json params

因此,从Terraform.TFVARS文件中动态构建矩阵 并通过矩阵或类似方法动态构建CFN JSON格式。

I have a number of variables required for use with AWS and GitHub Actions.
The source is not JSON, but a terraform.tfvars file stored as a secret, but it needs to be dynamically converted at runtime into AWS CLI v2 --parameters and --parameter-overrides JSON for use with cloudformation.

example of variables in secret value PARAMS (stored as base64 to preserve LF):

unique_prefix = "a-unique-value"
base_region   = "ap-southeast-2"

The variables are also used outside CloudFormation calls (eg base_region is used in assume roles) so GitHub Actions needs a Matrix with its JSON format built also

I want these variables to be dynamically generated into various JSON formats with GitHub actions ideally with pure jq, and with minimal bash string manipulations into the inconsistent formats needed for AWS CLI CloudFormation commands, and GitHub action Matrix.

So a GitHub matrix with the following JSON input/output

{
  "include": [
    {
      "unique_prefix": "a-unique-value",
      "base_region": "ap-southeast-2",
    }
  ]
}

with a github job that reference this build-matrix job, can produce two variables that can be referenced with

          echo ${{ matrix.unique_prefix }}
          echo ${{ matrix.base_region }}

so using this strategy matrix, it should be possible to build the CloudFormation formats:

for input for a cloudformation Stack (--parameter-overrides file://param.json), the result would be:

[
  {
    "ParameterKey": "unique_prefix",
    "ParameterValue": "a-unique-value"
  },
  {
    "ParameterKey": "base_region",
    "ParameterValue": "ap-southeast-2"
  }
]

for CloudFormation StackSets (--parameters), the result would be:

[
  {
    "Key": "unique_prefix",
    "Value": "a-unique-value"
  },
  {
    "Key": "base_region",
    "Value": "ap-southeast-2"
  }
]

So ideally, I would need a github action job that just creates these two or three JSON formats dynamically for further use in other jobs, or just the strategy matrix, and just convert that in further jobs., but i cant work out how to dynamically grab the input

currently im at this point of building a GitHub strategy matrix (and checked with debug)

name: Test
on:
  workflow_dispatch:

jobs:
  build-matrix:
    name: Build JSON from PARAMS
    runs-on: ubuntu-latest
    outputs:
      matrix: ${{ steps.set-matrix.outputs.matrix }}
    steps:
      - name: Create Mapping JSON
        id: set-matrix
        run: |
          echo '${{ secrets.PARAMS }}' | base64 --decode > "${{ github.workspace }}/terraform.fvars"
          # I dont know how to pull in the file here for jq
          JSON=$(jq -c -n --argjson v '{"unique_prefix":"a-unique-value","base_region":"ap-southeast-2"}' '{"include": [$v]}')
          echo "::set-output name=matrix::${JSON}"

and a debug job i use to show the yaml output of the matrix:

  check-matrix:
    name: Check Parameters
    runs-on: ubuntu-latest
    needs: build-matrix
    strategy:
      matrix: ${{fromJson(needs.build-matrix.outputs.matrix)}}
    steps:
      - name: Install json2yaml
        run: |
          sudo npm install -g json2yaml

      - name: Check matrix definition
        run: |
          matrix='${{ needs.build-matrix.outputs.matrix }}'
          echo $matrix
          echo $matrix | jq .
          echo $matrix | json2yaml
          echo ${{ matrix.unique_prefix }} # Will show a-unique-value
          echo ${{ matrix.base_region }}   # Will show ap-southeast-2
          # jq to create CFN stacksets json params
          # jq to create CFN stack json params

so dynamically build the matrix from the terraform.tfvars file
and dynamically build the CFN JSON formats from either the matrix or similar method.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。
列表为空,暂无数据
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文