使用JQ动态构建JSON,以与GitHub-Actions策略矩阵一起使用
我有许多与AWS和GitHub动作一起使用的变量。 源不是JSON,而是TerraForm.TFVARS文件存储为秘密,但需要在运行时动态转换为AWS CLI V2-Parameters和-parameter-overrides JSON,以用于云形式。
秘密值参数中的变量的示例(存储为base64来保留LF):
unique_prefix = "a-unique-value"
base_region = "ap-southeast-2"
变量也用于外部云形式调用(例如,在假定角色中使用base_region),因此github操作需要构建其JSON格式的矩阵,
我也希望这些变量用于这些变量使用纯JQ的GitHub动作,将动态生成各种JSON格式,并使用纯JQ进行操作,并将最小的bash字符串操作成AWS Cli Cliformation命令所需的不一致格式和GitHub Action Matrix。
因此,带有以下JSON输入/输出的GitHub矩阵,
{
"include": [
{
"unique_prefix": "a-unique-value",
"base_region": "ap-southeast-2",
}
]
}
带有GitHub作业,引用此构建Matrix作业,可以产生两个可以
echo ${{ matrix.unique_prefix }}
echo ${{ matrix.base_region }}
使用此策略矩阵进行引用的变量,应该可以构建云形式形式:
用于输入。对于CloudFormation stack(-parameter-overrides file://param.json),结果是:
[
{
"ParameterKey": "unique_prefix",
"ParameterValue": "a-unique-value"
},
{
"ParameterKey": "base_region",
"ParameterValue": "ap-southeast-2"
}
]
对于CloudFormation stacksets( - Parameters),结果是:
[
{
"Key": "unique_prefix",
"Value": "a-unique-value"
},
{
"Key": "base_region",
"Value": "ap-southeast-2"
}
]
因此,理想情况下,我需要一个GitHub Action作业,只是 动态创建这两种或三种JSON格式,以进一步在其他作业中使用,或者只是在策略矩阵中,然后将其转换为进一步的作业。
通过 策略矩阵(并通过调试检查)
name: Test
on:
workflow_dispatch:
jobs:
build-matrix:
name: Build JSON from PARAMS
runs-on: ubuntu-latest
outputs:
matrix: ${{ steps.set-matrix.outputs.matrix }}
steps:
- name: Create Mapping JSON
id: set-matrix
run: |
echo '${{ secrets.PARAMS }}' | base64 --decode > "${{ github.workspace }}/terraform.fvars"
# I dont know how to pull in the file here for jq
JSON=$(jq -c -n --argjson v '{"unique_prefix":"a-unique-value","base_region":"ap-southeast-2"}' '{"include": [$v]}')
echo "::set-output name=matrix::${JSON}"
和我用来显示矩阵的YAML输出的调试工作:
check-matrix:
name: Check Parameters
runs-on: ubuntu-latest
needs: build-matrix
strategy:
matrix: ${{fromJson(needs.build-matrix.outputs.matrix)}}
steps:
- name: Install json2yaml
run: |
sudo npm install -g json2yaml
- name: Check matrix definition
run: |
matrix='${{ needs.build-matrix.outputs.matrix }}'
echo $matrix
echo $matrix | jq .
echo $matrix | json2yaml
echo ${{ matrix.unique_prefix }} # Will show a-unique-value
echo ${{ matrix.base_region }} # Will show ap-southeast-2
# jq to create CFN stacksets json params
# jq to create CFN stack json params
因此,从Terraform.TFVARS文件中动态构建矩阵 并通过矩阵或类似方法动态构建CFN JSON格式。
I have a number of variables required for use with AWS and GitHub Actions.
The source is not JSON, but a terraform.tfvars file stored as a secret, but it needs to be dynamically converted at runtime into AWS CLI v2 --parameters and --parameter-overrides JSON for use with cloudformation.
example of variables in secret value PARAMS (stored as base64 to preserve LF):
unique_prefix = "a-unique-value"
base_region = "ap-southeast-2"
The variables are also used outside CloudFormation calls (eg base_region is used in assume roles) so GitHub Actions needs a Matrix with its JSON format built also
I want these variables to be dynamically generated into various JSON formats with GitHub actions ideally with pure jq, and with minimal bash string manipulations into the inconsistent formats needed for AWS CLI CloudFormation commands, and GitHub action Matrix.
So a GitHub matrix with the following JSON input/output
{
"include": [
{
"unique_prefix": "a-unique-value",
"base_region": "ap-southeast-2",
}
]
}
with a github job that reference this build-matrix job, can produce two variables that can be referenced with
echo ${{ matrix.unique_prefix }}
echo ${{ matrix.base_region }}
so using this strategy matrix, it should be possible to build the CloudFormation formats:
for input for a cloudformation Stack (--parameter-overrides file://param.json), the result would be:
[
{
"ParameterKey": "unique_prefix",
"ParameterValue": "a-unique-value"
},
{
"ParameterKey": "base_region",
"ParameterValue": "ap-southeast-2"
}
]
for CloudFormation StackSets (--parameters), the result would be:
[
{
"Key": "unique_prefix",
"Value": "a-unique-value"
},
{
"Key": "base_region",
"Value": "ap-southeast-2"
}
]
So ideally, I would need a github action job that just creates these two or three JSON formats dynamically for further use in other jobs, or just the strategy matrix, and just convert that in further jobs., but i cant work out how to dynamically grab the input
currently im at this point of building a GitHub strategy matrix (and checked with debug)
name: Test
on:
workflow_dispatch:
jobs:
build-matrix:
name: Build JSON from PARAMS
runs-on: ubuntu-latest
outputs:
matrix: ${{ steps.set-matrix.outputs.matrix }}
steps:
- name: Create Mapping JSON
id: set-matrix
run: |
echo '${{ secrets.PARAMS }}' | base64 --decode > "${{ github.workspace }}/terraform.fvars"
# I dont know how to pull in the file here for jq
JSON=$(jq -c -n --argjson v '{"unique_prefix":"a-unique-value","base_region":"ap-southeast-2"}' '{"include": [$v]}')
echo "::set-output name=matrix::${JSON}"
and a debug job i use to show the yaml output of the matrix:
check-matrix:
name: Check Parameters
runs-on: ubuntu-latest
needs: build-matrix
strategy:
matrix: ${{fromJson(needs.build-matrix.outputs.matrix)}}
steps:
- name: Install json2yaml
run: |
sudo npm install -g json2yaml
- name: Check matrix definition
run: |
matrix='${{ needs.build-matrix.outputs.matrix }}'
echo $matrix
echo $matrix | jq .
echo $matrix | json2yaml
echo ${{ matrix.unique_prefix }} # Will show a-unique-value
echo ${{ matrix.base_region }} # Will show ap-southeast-2
# jq to create CFN stacksets json params
# jq to create CFN stack json params
so dynamically build the matrix from the terraform.tfvars file
and dynamically build the CFN JSON formats from either the matrix or similar method.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论