仅当先前任务的输出更改时,才运行大厅构建步骤

发布于 2025-01-25 07:28:14 字数 734 浏览 2 评论 0原文

鉴于输入,我具有便宜的功能和昂贵的功能;这些中的每一个都是建模为一项大厅任务。

如果廉价功能的两个调用具有相同的输出,我知道昂贵功能的两个调用同样将具有相同的输出。

如何设置只有在廉价功能的结果变化时才能运行昂贵功能的管道?


为了示例,假设廉价功能将代码库的评论和空格剥离,然后计算出校验和校验和。而昂贵的功能实际运行了包含的代码。在这种情况下,我的目标是不必费心建立任何仅在评论或空格中与前面区别的修订。

我已经考虑过使用git资源,并且(在我们的示例中)将每个汇编目标存储在另一个文件中的预处理器输出哈希,因此执行实际编译(和适用的单位测试)的任务可以使用哈希触发文件的更改。构建该文件的输入。但是,拥有一个单独的GIT资源,可以无限期地维持历史哈希,这似乎是过分的。有更好的方法吗?


这类似于仅在文件diff上构建新的docker容器 ,但我正在尝试测试是否针对文件运行函数的结果 更改,以仅在可以修改的更改上触发建立结果,而不是所有可能的更改。 (上述提案,创建了一个带有廉价功能输出的中介存储库,实际上将使用该问题的答案作为其组件之一;但是我希望有一个可以使用更少的运动部件的选项)。

Given an input, I have a cheap function and an expensive function; each of these is modeled as a Concourse task.

If two invocations of the cheap function have the same output, I know that two invocations of the expensive function will likewise have the same output.

How can I set up a pipeline that only runs the expensive function when the result of the cheap function changes?


For the sake of an example, let's say that the cheap function strips comments and whitespace from a codebase and then calculates a checksum; whereas the expensive function actually runs the code contained. My goal, in this scenario, is to not bother building any revision that differs from the prior one only in comments or whitespace.

I've considered using a git resource and (in our example) storing a hash of preprocessor output for each compilation target in a different file, so the task doing actual compilation (and applicable unit tests) can trigger on changes to the file with hash of the inputs that went into building that file. Having a separate git resource that maintains historical hashes indefinitely seems like overkill, though. Is there a better approach?


This is similar to Have Concourse only build new docker containers on file diff not on commit, but I'm trying to test whether the result of running a function against a file changes, to trigger only on changes that could modify build results rather than all possible changes. (The proposal described above, creating an intermediary repo with outputs from the cheap function, would effectively be using the answers to that question as one of its components; but I'm hoping there's an option with fewer moving parts).

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

兔小萌 2025-02-01 07:28:14

考虑使用put嵌套在中尝试:修改器:

“有条件触发器”

便宜 job abor

  • cop tup tust code:git recoto with code assh
  • hash 最后一个廉价计算中

在每项提交code-repo的每项提交的 并将其与计算结果进行比较(在下面的愚蠢示例中,hash.txt的内容> code> code> code-repo)。

如果它确定来自传入提交的哈希值与先前记录的哈希值有所不同,则将填充put param hash/hash.txt带有新的哈希值,结果结果在新的资源中,这又将触发昂贵的作业。

如果未检测到更改,则PUT尝试将失败,因为PUT参数将不存在,但是总体便宜作业将成功。

resources:
  - name: code-repo
    type: git
    source:
      branch: master
      private_key: ((key))
      uri: [email protected]:myorg/code-repo.git

  - name: hash
    type: s3
    source:
      access_key_id: ((aws_access))
      secret_access_key: ((aws_secret))
      region_name: ((aws_region))
      bucket: my-versioned-aws-bucket
      versioned_file: hash/hash.txt

jobs:
  - name: cheap
    plan:
    - get: code-repo
      trigger: true
    - get: hash
    - task: check
      input_mapping:
        last-hash: hash
      config:
        platform: linux
        image_resource:
          type: docker-image
          source: { repository: alpine }
        inputs:
          - name: code-repo
          - name: last-hash
        outputs:
          - name: hash
        run:
          path: /bin/sh
          args:
          - -c
          - |
            LAST="$(cat last-hash/hash.txt)"
            NEW=$(cat code-repo/hash.txt)
            if [ "$LAST" != "$NEW" ]; then
              cp code-repo/hash.txt hash/hash.txt
            fi
      on_success:
        try:
          put: hash
          params:
            file: hash/hash.txt

  - name: expensive
    plan:
    - get: hash
      trigger: true
      passed: [ cheap ]

注意:您必须在s3中填充初始状态文件,或者便宜作业不会起飞。

Consider using put nested in the try: modifier:

conditional trigger

The cheap job takes two inputs:

  • git repo with the code
  • hash of the last cheap computation

On every commit to code-repo, the cheap job reads the last-hash input, mapped from hash and compares it to the computation result (in the silly example below, the contents of hash.txt checked into the root of code-repo).

If it determines that the hash value from incoming commit differs from the previously recorded hash value, it populates the put param hash/hash.txt with the new hash value, which results in a new put to the resource which in turn will trigger the expensive job.

If no change is detected, the put attempt will fail because the put param will not exist, but the overall cheap job will succeed.

resources:
  - name: code-repo
    type: git
    source:
      branch: master
      private_key: ((key))
      uri: [email protected]:myorg/code-repo.git

  - name: hash
    type: s3
    source:
      access_key_id: ((aws_access))
      secret_access_key: ((aws_secret))
      region_name: ((aws_region))
      bucket: my-versioned-aws-bucket
      versioned_file: hash/hash.txt

jobs:
  - name: cheap
    plan:
    - get: code-repo
      trigger: true
    - get: hash
    - task: check
      input_mapping:
        last-hash: hash
      config:
        platform: linux
        image_resource:
          type: docker-image
          source: { repository: alpine }
        inputs:
          - name: code-repo
          - name: last-hash
        outputs:
          - name: hash
        run:
          path: /bin/sh
          args:
          - -c
          - |
            LAST="$(cat last-hash/hash.txt)"
            NEW=$(cat code-repo/hash.txt)
            if [ "$LAST" != "$NEW" ]; then
              cp code-repo/hash.txt hash/hash.txt
            fi
      on_success:
        try:
          put: hash
          params:
            file: hash/hash.txt

  - name: expensive
    plan:
    - get: hash
      trigger: true
      passed: [ cheap ]

Note: you must populate the initial state file in s3 with some value, or the cheap job won't take off.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文