AWS AppFlow CloudFormation 模板大小限制 - 映射 500+领域

发布于 2025-01-15 13:22:35 字数 1616 浏览 3 评论 0原文

我正在尝试创建一个 AWS::AppFlow::Flow ，它可以将大量（500+）字段从 Salesforce 源对象映射到 S3 对象。必须指定字段列表（而不是简单地删除所有字段）。

如果我在这里错了，请纠正我，但我的模板似乎必须有一个 TaskType: Filter 和一个 ConnectorOperator: PROJECTION 以及 SourceFields 的完整列表 我想复制。
此外，对于每个字段，我似乎需要为该字段定义一个单独的 TaskType: Map...

我使用此示例模板开始使用，相关位：

      Tasks:
        - TaskType: Filter
          SourceFields:
            - FieldA
            - FieldB
            - FieldN
          ConnectorOperator:
            Salesforce: PROJECTION
        - TaskType: Map
          SourceFields:
            - FieldA
          TaskProperties:
            - Key: SOURCE_DATA_TYPE
              Value: string
        - TaskType: Map
          SourceFields:
            - FieldB
          TaskProperties:
            - Key: SOURCE_DATA_TYPE
              Value: datetime
        - TaskType: Map
          SourceFields:
            - FieldN
          TaskProperties:
            - Key: SOURCE_DATA_TYPE
              Value: integer

不幸的是，我需要映射的字段列表非常大，并且模板超过 CloudFormation 模板允许的最大大小（51200 字节）。我想大部分空间都被每个字段的重复 Map 任务块占用。

我知道您可以将模板文件上传到S3，允许更大的文件大小配额；但是，即使我可以采用此选项（我不能），文件大小也会超出 CloudFormation 限制！

是否有另一种方法可以节省模板空间？

我发现了一两个提及Map_all TaskType 的 a>，表面上听起来很有希望；但是，我找不到任何有关如何使用它的具体文档。

PS：就其价值而言，仅字段名称列表（以逗号分隔）约为 14KB。
PPS：这个模式不是我设计的，我也没有能力改变它。

原文

I am trying to create an AWS::AppFlow::Flow that can map a large number (500+) of fields from a Salesforce source object into an S3 object. The list of fields must be specified (rather than simply slurping all fields).

Please correct me if I'm wrong here, but it seems that my template MUST have a TaskType: Filter with a ConnectorOperator: PROJECTION and the full list of SourceFields I wish to copy.
Further, for each field, I seem to need a separate TaskType: Map defined for that field...

I used this example template to get started, with the relevant bits:

      Tasks:
        - TaskType: Filter
          SourceFields:
            - FieldA
            - FieldB
            - FieldN
          ConnectorOperator:
            Salesforce: PROJECTION
        - TaskType: Map
          SourceFields:
            - FieldA
          TaskProperties:
            - Key: SOURCE_DATA_TYPE
              Value: string
        - TaskType: Map
          SourceFields:
            - FieldB
          TaskProperties:
            - Key: SOURCE_DATA_TYPE
              Value: datetime
        - TaskType: Map
          SourceFields:
            - FieldN
          TaskProperties:
            - Key: SOURCE_DATA_TYPE
              Value: integer

Unfortunately, the list of fields I need to map is very large, and the template exceeds the maximum size allowed for CloudFormation templates (51200 bytes). I imagine most of the space is used by the repetitive Map task blocks for each field.

I am aware that you can upload the template file to S3, allowing a larger file size quota; however, even if I could pursue this option (I can't), the file size would also exceed the CloudFormation limit!

Is there another way to do this that would save template space?

I've found one or two mentions of a Map_all TaskType, which sounds promising on the surface; however, I can't find any concrete documentation on how to use it.

PS: For what it's worth, just the list of field names, comma-separated, is ~14KB.
PPS: I didn't design this schema, and I have no ability to change it.

分享到QQ

分享到微博