无法使用数据工厂管道将数据从Azure Data Lake Gen2复制到Azure SQL DB

发布于 2025-02-09 04:29:14 字数 903 浏览 1 评论 0原文

失败在“水槽”一侧发生。 ERRORCODE = USERERRINVALIDCOLUMNNAME,'type = Microsoft.datatransfer.common.shared.hybriddeliveryexception,message = column prop_0在目标方面找不到列prop_0,source = microsoft.datatatransfer.clientlibrary。

ADLS Gen2中的所有零件文件都有8列,而接收器表也有8列,并且在部分文件中没有称为prop_0的列。

输入是保存在ADLS gen2-的部分文件 -

部分文件之一的内容 -

adf上的映射 -

​=“ nofollow noreferrer”> “

Failure happened on 'Sink' side. ErrorCode=UserErrorInvalidColumnName,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The column Prop_0 is not found in target side,Source=Microsoft.DataTransfer.ClientLibrary.

All the part files in the ADLS GEN2 have 8 columns and the sink table also has 8 columns and there is no such column called Prop_0 in part files.

Inputs are part files saved in ADLS GEN2 -
enter image description here

Content of one of the part file -
enter image description here

Mapping on ADF -

enter image description here

Output of sql query executed on Azure query editor -

enter image description here

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

喜爱纠缠 2025-02-16 04:29:14

当源文件没有标头时(或在源文件标题时将第一行作为标头)并且未启用列映射选项时,您会收到此错误。 prop_0prop_1等,当源文件没有标头(或未启用)时,将充当列名。

在这种情况下,当您禁用列映射选项(清除或跳过)时,仅在匹配您的水槽时,复制活动才会尝试使用源插入列(表)。在下面的图像中,我没有导入架构(跳过),并且在运行管道时会出现相同的错误。

”在此处输入图像描述”

由于您的目标没有prop_0列,因此会引发以下错误:

请按照下面指定的步骤进行纠正此错误:

  • 首先识别您的源部分文件是否具有标头。然后通过检查/取消选中第一行作为标头选项来编辑源数据集。在管道的“源”选项卡中发布并预览此数据。

  • 移动到映射部分,然后单击导入schemas(如果需要,请再次清除并导入)。根据您的要求,如有必要,对映射进行更改。

”在此处输入图像描述”

  • 由于源列和目标列不匹配,因此必须在此映射中进行更改。从您给出的部分文件示例中,适当的映射如下所示:

“在此处输入图像说明”

  • 现在运行管道。管道将成功运行,SQL表将反映插入的行。

You get this error when your source files don’t have a header (or consider first row as header when you have a header for source files) and you have not enabled the column mapping option. Prop_0, Prop_1 etc., will act as column names when the source file does not have a header (or not enabled).

In this case, when you disable the column mapping option (cleared or skipped), the copy activity will try to insert columns from source using the name, only when they match your sink (table). In the following image, I have not imported the schema (skipped) and it throws the same error when I run the pipeline.

enter image description here

enter image description here

Since your destination does not have Prop_0 column, it throws the following error:

enter image description here

Follow the steps specified below to rectify this error:

  • First identify if your source part files have header. Then edit your source dataset by checking/unchecking the first row as header option. Publish and preview this data in the source tab of your pipeline.

enter image description here

  • Move to Mapping section and click import schemas (Clear and import again if required). Make changes in the mapping if necessary, according to your requirements.

enter image description here

enter image description here

  • Changes have to be made in this mapping because the source columns and the destination columns don't match. From the part file sample you have given, the appropriate mapping would be as shown below:

enter image description here

  • Now run the pipeline. The pipeline will run successfully and the sql table will reflect the rows inserted.

enter image description here

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文