ssis - 使用 Foreach 从多个数据库读取

发布于 2024-12-04 02:53:58 字数 1711 浏览 1 评论 0原文

目前,我有一个连接到20多个数据库(相同结构)的控制流

在此处输入图像描述

在此 controlf 流中除了数据流仅在与数据库的连接方面不同之外,一切都是相同的。 这是数据流的示例:

在此处输入图像描述

OLE DB 源实际上有所不同。

在这种情况下是否可以使用 Foreach 循环来避免这些重复? 如果可以的话,该怎么做呢?我找不到任何描述如何连接到不同数据库的好文章。

谢谢,
伊利亚

编辑:
我发现一篇文章描述了我需要的大部分内容: http://sql-bi-dev.blogspot.com/2010/07/dynamic-database-connection-using-ssis.html

我按照此处的所有步骤进行操作,但我不明白带有查询的部分。这是该文章中的查询:

SELECT '' TableName, N'' ConnString, 0 RecordCount, GETDATE() ActionTime

我不明白这种语法,我猜这就是错误的来源:

[OLE DB 源 1] 错误:输出列出现错误 输出“OLE DB 源输出”(11) 上的“ConnString”(37)。专栏 返回的状态是:“文本被截断或者一个或多个字符被截断 目标代码页中没有匹配项。”。

[OLE DB 源 1] 错误:“输出列“ConnString”(37 )” 由于发生截断而失败,并且截断行处置 “输出列“ConnString”(37)”上指定截断失败。 指定对象的指定对象发生截断错误 组件。

[SSIS.Pipeline] 错误:SSIS 错误代码 DTS_E_PRIMEOUTPUTFAILED。这 组件“OLE DB Source”(1) 上的 PrimeOutput 方法返回错误 代码 0xC020902A。当组件返回失败代码时 管道引擎称为 PrimeOutput()。故障代码的含义 是由组件定义的,但是错误是致命的并且管道 停止执行。在此之前可能已发布错误消息 有关失败的更多信息。

我想从名为 Agency 的表中获取数据(我在变量 SourceTable = 'Agency' 中定义 - 如果我设置不存在的表,那么我会收到错误,所以我猜与源数据库的连接是好的)...让我感到困惑的是,我没有在任何地方定义我需要的列,所以我猜这个查询中有错误,但我不理解它的语法。有什么建议吗?

编辑2:
我遇到的另一个问题是消息框总是在连接字符串中返回相同的值 - 变量 ConnString 不会改变。如何将 SourceList 变量中的值赋给它?

Currently, I have a control flow that connects to more than 20 dbs (same structure)

enter image description here

In this controlf flow everything is the same except Data Flows which are different only in connection to db.
This is example of a dataflow:

enter image description here

OLE DB Source is actually what's different.

Is it possible to use a Foreach loop in this case so that I avoid these repeatings?
If possible, how to do it? I couldn't find any good article which describes how to connect to different dbs.

Thanks,
Ilija

EDIT:
I found an article which describes most of the thing I need: http://sql-bi-dev.blogspot.com/2010/07/dynamic-database-connection-using-ssis.html

I followed all steps form here but i don't understand a part with query. This is a query from that article:

SELECT '' TableName, N'' ConnString, 0 RecordCount, GETDATE() ActionTime

I don't understand this syntax and I guess this is where the errors come from:

[OLE DB Source 1] Error: There was an error with output column
"ConnString" (37) on output "OLE DB Source Output" (11). The column
status returned was: "Text was truncated or one or more characters had
no match in the target code page.".

[OLE DB Source 1] Error: The "output column "ConnString" (37)"
failed because truncation occurred, and the truncation row disposition
on "output column "ConnString" (37)" specifies failure on truncation.
A truncation error occurred on the specified object of the specified
component.

[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED. The
PrimeOutput method on component "OLE DB Source" (1) returned error
code 0xC020902A. The component returned a failure code when the
pipeline engine called PrimeOutput(). The meaning of the failure code
is defined by the component, but the error is fatal and the pipeline
stopped executing. There may be error messages posted before this
with more information about the failure.

I would like to get data from table called Agency (which I defined in variable SourceTable = 'Agency' - if I set non-existing table then I get error, so i guess the connection to Source DB is ok)... and it confuses me that I haven't defined anywhere which columns I need, so I guess an error is in this query, but I don't understand its syntax. Any suggestion?

EDIT 2:
Another problem I ran into is that Message box always returns me the same value in Connection string - Variable ConnString doesn't change. How can I assign value from SourceList variable to it?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

橘虞初梦 2024-12-11 02:53:58

我认为这是可能的。我会这样做:

  1. Foreach循环的集合应该是“Foreach Item Enumerator”。此时,您可以在“枚举器配置”面板中制作一个小表。添加一个类型为字符串的列。这里将出现数据库的连接字符串。
  2. 创建一个字符串变量并将其与此列关联:转到 Foreach 循环编辑器中的变量映射,选择字符串变量(例如 User::ConnString),给出索引 0。循环将迭代您在第一步中设置的条目,然后您可以通过此 User::ConnString 变量访问这些值。
  3. 将数据流任务添加到 Foreach 循环容器中。里面设置一个OLE DB Source。它需要一个连接管理器。选择与 OLE DB 连接关联的连接管理器,转到“属性”、“表达式”。选择 ConnectionString 属性,然后在表达式列中键入 @[User::ConnString](或使用您的变量名称)。
  4. 找到您当前使用的所有连接字符串并将其输入到枚举项中。

我认为通过这些步骤您可以简化数据流。

此过程假设您在所有数据库中具有相同的表名。在其他情况下,您还可以迭代要连接的表名称。 (枚举配置中的第二列,并用适当的表名称填充单元格。您还需要第二个变量。)

当然,您不必复制整个连接字符串,而是复制其中的更改部分。在这种情况下,你应该适当地形成你的表达方式。

编辑

你是对的。这个例子有一些问题。 (我认为这对于您的解决方案来说有点复杂,但可以使用。)

第一个是(正如您提到的)它没有将值传递给变量。我将用这些完成第六步:将 User::SourceList 设置为 ADO 对象源变量后,选择左侧的变量映射。在“变量”列中,首先选择“User::ConnString”(将索引设置为 0),然后选择“User::SourceTable”(将索引设置为 1)。这将解决您使用 MessageBox 进行的第二次编辑。从现在开始,它应该显示出正确的价值观。

第二个问题是你写的,有这个截断错误。如果您仔细查看错误消息,它们会说某些字符串列的长度为 1。这就是发生某些截断的原因。要解决此问题,请右键单击 OLE DB 源,选择“显示高级编辑器”。转到最后一个选项卡(输入和输出属性),展开 OLE DB 源输出/输出列。首先选择表名。在右侧的数据类型属性中,您应该看到长度 = 1。(我猜,SSIS 无法正确确定此列的长度,因为您选择 SQL 命令变量作为数据访问模式)。根据结果​​表定义,将此值设置为 128。还将 ConnString 列的长度从 1 更改为 256。

在这些更改之后,示例应该可以运行。 (在我的例子中,它有效。)这个示例使用了与我编写的基本相同的概念,只是它从数据存储中获取连接字符串,而不是从包本身内部获取连接字符串。我接受,前者可能是一个更强大的解决方案。我希望你能让你的包裹发挥作用。如果没有,请告诉我。

I think it is possible. I would make this way:

  1. Foreach Loop's Collection should be "Foreach Item Enumerator". At this point in the Enumerator Configuration panel you can make a little table. Add a column with a type string. Here will come the connection strings of the databases.
  2. Create a string variable and associate it with this column: Go to Variable Mappings in Foreach Loop editor, choose the string variable (e.g. User::ConnString), give index 0. The loop will iterate through the entries you set in first step, and you can access the values through this User::ConnString variable.
  3. Add a Data Flow task in to the Foreach Loop container. Inside set up an OLE DB Source. It needs a connection manager. Select the connection manager associated with OLE DB Connection, go to Propertes, Expressions. Select the ConnectionString property, and type @[User::ConnString] to expression column (or use your variable name).
  4. Locate all connection string you currently use and type them to Enumeration Items.

I think with these steps you can simplify your data flow.

This process assumes that you have the same table names in all databases. In other case you can also iterate the table names to connect to. (Second column in the Enumeration configuration and fill the cells with the apropriate table names. You also need a second variable.)

Of course you not have to copy the whole connection string but the changing part of it. I this case you should form your expression appropriately.

EDIT

You are right. There are some problem with this example. (I think it's a bit complicated for your solution, but it can be used.)

The first one is (as you mentioned) it did not pass the values to the variables. I would complete the 6th step with these: After you set User::SourceList as ADO object source variable, select Variable Mappings on the left side. In Variables column choose User::ConnString first (set index to 0), then User::SourceTable second (set index to 1). This will solve your second edit with the MessageBox. From now on it should show the right values.

The second problem is what you wrote, with this truncation error. If you look at the error messages carefully they say that some string columns have length of 1. And that's why some truncation occures. To solve this, right click OLE DB source, choose Show Advanced editor. Go to the last tab (Input and Output Properties), expand OLE DB Source output/Output columns. Choose TableName first. On the right side, in Data Type Properties you should see Lenght = 1. (SSIS could not correctly determine the length of this column, because you choose SQL command variable as Data access mode, I guess). According to the Result table definition, set this value to 128. Also change the length of ConnString column from 1 to 256.

After these changes the example should work. (In my case it worked.) This example uses much the same concept that I wrote, just it gets the connection string from a data store but not from inside the package itself. Former can be a more robust solution, I accept. I hope you can make your package work. Please, let me know if not.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文