如何在SSIS控制流任务中创建临时表,然后在数据流任务中使用它?
我有一个控制流,我使用 T-SQL 命令创建临时数据库和表。当我添加数据流时,我想查询该表,但我不能,因为该表不存在可从中获取信息。当我尝试时,我收到有关登录的错误,因为数据库尚不存在。我已将连接管理器的属性 DelayValidation
设置为 True
。
如果我手动创建数据库和表,然后添加带有查询的数据流并删除数据库,它会保留,但看起来不是一个干净的解决方案。
如果有更好的方法来创建临时登台数据库并在数据流中查询它,请告诉我。
I have a control flow where I create a temp database and table with a T-SQL Command. When I add a dataflow I would like to query the table but I can't because the table doesn't exist to grab information from. When I try I get errors about logging in because the database doesn't exist (yet). I have set the connection manager's property DelayValidation
to True
.
If I create the database and table manually and then add the dataflow with query and drop the database, it sticks but it doesn't seem like a clean solution.
If there is a better way to create a temporary staging database and query it in dataflows please let me know.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(3)
解决方案:
将
连接管理器
上的属性RetainSameConnection
设置为True
以便在一个控制流任务中创建的临时表可以保留在另一任务中。以下是用
SSIS 2008 R2
编写的示例 SSIS 包,演示了如何使用临时表。演练:
创建一个存储过程,该过程将创建一个名为
##tmpStateProvince
的临时表并填充一些记录。示例 SSIS 包将首先调用存储过程,然后获取临时表数据以将记录填充到另一个数据库表中。示例包将使用名为Sora
的数据库,使用下面的创建存储过程脚本。创建一个名为
dbo.StateProvince
的表,该表将用作目标表以填充临时表中的记录。使用下面的创建表脚本创建目标表。使用 Business Intelligence Development Studio (BIDS) 创建 SSIS 包。
Business Intelligence Development Studio (BIDS)
。右键单击包底部的“连接管理器”选项卡,然后单击“新建 OLE DB 连接...”以创建新连接访问 SQL Server 2008 R2 数据库。单击
新建...
配置 OLE DB 连接管理器。在“连接管理器”对话框中执行以下操作。
Native OLE DB\SQL Server Native Client 10.0
,因为该包将连接到SQL Server 2008 R2数据库 >MACHINENAME\INSTANCE
使用 Windows 身份验证
服务器部分或您喜欢的部分。选择或输入数据库名称
中选择数据库,示例使用数据库名称Sora
。测试连接
确定
。确定
新创建的数据连接将出现在配置 OLE DB 连接管理器上。单击
确定
。OLE DB 连接管理器
KIWI\SQLSERVER2008R2.Sora
将出现在包底部的连接管理器选项卡下。右键单击连接管理器,然后单击属性
将连接
KIWI\SQLSERVER2008R2.Sora
上的属性RetainSameConnection
设置为值True
。右键单击包内的任意位置,然后单击
变量
查看变量窗格。创建以下变量。包范围
中名为
并将变量设置为值PopulateTempTable
的新变量,数据类型为String
SO_5631010EXEC dbo.PopulateTempTable
。包范围
中名为
并将变量设置为值FetchTempData
的新变量,数据类型为String
SO_5631010SELECT CountryCode, StateCode, Name FROM ##tmpStateProvince
将
执行 SQL 任务
拖放到控制流选项卡上。双击“执行 SQL 任务”以查看“执行 SQL 任务编辑器”。在执行 SQL 任务编辑器的
常规
页面上,执行以下操作。创建并填充临时表
OLE DB
KIWI\SQLSERVER2008R2.Sora
变量
from SQLSourceType 从User::PopulateTempTable
确定< /code>
拖放
数据流任务
到控制流选项卡。将数据流任务重命名为
将临时数据传输到数据库表
。将绿色箭头从执行 SQL 任务连接到数据流任务。双击
数据流任务
进行切换到数据流选项卡。将OLE DB 源
拖放到数据流 选项卡上。双击OLE DB 源以查看OLE DB 源编辑器。在OLE DB 源编辑器 的
连接管理器
页面上,执行以下操作。KIWI\SQLSERVER2008R2.Sora
SQL 命令
> 从数据访问模式User::FetchTempData
Columns< /code>
页
单击
列
OLE DB 源编辑器 页面将显示以下错误,因为源命令变量中指定的表##tmpStateProvince
不存在并且 SSIS 无法读取列定义。要修复错误,请执行语句
EXEC dbo.PopulateTempTable
在数据库Sora
上使用SQL Server Management Studio (SSMS),以便存储过程创建临时表。执行存储过程后,单击OLE DB Source Editor上的Columns
页面,您将看到列信息。单击确定
。拖放
OLE DB 目标
到数据流选项卡。将绿色箭头从OLE DB 源连接到OLE DB 目标。双击OLE DB 目标
打开OLE DB 目标编辑器。在OLE DB 目标编辑器 的
连接管理器
页面上,执行以下操作。KIWI\SQLSERVER2008R2.Sora
表或视图 - 快速加载
< /em> 来自数据访问模式[dbo].[StateProvince]
查看映射
页面单击如果输入和输出列名称相同,OLE DB 目标编辑器 上的
映射
页面将自动映射列。单击确定
。列StateProvinceID
没有匹配的输入列,它在数据库中定义为IDENTITY
列。因此,不需要映射。配置所有组件后,数据流选项卡应如下所示。
单击数据流选项卡上的
OLE DB Source
然后按F4查看属性
。将属性ValidateExternalMetadata
设置为 False,以便 SSIS 在包执行的验证阶段不会尝试检查临时表是否存在。在 SQL Server Management 中执行查询
select * from dbo.StateProvince
Studio (SSMS) 查找表中的行数。在执行包之前它应该是空的。执行包。控制流显示执行成功。
在“数据流”选项卡中,您会注意到包已成功处理6 行。在此发布的早期创建的存储过程将 6 行插入到临时表中。
在 中执行查询
select * from dbo.StateProvince
>SQL Server Management Studio (SSMS) 查找成功插入表中的 6 行。数据应与存储过程中找到的行匹配。上面的示例说明了如何在程序包中创建和使用临时表。
Solution:
Set the property
RetainSameConnection
on theConnection Manager
toTrue
so that temporary table created in one Control Flow task can be retained in another task.Here is a sample SSIS package written in
SSIS 2008 R2
that illustrates using temporary tables.Walkthrough:
Create a stored procedure that will create a temporary table named
##tmpStateProvince
and populate with few records. The sample SSIS package will first call the stored procedure and then will fetch the temporary table data to populate the records into another database table. The sample package will use the database namedSora
Use the below create stored procedure script.Create a table named
dbo.StateProvince
that will be used as the destination table to populate the records from temporary table. Use the below create table script to create the destination table.Create an SSIS package using
Business Intelligence Development Studio (BIDS)
. Right-click on the Connection Managers tab at the bottom of the package and clickNew OLE DB Connection...
to create a new connection to access SQL Server 2008 R2 database.Click
New...
on Configure OLE DB Connection Manager.Perform the following actions on the Connection Manager dialog.
Native OLE DB\SQL Server Native Client 10.0
from Provider since the package will connect to SQL Server 2008 R2 databaseMACHINENAME\INSTANCE
Use Windows Authentication
from Log on to the server section or whichever you prefer.Select or enter a database name
, the sample uses the database nameSora
.Test Connection
OK
on the Test connection succeeded message.OK
on Connection ManagerThe newly created data connection will appear on Configure OLE DB Connection Manager. Click
OK
.OLE DB connection manager
KIWI\SQLSERVER2008R2.Sora
will appear under the Connection Manager tab at the bottom of the package. Right-click the connection manager and clickProperties
Set the property
RetainSameConnection
on the connectionKIWI\SQLSERVER2008R2.Sora
to the valueTrue
.Right-click anywhere inside the package and then click
Variables
to view the variables pane. Create the following variables.A new variable named
PopulateTempTable
of data typeString
in the package scopeSO_5631010
and set the variable with the valueEXEC dbo.PopulateTempTable
.A new variable named
FetchTempData
of data typeString
in the package scopeSO_5631010
and set the variable with the valueSELECT CountryCode, StateCode, Name FROM ##tmpStateProvince
Drag and drop an
Execute SQL Task
on to the Control Flow tab. Double-click the Execute SQL Task to view the Execute SQL Task Editor.On the
General
page of the Execute SQL Task Editor, perform the following actions.Create and populate temp table
OLE DB
KIWI\SQLSERVER2008R2.Sora
Variable
from SQLSourceTypeUser::PopulateTempTable
from SourceVariableOK
Drag and drop a
Data Flow Task
onto the Control Flow tab. Rename the Data Flow Task asTransfer temp data to database table
. Connect the green arrow from the Execute SQL Task to the Data Flow Task.Double-click the
Data Flow Task
to switch to Data Flow tab. Drag and drop anOLE DB Source
onto the Data Flow tab. Double-click OLE DB Source to view the OLE DB Source Editor.On the
Connection Manager
page of the OLE DB Source Editor, perform the following actions.KIWI\SQLSERVER2008R2.Sora
from OLE DB Connection ManagerSQL command from variable
from Data access modeUser::FetchTempData
from Variable nameColumns
pageClicking
Columns
page on OLE DB Source Editor will display the following error because the table##tmpStateProvince
specified in the source command variable does not exist and SSIS is unable to read the column definition.To fix the error, execute the statement
EXEC dbo.PopulateTempTable
using SQL Server Management Studio (SSMS) on the databaseSora
so that the stored procedure will create the temporary table. After executing the stored procedure, clickColumns
page on OLE DB Source Editor, you will see the column information. ClickOK
.Drag and drop
OLE DB Destination
onto the Data Flow tab. Connect the green arrow from OLE DB Source to OLE DB Destination. Double-clickOLE DB Destination
to open OLE DB Destination Editor.On the
Connection Manager
page of the OLE DB Destination Editor, perform the following actions.KIWI\SQLSERVER2008R2.Sora
from OLE DB Connection ManagerTable or view - fast load
from Data access mode[dbo].[StateProvince]
from Name of the table or the viewMappings
pageClick
Mappings
page on the OLE DB Destination Editor would automatically map the columns if the input and output column names are same. ClickOK
. ColumnStateProvinceID
does not have a matching input column and it is defined as anIDENTITY
column in database. Hence, no mapping is required.Data Flow tab should look something like this after configuring all the components.
Click the
OLE DB Source
on Data Flow tab and press F4 to viewProperties
. Set the propertyValidateExternalMetadata
to False so that SSIS would not try to check for the existence of the temporary table during validation phase of the package execution.Execute the query
select * from dbo.StateProvince
in the SQL Server Management Studio (SSMS) to find the number of rows in the table. It should be empty before executing the package.Execute the package. Control Flow shows successful execution.
In Data Flow tab, you will notice that the package successfully processed 6 rows. The stored procedure created early in this posted inserted 6 rows into the temporary table.
Execute the query
select * from dbo.StateProvince
in the SQL Server Management Studio (SSMS) to find the 6 rows successfully inserted into the table. The data should match with rows founds in the stored procedure.The above example illustrated how to create and use temporary table within a package.
我参加这个聚会迟到了,但我想为 user756519 的彻底、出色的回答添加一点内容。根据我最近的经验,我不认为“连接管理器上的 RetainSameConnection”属性与此实例相关。就我而言,相关的一点是他们建议将“ValidateExternalMetadata”设置为 False。
我使用临时表来促进将数据从一个数据库(和服务器)复制到另一个数据库,因此“RetainSameConnection”的原因与我的特定情况无关。我也不认为完成本例中发生的事情很重要,尽管它很彻底。
I'm late to this party but I'd like to add one bit to user756519's thorough, excellent answer. I don't believe the "RetainSameConnection on the Connection Manager" property is relevant in this instance based on my recent experience. In my case, the relevant point was their advice to set "ValidateExternalMetadata" to False.
I'm using a temp table to facilitate copying data from one database (and server) to another, hence the reason "RetainSameConnection" was not relevant in my particular case. And I don't believe it is important to accomplish what is happening in this example either, as thorough as it is.
关于
此答案提供了 Microsoft Visual Studio 2017 中 SSIS 的屏幕截图。
虽然此处的另一个答案要求您运行存储过程,但此处的此答案连接到正确的“tempdb”数据库,并且不需要存储过程。对于那些想要避免使用存储过程向生产服务器发送垃圾邮件的人来说,这可能会很好。
这个答案表明您可以在控制流内使用临时表并将它们作为 DFT 中的目标,您也可以在 在 SSIS 中使用临时表?,正如我后来发现的,但该指南忘记显示它的数据流任务方面。如果在控制流窗格中创建临时表后更改为 DFT 窗格,则可以与“tempdb”数据库建立新连接,并找到在控制流中创建的仍然有效的临时表。
tempdb 系统数据库作为临时表的唯一数据库
如果我逐字逐句地理解这个问题的答案,那就是:没有答案。您需要一个保存在新数据库中的临时表,该表在运行时生效。但临时表始终保存在服务器级别的系统数据库“tempdb”中,请参阅tempdb 数据库 - Microsoft - Learn - SQL - SQL Server。您不能将临时表放入您自己的数据库中。奇怪的是,我在 Stack Overflow 上发现只有一条评论强调了这一点,请参阅 检查临时表是否存在,如果存在则删除创建临时表。
因此,问题还剩一半,你问可以做什么。答:不要为临时表创建新的数据库。相反,按原样使用服务器并在“tempdb”系统数据库中创建临时表。没有其他办法。如果您随后在 SSIS 中采用此连接,您还可以像处理普通表一样处理临时表。您不需要存储过程来进行设置。您可以从“tempdb”数据库中选择临时表并将其作为SSIS中的“OLE DB”对象。
屏幕截图中的步骤
建立 tempdb 连接
< img src="https://i.sstatic.net/4goiY.png" alt="在此处输入图像描述">
主要技巧:将“RetainSameConnection”更改为
True
正如其他答案正确所说,这是唯一的方法在控制流步骤之间保持临时表处于活动状态。
控制流
制作并填充临时表
其他序列容器需要“延迟验证”=
True
如果您需要其他序列中的临时表,则 在创建临时表之后的容器,您还需要在序列容器的属性中将
Delay Validation
设置为True
,否则您获取 SSIS 验证失败并返回验证状态“VS_ISBROKEN”。,如果您在其他容器中看到这些红叉,则此设置可以避免在包尚未运行时检查那些由于缺少临时表而产生的错误。一旦你运行它并使临时表处于活动状态,一旦 SSIS 到达红色标记,它们就会消失:
控制流 3:写入 TempDB ##tmpTest
从最后一个控制流步骤 3(数据)开始Flow Task作为写入TempDB临时表的序列容器的核心。从这里开始,您将了解控制流步骤 2 中表所需的数据类型。
OLE DB 源编辑器
OLE DB 目标对象
< a href="https://i.sstatic.net/sKC5d.png" rel="nofollow noreferrer">
复制单击“新建”时获得的代码。
OLE DB 目标编辑器
控制流程 2:
使用复制的代码制作 TempDB ##tmpTest 创建表:
OLE DB 目标编辑器
控制流程1: Drop TempDB ##tmpTest
这不是必需的,因为一旦控制流停止,临时表就会丢失,请参阅如何查看临时表sql server 中的代码创建的表?。但是,如果您担心某人(或您)可能在 SSMS 中的同一个临时表上工作,以便在容器启动时它可能不为空,那么您是安全的。
或者在 2016 年服务器上,您可以运行
DROP TABLE IF EXISTS ##tmpTest;
请参阅 在创建临时表之前检查临时表是否存在并删除它是否存在,并注意同一链接:
另请参阅 如何查看代码创建的临时表在 SQL Server 中?。
运行并检查
如果无法访问该表,您将看到:
如果有效,您会看到:
在调试模式下保持绿色,您将看到临时表仍然活着:
现在退出调试模式:
如果刷新,临时表将消失:
表格为填充:
您可以继续使用此表,就好像它不是临时的一样,并用它提供另一个查询或表。由于它前面有两个主题标签“##”,因此它是全局的,您可以在整个包中使用它,直到包停止运行。一旦包停止并且您未处于调试模式,临时表将被删除。
About
This answer affords screenshots of SSIS in Microsoft Visual Studio 2017.
While the other answer here asks you to run a stored procedure, this answer here connects to the right "tempdb" database and does not need a stored procedure. This may be nice for those who want to avoid spamming the production server with stored procedures.
This answer shows that you can work with temporary tables inside the Control Flow and make them destinations in the DFT, something which you can also find at Use Temp Table in SSIS?, as I found out later, but that guide forgets to show the Data Flow Task side of it. If you change to the DFT pane after creating the temporary table in the Control Flow pane, you can make a new connection to the "tempdb" database and find the still living temporary tables that you made in the Control Flow.
tempdb system database as the only database for temporary tables
The answer to the question, if I take every word of it, is: there is no answer. You ask for a temporary table that is saved in the new database that comes to life on the run. But temporary tables are saved - always - on the server level in the system database "tempdb", see tempdb database - Microsoft - Learn - SQL - SQL Server. You cannot put a temporary table in your own database. Strangely, I found only one remark on Stack Overflow that stressed this, see the one under Check if a temporary table exists and delete if it exists before creating a temporary table.
Therefore, there is half of a question left, you ask what could be done instead. Answer: Do not make a new database for the temporary table. Instead take the server as it is and make the temporary table in the "tempdb" system database. There is no other way. If you then take this connection in SSIS, you can also deal with the temporary table as if it was a normal table. You do not need a stored procedure to set this up. You can just choose the temporary table from the "tempdb" database and take that as the "OLE DB" object in SSIS.
Steps in screenshots
Make a tempdb connection
Main trick: change "RetainSameConnection" to
True
As the other answer rightly said, this is the only way to keep the temporary table alive between the Control Flow steps.
Control Flow
Make and fill a temporary table
Further Sequence Containers need "Delay Validation" =
True
If you need the temporary table in other Sequence Containers after the one the makes the temporary table, you need to set
Delay Validation
toTrue
also in the properties of the Sequence Container, else you get SSIS failed validation and returned validation status "VS_ISBROKEN".Thus, if you see those red crosses in the other containers, this setting avoids checking those errors that stem just from the missing temporary table when the packages does not yet run. Once you run it and put that temporary table alive, the red markers will go away once SSIS reaches them:
Control Flow 3: Write TempDB ##tmpTest
Begin with the last Control Flow step 3, the Data Flow Task as the core of the sequence container that writes to the TempDB temporary table. By beginning with this, you will get to know the data types that you need for the table in Control Flow step 2.
OLE DB Source Editor
OLE DB destination object
Copy the code that you get at a click on "New".
OLE DB Destination Editor
Control Flow 2: Make TempDB ##tmpTest
Create Table with the copied code:
OLE DB Destination Editor
Control Flow 1: Drop TempDB ##tmpTest
This is not needed since the temporary table gets lost as soon as the Control Flow stops, see How to see temp table created by code in sql server?. But you are on the safe side if you fear that someone (or you) might work on that same temporary table in SSMS so that it might not be empty when the container starts.
Or on a 2016 server, you can run
DROP TABLE IF EXISTS ##tmpTest;
See Check if a temporary table exists and delete if it exists before creating a temporary table, and mind in that same link:
Also see How to see temp table created by code in sql server?.
Run and check
If the table cannot be reached, you will see:
If it works, you see:
Keep it green in debug mode and you will see the temporary table still alive:
Now get out of the debug mode:
And the temporary table will be gone if you refresh:
And the table is filled:
You can go on with this table as if it was not temporary and feed another query or table with it. Since it has two hashtags in front, "##", it is global, and you can work with it in the whole package until the package has stopped running. Once the package stops and if you are not in debug mode, the temporary table is dropped.