在不锁定表的情况下更新正在使用的表中的数据的最佳方法是什么?

发布于 2024-07-19 21:34:23 字数 634 浏览 4 评论 0原文

我在 SQL Server 2005 数据库中有一个经常使用的表。 它有我们现有产品的可用性信息。 我们每小时都会从仓库获取更新,在过去的几年里,我们一直在运行一个例程来截断表并更新信息。 这只需要几秒钟,而且到目前为止还不是问题。 现在有更多的人使用我们的系统来查询此信息,因此我们看到由于阻塞进程而导致大量超时。

...所以...

我们研究了我们的选择,并提出了一个缓解问题的想法。

  1. 我们会有两张桌子。 表 A(活动)和表 B(非活动)。
  2. 我们将创建一个指向活动表(表 A)的视图。
  3. 所有需要此表信息的事物(4 个对象)现在都必须通过视图。
  4. 每小时例程将截断非活动表,用最新信息更新它,然后更新视图以指向非活动表,使其成为活动表。
  5. 该例程将确定哪个表处于活动状态并基本上在它们之间切换视图。

这有什么问题吗? 在查询中切换视图会导致问题吗? 这可以吗?

感谢您的专业知识。

额外信息

  • 例程是一个 SSIS 包,执行许多步骤并最终截断/更新相关表

  • 阻塞进程是查询此表的另外两个存储过程。

I have a table in a SQL Server 2005 Database that is used a lot. It has our product on hand availability information. We get updates every hour from our warehouse and for the past few years we've been running a routine that truncates the table and updates the information. This only takes a few seconds and has not been a problem, until now. We have a lot more people using our systems that query this information now, and as a result we're seeing a lot of timeouts due to blocking processes.

... so ...

We researched our options and have come up with an idea to mitigate the problem.

  1. We would have two tables. Table A (active) and table B (inactive).
  2. We would create a view that points to the active table (table A).
  3. All things needing this tables information (4 objects) would now have to go through the view.
  4. The hourly routine would truncate the inactive table, update it with the latest information then update the view to point at the inactive table, making it the active one.
  5. This routine would determine which table is active and basically switch the view between them.

What's wrong with this? Will switching the view mid query cause problems? Can this work?

Thank you for your expertise.

Extra Information

  • the routine is a SSIS package that peforms many steps and eventually truncates/updates the table in question

  • The blocking processes are two other stored procedures that query this table.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(10

笔芯 2024-07-26 21:34:26

刚刚读到您正在使用 SSIS,

您可以使用以下位置的 TableDiference 组件: http://www.sqlbi.eu/Home/tabid/36/ctl/Det​​ails/mid/374/ItemID/0/Default.aspx

替代文字< br>
(来源:sqlbi.eu)

通过这种方式,您可以将更改一一应用于表,但当然,这会慢得多,并且根据表大小,将需要服务器上更多的 RAM,但锁定问题将是完全纠正了。

Just read you are using SSIS

you could use the TableDiference Component from: http://www.sqlbi.eu/Home/tabid/36/ctl/Details/mid/374/ItemID/0/Default.aspx

alt text
(source: sqlbi.eu)

This way you can apply the changes to the table, ONE by ONE but of course, this will be much more slower and depending on the table size will require more RAM at the server, but the Locking problem will be totally corrected.

风柔一江水 2024-07-26 21:34:26

为什么不使用事务来更新信息而不是使用截断操作。

截断是不记录的,因此不能在事务中完成。

如果您的操作是在事务中完成的,则现有用户不会受到影响。

如何完成此操作取决于表的大小以及数据变化的剧烈程度等因素。 如果您提供更多详细信息,也许我可以提供进一步的建议。

Why not use transactions to update the information rather than a truncate operation.

Truncate is non logged so it cannot be done in a transaction.

If you're operation is done in a transaction then existing users will not be affected.

How this is done would depend on things like the size of the table and how radically the data changes. If you give more detail perhaps I could advise further.

贱贱哒 2024-07-26 21:34:26

一种可能的解决方案是最大限度地减少更新表所需的时间。

我首先创建一个临时表来从仓库下载数据。

如果您必须在最终表中执行“插入、更新和删除”

让我们假设最终表如下所示:

Table Products:
    ProductId       int
    QuantityOnHand  Int

并且您需要从仓库更新 QuantityOnHand。

首先创建一个临时表,如下所示:

Table Prodcuts_WareHouse
    ProductId       int
    QuantityOnHand  Int

然后创建一个“操作”表,如下所示:

Table Prodcuts_Actions
    ProductId       int
    QuantityOnHand  Int
    Action          Char(1)

更新过程应如下所示:

1.截断表 Prodcuts_WareHouse

2.截断表 Prodcuts_Actions

3.使用仓库中的数据填充 Prodcuts_WareHouse 表

4.用以下内容填充 Prodcuts_Actions 表:

插入:

INSERT INTO Prodcuts_Actions (ProductId, QuantityOnHand,Action)
SELECT     SRC.ProductId, SRC.QuantityOnHand, 'I' AS ACTION
FROM         Prodcuts_WareHouse AS SRC LEFT OUTER JOIN
                      Products AS DEST ON SRC.ProductId = DEST.ProductId
WHERE     (DEST.ProductId IS NULL)

删除

INSERT INTO Prodcuts_Actions (ProductId, QuantityOnHand,Action)
SELECT     DEST.ProductId, DEST.QuantityOnHand, 'D' AS Action
FROM         Prodcuts_WareHouse AS SRC RIGHT OUTER JOIN
                      Products AS DEST ON SRC.ProductId = DEST.ProductId
WHERE     (SRC.ProductId IS NULL)

更新

INSERT INTO Prodcuts_Actions (ProductId, QuantityOnHand,Action)
SELECT     SRC.ProductId, SRC.QuantityOnHand, 'U' AS Action
FROM         Prodcuts_WareHouse AS SRC INNER JOIN
                      Products AS DEST ON SRC.ProductId = DEST.ProductId AND SRC.QuantityOnHand <> DEST.QuantityOnHand

到目前为止,您还没有锁定最终表。

5.在事务中更新最终表:

BEGIN TRANS

DELETE Products FROM Products INNER JOIN
Prodcuts_Actions ON Products.ProductId = Prodcuts_Actions.ProductId
WHERE     (Prodcuts_Actions.Action = 'D')

INSERT INTO Prodcuts (ProductId, QuantityOnHand)
SELECT ProductId, QuantityOnHand FROM Prodcuts_Actions WHERE Action ='I';

UPDATE Products SET QuantityOnHand = SRC.QuantityOnHand 
FROM         Products INNER JOIN
Prodcuts_Actions AS SRC ON Products.ProductId = SRC.ProductId
WHERE     (SRC.Action = 'U')

COMMIT TRAN

通过上述所有过程,您可以将要更新的记录量最小化到所需的最低限度,因此更新时最终表将被锁定。

您甚至可以在最后一步中不使用事务,因此在命令之间表将被释放。

One possible solution would be to minimize the time needed to update the table.

I would first Create a staging table to download the data from the warehouse.

If you have to do "inserts, updates and deletes" in the final table

Lets suppose the finale table looks like this:

Table Products:
    ProductId       int
    QuantityOnHand  Int

And you need to update QuantityOnHand from the warehouse.

First Create a Staging table like:

Table Prodcuts_WareHouse
    ProductId       int
    QuantityOnHand  Int

And then Create an "Actions" Table like this:

Table Prodcuts_Actions
    ProductId       int
    QuantityOnHand  Int
    Action          Char(1)

The update process should then be something like this:

1.Truncate table Prodcuts_WareHouse

2.Truncate table Prodcuts_Actions

3.Fill the Prodcuts_WareHouse table with the data from the warehouse

4.Fill the Prodcuts_Actions table with this:

Inserts:

INSERT INTO Prodcuts_Actions (ProductId, QuantityOnHand,Action)
SELECT     SRC.ProductId, SRC.QuantityOnHand, 'I' AS ACTION
FROM         Prodcuts_WareHouse AS SRC LEFT OUTER JOIN
                      Products AS DEST ON SRC.ProductId = DEST.ProductId
WHERE     (DEST.ProductId IS NULL)

Deletes

INSERT INTO Prodcuts_Actions (ProductId, QuantityOnHand,Action)
SELECT     DEST.ProductId, DEST.QuantityOnHand, 'D' AS Action
FROM         Prodcuts_WareHouse AS SRC RIGHT OUTER JOIN
                      Products AS DEST ON SRC.ProductId = DEST.ProductId
WHERE     (SRC.ProductId IS NULL)

Updates

INSERT INTO Prodcuts_Actions (ProductId, QuantityOnHand,Action)
SELECT     SRC.ProductId, SRC.QuantityOnHand, 'U' AS Action
FROM         Prodcuts_WareHouse AS SRC INNER JOIN
                      Products AS DEST ON SRC.ProductId = DEST.ProductId AND SRC.QuantityOnHand <> DEST.QuantityOnHand

Until now you haven't locked the final table.

5.In a transaction update the final table:

BEGIN TRANS

DELETE Products FROM Products INNER JOIN
Prodcuts_Actions ON Products.ProductId = Prodcuts_Actions.ProductId
WHERE     (Prodcuts_Actions.Action = 'D')

INSERT INTO Prodcuts (ProductId, QuantityOnHand)
SELECT ProductId, QuantityOnHand FROM Prodcuts_Actions WHERE Action ='I';

UPDATE Products SET QuantityOnHand = SRC.QuantityOnHand 
FROM         Products INNER JOIN
Prodcuts_Actions AS SRC ON Products.ProductId = SRC.ProductId
WHERE     (SRC.Action = 'U')

COMMIT TRAN

With all the process above, you minimize the amount of records to be updated to the minimum necessary, and so the time the final table will be locked while updating.

You can even don't use a transaction in the final step, so between command the table will be released.

痴者 2024-07-26 21:34:26

如果您有 SQL Server 企业版可供使用,那么我建议您使用 SQL Server 分区技术。

您可以将当前所需的数据驻留在“实时”分区中,并将更新版本的数据驻留在“辅助”分区中(不可用于查询,而是用于管理数据)。

将数据导入“辅助”分区后,您可以立即切换“实时”分区 OUT 和“辅助”分区 IN,从而实现零停机且无阻塞。

完成切换后,您可以截断不再需要的数据,而不会影响新实时数据(以前是辅助分区)的用户。

每次您需要执行导入作业时,只需重复/反转该过程即可。

要了解有关 SQL Server 分区的更多信息,请参阅:

http:// msdn.microsoft.com/en-us/library/ms345146(SQL.90).aspx

或者您可以问我:-)

编辑:

附注,为了解决任何阻塞问题,您可以使用 SQL Server 行版本控制技术。

http://msdn.microsoft.com/en-us /library/ms345124(SQL.90).aspx

If you have the Enterprise Edition of SQL Server at your disposal then may I suggest that you use SQL Server Partitioning technology.

You could have your currently required data reside within the 'Live' partition and the updated version of the data in the 'Secondary' partition (which is not available for querying but rather for administering data).

Once the data has been imported into the 'Secondary' parition you can instantly SWITCH the 'LIVE' partition OUT and the 'Secondary' partition IN, thereby incurring zero downtime and no blocking.

Once you have made the switch, you can go about truncating the no longer needed data without adversley affecting users of the newly live data (previously the Secondary partition).

Each time you need to do an import job, you simply repeat/reverse the process.

To learn more about SQL Server Partitioning see:

http://msdn.microsoft.com/en-us/library/ms345146(SQL.90).aspx

Or you can just ask me :-)

EDIT:

On a side note, in order to address any blocking issues, you could use SQL Server Row Versioning technology.

http://msdn.microsoft.com/en-us/library/ms345124(SQL.90).aspx

╰沐子 2024-07-26 21:34:26

我们在高使用率的系统上执行此操作,并且没有遇到任何问题。 然而,与所有数据库一样,确保它有帮助的唯一方法是在开发中进行更改,然后对其进行负载测试。 不知道你的 SSIS 包还做了什么,它仍然可能会导致阻塞。

We do this on our high usage systems and haven't had any problems. However, as with all things database, the only way to be sure it would help would be to make the change in dev and then load test it. Not knowing waht else your SSIS package does, it may still cause blocks.

窝囊感情。 2024-07-26 21:34:26

如果表不是很大,您可以将数据缓存在应用程序中一小段时间。 它可能无法完全消除阻塞,但会减少发生更新时查询表的机会。

If the table is not very large you could cache the data in your application for a short time. It may not eliminate blocking altogether, but it would reduce the chances that the table would be queried when an update occurs.

甜警司 2024-07-26 21:34:26

也许对阻塞的进程进行一些分析是有意义的,因为它们似乎是您的景观中已发生变化的一部分。 只需要一个写得不好的查询就可以创建您所看到的块。 除非查询写得不好,否则该表可能需要一个或多个覆盖索引来加速这些查询,并让您回到正轨,而无需重新设计已经工作的代码。

希望这有帮助,

比尔

Perhaps it would make sense to do some analysis of the processes which are blocking since they seem to be the part of your landscape which has changed. It only takes one poorly written query to create the blocks which your are seeing. Barring a poorly written query, maybe the table needs one or more covering indexes to speed up those queries and get you back on your way without having to re-engineer your already working code.

Hope this helps,

Bill

变身佩奇 2024-07-26 21:34:25

就个人而言,如果您总是要引入停机时间来对表运行批处理,我认为您应该在业务/数据访问层管理用户体验。 引入一个表管理对象来监视与该表的连接并控制批处理。

当新的批处理数据准备好时,管理对象会停止所有新的查询请求(甚至可能排队?),允许现有查询完成,运行批处理,然后重新打开表进行查询。 管理对象可以引发 UI 层可以解释的事件 (BatchProcessingEvent),以让人们知道该表当前不可用。

我的 0.02 美元,

内特

Personally, if you are always going to be introducing down time to run a batch process against the table, I think you should manage the user experience at the business/data access layer. Introduce a table management object that monitors connections to that table and controls the batch processing.

When new batch data is ready, the management object stops all new query requests (maybe even queueing?), allows the existing queries to complete, runs the batch, then reopens the table for queries. The management object can raise an event (BatchProcessingEvent) that the UI layer can interpret to let people know that the table is currently unavailable.

My $0.02,

Nate

抱着落日 2024-07-26 21:34:23

您是否考虑过使用快照隔离。 它将允许您为您的 SSIS 内容开始一个大的事务,并且仍然从表中读取。

这个解决方案看起来比切换表干净得多。

Have you considered using snapshot isolation. It would allow you to begin a big fat transaction for your SSIS stuff and still read from the table.

This solution seems much cleaner than switching the tables.

梦中楼上月下 2024-07-26 21:34:23

我认为这是错误的方式 - 更新表必须锁定它,尽管您可以将锁定限制为每页甚至每行。

我会考虑不要截断桌子并重新填充它。 这总是会干扰用户尝试阅读它。

如果您确实更新而不是替换表,您可以通过其他方式控制它 - 读取用户不应该阻止表并且可能能够摆脱乐观读取。

尝试将 with(nolock) 提示添加到读取 SQL 视图语句中。 即使表格定期更新,您也应该能够吸引大量用户阅读。

I think this is going about it the wrong way - updating a table has to lock it, although you can limit that locking to per page or even per row.

I'd look at not truncating the table and refilling it. That's always going to interfere with users trying to read it.

If you did update rather than replace the table you could control this the other way - the reading users shouldn't block the table and may be able to get away with optimistic reads.

Try adding the with(nolock) hint to the reading SQL View statement. You should be able to get very large volumes of users reading even with the table being regularly updated.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文