SQL2008:有没有办法限制变更数据捕获(CDC)吞吐量?
我正在尝试找到一种方法来“限制”SQL2008 上的 CDC。
原因是,在正常情况下,CDC 表现出色,但一旦需要处理“大量”行,它就开始陷入困境。
典型吞吐量为每秒 1000 到 3000 行。它以每秒大约 5000 行的速度开始死亡。
通常,这不是问题,因为我们使用 CDC 来保持两个数据库同步,作为统计建模的近实时 ETL 流程。过去,对于批量数据移动,我们不得不想出狡猾的手动方法。我想知道我是否可以处理大量数据,但找到一种方法告诉它一次只执行 5 个事务,或者以其他方式强制它处理一口大小的数据块(无论需要多长时间),而不是尝试一次完成所有这些任务并遭受痛苦。
I'm trying to find a way to 'throttle' CDC on SQL2008.
The reason being that under normal circumstances, CDC performs brilliantly, but as soon as it needs to deal with a 'large' number of rows, it starts tanking.
Typical throughput is between 1000 and 3000 rows a second. It starts to die at about 5000 rows per second.
Usually, this is not an issue, since we're using CDC to keep two databases in sync as a near real-time ETL process for statistical modelling. In the past, for bulk data moves we've had to come up with dodgy manual methods. I'm wondering if I can through a huge amount of data at it, but find a way to tell it to only do 5 transactions at a time, or otherwise force it to work through bite-sized chunks (however long that takes), rather than try and do them all at once and suffer poorly.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
请参阅:调整 SQL Server 2008 中变更数据捕获的性能
您确定 CDC 是适合您想要实现的目标的解决方案吗?我只是想知道 SQL Server 更改跟踪是否使用 ADO.NET同步服务可能更合适?
Please see: Tuning the Performance of Change Data Capture in SQL Server 2008
Are you sure that CDC is the right solution for what you are trying to achieve? I'm just wondering if SQL Server Change Tracking using ADO.NET Sync services might be more appropriate?