海量数据迁移至SQL

发布于 2024-07-14 05:38:26 字数 472 浏览 5 评论 0原文

我需要将大量数据(数百万个文件、TB 级数据)迁移到 SQL 集群。 迁移过程分为每周迁移,每周有数百个新用户(即请求生成者)。

到目前为止,一切都运行得相当顺利,但最近我发现 SQL 集群开始表现得很奇怪。 CPU 使用率稳定在 20% 左右,但 SQL 进程不断分配新内存,直到没有剩余内存(约 12GB)。 当这种情况发生时,进程会“转储”所有内存并开始重新向 12GB 攀升。 在此转储期间,通常会出现服务器无响应并最终超时的情况,而这在本周的迁移期间不会发生。

这种分配和转储行为对于 SQL 集群来说常见吗? 是否可以对其进行配置,以便永远不会发生这种情况,或者至少不会堵塞整个数据库? 有人有大型移民工作的经验吗?

在查看事件日志时,我发现超时之前有一些 WMI 警告。 我们使用 System Center Operations Manager 2007 来忽略系统,这可以解释这种行为吗?

感谢您的帮助!

I need to migrate extremely large quantities of data (millions of files, Terabytes of data) to an SQL cluster. The migration process is partitioned into weekly migrations, with a few hundreds of new users (i.e request-generators) every week.

So far things have run pretty smoothly, but lately i've discovered that the SQL cluster started to act quite strangely. The CPU usage is sound and stable at about 20%, but the SQL-process is constantly allocating new memory until there is nothing left (about 12GB). When this happens the process "dumps" all memory and starts climbing towards 12GB anew. During this dump, it is often the case that the server gets unresponsive and eventually timeouts, which just cannot happen during this weeks migration.

Is this allocate-and-dump behaviour common for SQL clusters? Is it possible to make configurations to it so this will never happen, or at least not congest the whole database? Does anyone have experience with large migration jobs?

When looking through the event logs i found some WMI-warning preceeding the timout. We're using System Center Operations Manager 2007 to overlook the system, could this explain this behaviour?

Thankful for any help!

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

滿滿的愛 2024-07-21 05:38:26

不,这不是正常行为。

SQL Server 将根据需要动态分配内存,并在有压力时适当释放内存。 但它不应该转储全部内容。

您能否提供有关您的环境的更多详细信息并限定您的 SQL 内存配置。

如果您需要详细帮助,请随时给我发送电子邮件并请提供 DBCC 命令的结果。

DBCC memorystatus

No, this is not normal behaviour.

SQL Server will dynamically allocate memory as required and will free memory appropriately when under pressure to do so. It should not however dump the entire contents.

Can you please provide more details about your environment and qualify what your SQL memory configuration is.

If you require detailed assistance, feel free to drop me an email and please provide the results of the DBCC command.

DBCC memorystatus
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文