SqlBulkCopy.WriteToServer 挂起 Thread.Abort 被调用但不确定原因

发布于 2024-11-03 11:50:18 字数 2273 浏览 0 评论 0原文

Given:

  • 一个 BenchMark 类,让我知道某件事何时完成。
  • 一个非常大的 XML 文件 (~120MB),已被解析为多个列表

一些代码:

SqlConnection con = null;
SqlTransaction transaction = null;

try
{
  con = getCon(); // gets a new connection object
  con.Open();
  transaction = con.BeginTransaction();

  var bulkCopy = new SqlBulkCopy(con, SqlBulkCopyOptions.Default, transaction)
  {
    BatchSize = 1000,
    DestinationTableName = "Table1"
  };

  // assume that the BenchMark class is working
  b = new BenchMark("Table1");
  bulkCopy.WriteToServer(_insertTable1s.AsDataReader()); // _insertTables1s is a List<Table1>
  b.Complete();
  LogHelper.WriteLogItem(b);

  b = new BenchMark("Table2");
  bulkCopy.DestinationTableName = "Table2";
  bulkCopy.WriteToServer(_insertTable2s.AsDataReader()); // _insertTables2s is a List<Table2>
  b.Complete();
  LogHelper.WriteLogItem(b);

  // etc... this code does a batch insert into about 7 tables all having about 40,000 records being inserted.

  b = new BenchMark("Transaction Commit");
  transaction.Commit();
  b.Complete();
}
catch (Exception e)
{
  transaction.Rollback();

  LogHelper.WriteLogItem(
    LogLevel.Critical,
    LogType.DataProcessing,
    e.ToString());
}
finally
{
  con.Close();
}

问题:

在我的本地开发环境中,一切都很好。当我在云中运行此操作时,导致它挂起。使用 LogHelper.WriteLogItem 方法,我可以观察此过程的进度。我观察到它随机挂在一张特定的桌子上。不会引发异常,因此事务不会回滚。假设它挂在 Table2 批量插入上。 事务已中止?)

使用 MS SQL Management Studio,我在 Table3Table2Table1 上运行查询,没有出现任何问题(这意味着 它挂起,我将重新运行该过程。这次它挂起得更快,所以我可能会得到这样的日志:

7755   Benchmark   LoadXML took 00:00:04.2432816
7756   Benchmark   Table1 took 00:00:06.3961230
7757   Benchmark   Table2 took 00:00:05.2566890
7758   Benchmark   Table3 took 00:00:08.4900921
7759   Benchmark   Table4 took 00:00:02.0000123

...它挂在 Table5 上(因为 BenchMark 从未完成)。我再次运行它,日志的其余部分看起来像:

7780   Benchmark   LoadXML took 00:00:04.1203923

...它现在挂在这里。

如果有帮助的话,我正在使用机架空间云托管。过去我已经能够通过从 dbml 文件中删除所有表并读取它们来解决此问题,但这次它不起作用。我想知道是否是正在处理的数据量导致了问题?

编辑:此示例中的代码在异步线程中运行。我发现线程由于未知原因而中止,我需要找出原因来解决此问题。

Given:

  • A BenchMark class that lets me know when something has completed.
  • A very large XML file (~120MB) that has been parsed into multiple Lists

Some code:

SqlConnection con = null;
SqlTransaction transaction = null;

try
{
  con = getCon(); // gets a new connection object
  con.Open();
  transaction = con.BeginTransaction();

  var bulkCopy = new SqlBulkCopy(con, SqlBulkCopyOptions.Default, transaction)
  {
    BatchSize = 1000,
    DestinationTableName = "Table1"
  };

  // assume that the BenchMark class is working
  b = new BenchMark("Table1");
  bulkCopy.WriteToServer(_insertTable1s.AsDataReader()); // _insertTables1s is a List<Table1>
  b.Complete();
  LogHelper.WriteLogItem(b);

  b = new BenchMark("Table2");
  bulkCopy.DestinationTableName = "Table2";
  bulkCopy.WriteToServer(_insertTable2s.AsDataReader()); // _insertTables2s is a List<Table2>
  b.Complete();
  LogHelper.WriteLogItem(b);

  // etc... this code does a batch insert into about 7 tables all having about 40,000 records being inserted.

  b = new BenchMark("Transaction Commit");
  transaction.Commit();
  b.Complete();
}
catch (Exception e)
{
  transaction.Rollback();

  LogHelper.WriteLogItem(
    LogLevel.Critical,
    LogType.DataProcessing,
    e.ToString());
}
finally
{
  con.Close();
}

The Problem:

On my local development environment, everything is fine. Its when I run this operation in the cloud that causes it to hang. Using the LogHelper.WriteLogItem method, I can watch the progress of this process. I observe it hang randomly on a particular table. No exception is thrown so the transaction isn't rolled back. Say it hangs on Table2 bulk insert. Using MS SQL Management Studio, I run queries on Table3, Table2 and Table1 with no issue (this means that the transaction was aborted?)

Since it hangs, I'll go rerun the process. This time it hangs sooner so I might get logs like this:

7755   Benchmark   LoadXML took 00:00:04.2432816
7756   Benchmark   Table1 took 00:00:06.3961230
7757   Benchmark   Table2 took 00:00:05.2566890
7758   Benchmark   Table3 took 00:00:08.4900921
7759   Benchmark   Table4 took 00:00:02.0000123

... it hangs on Table5 (because the BenchMark never completed). I go to run it again and the rest of the log looks like:

7780   Benchmark   LoadXML took 00:00:04.1203923

... and it hangs here now.

I'm using rackspace cloud hosting if that helps. I have been able to fix this in the past by deleting all the tables from my dbml file and readding them but this time its not working. I'm wondering if the amount of data being processed is causing the problem?

EDIT: The code in this example is run in an Asynchronous thread. I've found out that the Thread is Aborting for an unknown reason and I need to find out why to solve this problem.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

奢望 2024-11-10 11:50:18

如果您对服务器或数据库有管理员权限,则可以运行

SELECT * FROM sys.dm_tran_session_transactions

以查看当前活动的事务 - 来自 Pinal

此外,您可以运行 sp_lock 确保没有任何东西阻止您的事务。

If you have admin to your server or database, you can run

SELECT * FROM sys.dm_tran_session_transactions

to see what transactions are currently active - From Pinal

Additionally, you can run sp_lock to make sure there isn't something blocking your transaction.

深陷 2024-11-10 11:50:18

因为这个过程是异步完成的(即启动一个线程来处理这个问题),所以线程有一个问题导致它中止,这就是为什么我会得到奇怪的行为,代码在不同的地方停止。我通过同步完成此任务解决了这个问题(它有效,但并不理想)。

我想真正的问题是为什么我的线程中止,因为我没有在任何代码中中止它。我相信这是由于正在处理的数据量所致,但我可能是错的。

不管怎样,我已经解决了我的问题。

Because this process is done asynchronously (i.e. a thread is kicked off to handle this) the thread has a problem which aborts it and that is why I get strange behavior where the code stalls at different places. I've solved this by completing this task synchronously (it works but its not ideal).

I guess the real issue is why my thread is aborting since I'm not aborting it in any of my code. I believe that its due to amount of data that is being processed, but I could be wrong.

Either way, I've solved my problem.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文