LINQ Skip() 问题
下面的 C# 语句会阻塞进程并且无法检索数据,如果 itemToSkip 大于 0。
int itemToSkip = 100;
int itemToTake = 1000;
var itemList = db.MYTABLEs.Skip(itemToSkip).Take(itemToTake).ToList();
我该如何修复它?问题是什么?
The C# statement below blocks the process and cannot retrieve data if
itemToSkip is greater than 0.
int itemToSkip = 100;
int itemToTake = 1000;
var itemList = db.MYTABLEs.Skip(itemToSkip).Take(itemToTake).ToList();
How can I fix it? what is the problem?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(4)
不确定您有哪个提供者提供 db.MYTABLEs。除非我们知道 db.MYTABLEs 的行为方式,否则确实无法回答您的问题。
在正常的 LINQ 中,skip 不仅仅是向前跳过;它必须迭代数据量才能跳过。因此,对于 14GB 的数据表,它将迭代第一个“跳过”的记录数。如果此迭代很慢,则跳过不会节省任何 cpu/时间。
对于某些提供程序(例如 SQL 源),可以使用游标来实现跳过,这又会很慢。如果是SQL Server,可能用一个关键字优化,可能会更快。
如果是 LINQ-to-SQL,它会使用“NOT EXISTS”子句将查询转换为 SQL,这将极其慢,因为它必须遍历整个如果 NOT EXISTS 子句未命中索引。请参阅以下内容(链接):
换句话说,文档说“不要这样做”。
仅对于具有随机访问功能的提供程序(例如内存中的数组),跳过速度会非常快,因为提供程序可以直接跳转。
最糟糕的情况是,如果您运行的提供程序在您使用 Skip/Take 时会自动对整个数据集进行排序。如果你有 14GB 的数据,那么这种排序会非常慢。
您需要进行更多实验,看看您的程序是否挂在跳过状态,或者只是占用了所有 cpu 来尝试迭代。
如果您只是尝试将数据划分为可管理的块,那么您可能不应该使用skip/take,因为它每次都会重新查询数据源。
Not sure what provider you have that provides db.MYTABLEs. It is really not possible to answer your question unless we know how db.MYTABLEs behaves.
In normal LINQ, skip does not just skip ahead; it has to iterate through the amount of data in order to skip. Therefore, for your 14gb data table, it will be iterating through the first "skip" number of records. If this iteration is slow, you are not saving any cpu/time by skipping.
For some providers, e.g. an SQL source, skip may be implemented using cursors, which can again be slow. If it is SQL Server, it may be optimized with a keyword which may be faster.
If it is LINQ-to-SQL, it translates the query into SQL using a "NOT EXISTS" clause, which will be extremely slow because it has to go through the entire table if the NOT EXISTS clause does not hit an index. See the following (link):
In other words, the docs says "don't do it."
Only for providers with random-access features, e.g. an in-memory array, will skip be really fast because the provider can just jump ahead.
The worst case will be if you are running on a provider that automatically sorts the entire data set if you use Skip/Take. If you have 14gb of data, then this sort is going to be really slow.
You need to experiment some more to see if your program is hanging on skip, or just hogging all the cpu trying to iterate through.
If you are only trying to divide your data into manageable chunks, you probably should not be using skip/take, which requeries the data source every time.
Skip 通常坚持有明确的排序顺序。尝试
或类似。
Skip usually insists on having an explicit sort ordering. Try
or similar.
我假设您的 DBMS 不支持直接跳过,因此它可能会请求所有数据,具体取决于提供商。
I assume, that your DBMS isn't supporting skip directly and therefore it might request all data, depending on the provider.
对于这么大的表,可能需要很长时间才能返回,并且由于缺乏索引而需要很长时间,您需要进行一些分析才能查看底层查询,仅通过查看无法诊断问题LINQ 代码
with such large table is possible that is just taking a long time to return, and it's taking that long due to lack of indexes, you need to do some profiling to see the underlying query, you can't diagnose the problem just by looking at the LINQ code