枚举链接和重置
我正在尝试将文件导入数据库，并在此过程中学习更有效的处理方法。 本文建议链式枚举可以降低内存使用量并提高性能。 这是我第一次链接多个枚举，我不太确定如何正确处理重置......
短篇故事： 我有一个从文本文件读取并映射到 DTO（请参阅映射函数）的枚举，一个Where枚举器，后跟一个采用枚举的Import...这一切都运行良好，除了当过滤器返回0条记录时...在这种情况下，SQL 错误显示 System.ArgumentException: There are norecords in the SqlDataRecord enumeration....
所以我放了一个 if(!list.Any()) 返回;在我的导入方法的顶部，它似乎工作没有错误。除了它总是会跳过文本文件中第一个有效行之前的所有行...
我是否需要在 Any() 调用后以某种方式 Reset() 枚举器？当在 Linq 和其他 Enumerable 实现中使用相同的结构时，为什么没有必要这样做？
代码：
public IEnumerable<DTO> Map(TextReader sr)
{
while (null != (line = sr.ReadLine()))
{
var dto = new DTO();
var row = line.Split('\t');
// ... mapping logic goes here ...
yield return (T)obj;
}
}
//Filter, called elsewhere
list.Where(x => Valid(x)).Select(x => LoadSqlRecord(x))
public void Import(IEnumerable<SqlDataRecord> list)
{
using (var conn = new SqlConnection(...))
{
if (conn.State != ConnectionState.Open)
conn.Open();
var cmd = conn.CreateCommand();
cmd.CommandText = "Import_Data";
cmd.CommandType = System.Data.CommandType.StoredProcedure;
var parm = new SqlParameter();
cmd.Parameters.Add(parm);
parm.ParameterName = "Data"
parm.TypeName = "udt_DTO";
parm.SqlDbType = SqlDbType.Structured;
parm.Value = list;
cmd.ExecuteNonQuery();
conn.Close();
}
}
抱歉示例太长。感谢您的阅读...
I'm trying to import a file into a database and learn a more efficient way of doing things along the way. This article suggested chaining enumerations yields low memory usage and good performance.
This is my first time chaining several enumerations, and I'm not quite sure how to handle a reset appropriately...
Short story:
I have an enumeration which reads from a text file and maps to a DTO (see the Map Function), a Where enumerator, followed by an Import that takes an enumeration... It all works perfectly, except that when the filter returns 0 records... In that case, SQL errors saying System.ArgumentException: There are no records in the SqlDataRecord enumeration....
So I put a if(!list.Any()) return; at the top of my Import method, and it seems to work not error. Except it will always skip all the rows up to (and including) the first Valid row in the text file...
Do I need to somehow Reset() the enumerator after my Any() call? Why is this not necessary when the same struct is used in Linq and other Enumerable implementations?
Code:
public IEnumerable<DTO> Map(TextReader sr)
{
while (null != (line = sr.ReadLine()))
{
var dto = new DTO();
var row = line.Split('\t');
// ... mapping logic goes here ...
yield return (T)obj;
}
}
//Filter, called elsewhere
list.Where(x => Valid(x)).Select(x => LoadSqlRecord(x))
public void Import(IEnumerable<SqlDataRecord> list)
{
using (var conn = new SqlConnection(...))
{
if (conn.State != ConnectionState.Open)
conn.Open();
var cmd = conn.CreateCommand();
cmd.CommandText = "Import_Data";
cmd.CommandType = System.Data.CommandType.StoredProcedure;
var parm = new SqlParameter();
cmd.Parameters.Add(parm);
parm.ParameterName = "Data"
parm.TypeName = "udt_DTO";
parm.SqlDbType = SqlDbType.Structured;
parm.Value = list;
cmd.ExecuteNonQuery();
conn.Close();
}
}
Sorry for the long example. Thanks for reading...
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
您看到的问题可能不是由
IEnumerable/IEnumerator接口引起的，而是由您用来生成值的底层资源引起的。对于大多数枚举，添加
list.Any()不会导致list的未来枚举跳过项目，因为每次调用list.GetEnumerator都会返回独立的对象。您可能有其他原因不想创建多个枚举器，例如通过 LINQ to SQL 多次调用数据库，但每个枚举器都会获取所有项目。然而，在这种情况下，由于底层资源的原因，创建多个枚举器不起作用。根据您发布的代码，我假设传递给
Import的参数基于对您发布的Map方法的调用。每次通过从 Map 返回的枚举时，您都将从方法的顶部“开始”，但 TextReader 及其当前位置在所有枚举器之间共享。即使您尝试在 IEnumerator 上调用 Reset，也不会重置 TextReader。要解决您的问题，您要么需要缓冲可枚举（例如 ToList），要么找到重置 TextReader 的方法。The issue you are seeing is likely not because of the
IEnumerable/IEnumeratorinterfaces, but rather with the underlying resources you are using to produce values.For most enumerables, adding a
list.Any()would not cause future enumerations oflistto skip items because each call tolist.GetEnumeratorreturns independent objects. You may have other reasons to not want to make multiple enumerators, such as multiple calls to the database via LINQ to SQL, but it every enumerator will get all the items.In this case however, making multiple enumerators is not working because of the underlying resources. Based on the code you posted, I assume that the parameter passed to
Importis based on a call to theMapmethod you posted. Each time through the enumerable returned from Map, you will "start" at the top of the method, but the TextReader and its current position is shared between all enumerators. Even if you did try to call Reset on the IEnumerators, this would not reset the TextReader. To solve your problem, you either need buffer the enumerable (eg ToList) or find a way to reset the TextReader.