如何解决 nHibernate 集合初始化不佳的问题
nHibernate3;从 EAV 数据模式中检索 4xxx 记录。当 nHibernate 或 .NET 第一次初始化这些集合时,我们会看到严重的惩罚。后续调用的执行效率似乎更高。在 SQL Server Management Studio 中运行相同的查询会产生预期的快速返回时间。
使用 Fluent 和运行时映射而不是 .hbm.xml;好奇序列化映射是否有帮助?
nHibernate Profiler 和 log4net 日志记录似乎没有给我太多的帮助。在此过程中,总共约有 140,000 个实体被水合。
附上我的 dotTrace 性能跟踪的屏幕截图,显示集合初始化损失:
已尝试过 join 和 eager fetchtypes,没有明显的结果,但不能 100% 确定我实现了那些正确 - 是否只需要指定父表,或者子表也需要标记?
var products = ((HandleSession)_handleSession).Session.CreateCriteria(typeof(Product))
.SetFetchMode("Product", FetchMode.Eager)
.List<Product>()
.AsEnumerable();
通过 web.config 启用反射优化器(我认为):
这是花费最多时间的地方:
return new ProductList(products.Select(p => p.ToProductContract()));
这只是执行此操作的扩展方法:
public static ProductContract ToProductContract(this Product product)
{
return new ProductContract
{
Name = product.ProductName,
ProductTypeName = product.ProductType.ProductTypeName,
UpdateTimeStamp = product.UpdateDateTime,
ProductNumber = product.ProductNumber,
Attributes = product.ProductAttributes.ToCommonAttribute().ToList(),
GroupCategories = product.ProductGroups.ToGroupCategory().ToList(),
PublicUniqueId = product.PublicUniqueId
};
}
映射:
internal class ProductMapping : ClassMap<Product>
{
private const string _iscurrentindicator = "IsCurrentIndicator=1";
public ProductMapping()
{
Table("Product");
Id(Reveal.Member<Product>("ProductId")).GeneratedBy.Identity().Column("ProductID");
Map(x => x.ProductNumber).Column("ProductNumber").Not.Nullable();
Map(x => x.ProductName).Column("ProductName").Not.Nullable();
Map(x => x.InsertDateTime).Column("InsertedDateTime").Nullable().ReadOnly();
Map(x => x.UpdateDateTime).Column("UpdatedDateTime").Nullable();
Map(x => x.PublicUniqueId).Column("ProductGUID").Generated.Insert();
References(x => x.ProductType).Column("ProductTypeId").Not.Nullable();
HasMany(x => x.ProductAttributes)
.KeyColumn("ProductId")
.Inverse()
.Fetch
.Subselect()
.Where(_iscurrentindicator)
.Cascade
.SaveUpdate();
HasMany(x => x.ProductGroups).KeyColumn("ProductId").Fetch.Subselect().Where(_iscurrentindicator);
DynamicUpdate();
DynamicInsert();
BatchSize(500);
}
}
internal class ProductGroupMapping : ClassMap<ProductGroup>
{
public ProductGroupMapping()
{
Table("ProductGroup");
Id(x => x.ProductGroupId).Column("ProductGroupId").GeneratedBy.Identity();
References(x => x.Product).Column("ProductId").Not.Nullable();
References(x => x.Group).Column("GroupId").Not.Nullable();
//Where("IsCurrentIndicator=1");
}
}
internal class ProductAttributeMapping : ClassMap<ProductAttribute>
{
public ProductAttributeMapping()
{
Table("ProductAttribute");
LazyLoad();
Id(x => x.ProductAttributeId).GeneratedBy.Identity().Column("ProductAttributeID");
References(x => x.Product).Column("ProductID").Not.Nullable();
References(x => x.Attribute).Column("AttributeID").Not.Nullable().Fetch.Join();
Map(x => x.PositionNumber).Column("PositionNumber").Nullable();
Map(x => x.ValueText).Column("ValueText").Nullable();
Map(x => x.ValueBinary).Column("ValueBinary").Nullable();
Component(x => x.OperationalAuditHistory, m =>
{
Table("ProductAttribute");
m.Map(x => x.ExpirationDateTime).Column("ExpirationDateTime").Nullable();
m.Map(x => x.IsCurrent).Column("IsCurrentIndicator").Not.Nullable();
m.Map(x => x.OperationCode).Column("OperationCode").Nullable();
m.Map(x => x.OperationDateTime).Column("OperationDateTime").Nullable();
m.Map(x => x.OperationSystemName).Column("OperationSystemName").Nullable();
m.Map(x => x.OperationUserName).Column("OperationUserName").Nullable();
m.Map(x => x.LastUserPriority).Column("LastUserPriority").Nullable();
});
DynamicInsert();
BatchSize(50);
}
}
不幸的是。未来我似乎仍然会得到类似的结果。这是一条新的踪迹;暂时我已经切换到Release,关键项目使用x64,所以时间比较短,但是比例还是差不多的;以及 .Eager:
var products = ((HandleSession) _handleSession).Session.CreateCriteria(typeof (Product))
.SetFetchMode("ProductAttribute", FetchMode.Join)
.SetFetchMode("ProductGroup", FetchMode.Join)
.SetFetchMode("ProductType", FetchMode.Join)
.Future<Product>()
.AsEnumerable();
使用 .Eager 生成 SQL 和.未来到位:
选择 this_.ProductID 作为 ProductID0_1_, this_.ProductNumber 为 ProductN2_0_1_, this_.ProductName 为 产品N3_0_1_,this_.InsertedDateTime 作为插入4_0_1_, this_.UpdatedDateTime 为 更新D5_0_1_,this_.ProductGUID为 ProductG6_0_1_, this_.ProductTypeId 为 产品T7_0_1_, ProductType2_.ProductTypeID 为 产品T1_6_0_, 产品类型2_.产品类型名称为 产品T2_6_0_来自产品this_ 内连接 ProductType ProductType2_ on this_.ProductTypeId=producttyp2_.ProductTypeID;
选择productatt0_.ProductId为 产品ID2_, 产品att0_.ProductAttributeID为 产品A1_2_, 产品att0_.ProductAttributeID为 产品A1_2_1_, 产品att0_.PositionNumber为 Position2_2_1_,productatt0_.ValueText 作为 ValueText2_1_, 产品att0_.ValueBinary为 ValueBin4_2_1_,productatt0_.ProductID 作为产品ID2_1_, Productatt0_.AttributeID 为 属性6_2_1_, Productatt0_.ExpirationDateTime 为 到期7_2_1_, Productatt0_.IsCurrentIndicator 为 是Curren8_2_1_, 产品att0_.操作代码为 操作9_2_1_, 产品att0_.OperationDateTime 为 操作10_2_1_, Productatt0_.OperationSystemName 为 操作11_2_1_, 产品att0_.OperationUserName 为 操作12_2_1_, 产品att0_.LastUserPriority为 最后使用13_2_1_, attribute1_.AttributeId 为 属性1_1_0_, attribute1_.AttributeName 为 属性2_1_0_, attribute1_.DisplayName 为 显示N3_1_0_, attribute1_.DataTypeName 为 数据类型4_1_0_, attribute1_.ConstraintText 为 约束5_1_0_, attribute1_.ConstraintMin 为 约束6_1_0_, attribute1_.ConstraintMax 为 Constrai7_1_0_,attribute1_.ValuesMin 作为 ValuesMin1_0_, attribute1_.ValuesMax 为 ValuesMax1_0_,attribute1_.Precision 作为 Precision1_0_ FROM ProductAttribute Productatt0_内连接属性 属性1_开 产品att0_.AttributeID=attribute1_.AttributeId 在哪里 (productatt0_.IsCurrentIndicator=1) 和productatt0_.ProductId(选择 this_.ProductID 来自产品 this_ 内连接 ProductType ProductType2_ on this_.ProductTypeId=producttyp2_.ProductTypeID)
选择productgro0_.ProductId为 产品ID1_, Productgro0_.ProductGroupId 为 产品G1_1_, Productgro0_.ProductGroupId 为 产品G1_3_0_、产品gro0_.产品Id 作为 ProductId3_0_、productgro0_.GroupId 作为 GroupId3_0_ FROM ProductGroup 产品gro0_ 哪里 (productgro0_.IsCurrentIndicator=1) 和productgro0_.ProductId(选择 this_.ProductID 来自产品 this_ 内连接 ProductType ProductType2_ on this_.ProductTypeId=producttyp2_.ProductTypeID)
nHibernate3; retrieving 4xxx records out of an EAV data schema. When nHibernate, or .NET, goes to initialize those collections for the first time, we're seeing a severe penalty. Subsequent calls appear to perform more efficiently. Running the same queries in SQL Server Management Studio result in expected quick return times.
Using Fluent and runtime mapping instead of .hbm.xml; curious if serialized mapping would help here?
nHibernate Profiler and log4net logging didn't seem to give me much to go on. A total of something like 140,000 entities are hydrated in this process.
Attached a screenshot of my dotTrace performance tracing that shows the collection initialization penalty:
Have tried join and eager fetchtypes, with no apparent results, but am not 100% certain I implemented those correctly -- does just the parent need to be so designated, or do the children tables also need to be flagged?
var products = ((HandleSession)_handleSession).Session.CreateCriteria(typeof(Product))
.SetFetchMode("Product", FetchMode.Eager)
.List<Product>()
.AsEnumerable();
With reflection optimizer enabled (I think) via web.config:
This is where most time is spent:
return new ProductList(products.Select(p => p.ToProductContract()));
Which is simply an extension method doing this:
public static ProductContract ToProductContract(this Product product)
{
return new ProductContract
{
Name = product.ProductName,
ProductTypeName = product.ProductType.ProductTypeName,
UpdateTimeStamp = product.UpdateDateTime,
ProductNumber = product.ProductNumber,
Attributes = product.ProductAttributes.ToCommonAttribute().ToList(),
GroupCategories = product.ProductGroups.ToGroupCategory().ToList(),
PublicUniqueId = product.PublicUniqueId
};
}
mappings:
internal class ProductMapping : ClassMap<Product>
{
private const string _iscurrentindicator = "IsCurrentIndicator=1";
public ProductMapping()
{
Table("Product");
Id(Reveal.Member<Product>("ProductId")).GeneratedBy.Identity().Column("ProductID");
Map(x => x.ProductNumber).Column("ProductNumber").Not.Nullable();
Map(x => x.ProductName).Column("ProductName").Not.Nullable();
Map(x => x.InsertDateTime).Column("InsertedDateTime").Nullable().ReadOnly();
Map(x => x.UpdateDateTime).Column("UpdatedDateTime").Nullable();
Map(x => x.PublicUniqueId).Column("ProductGUID").Generated.Insert();
References(x => x.ProductType).Column("ProductTypeId").Not.Nullable();
HasMany(x => x.ProductAttributes)
.KeyColumn("ProductId")
.Inverse()
.Fetch
.Subselect()
.Where(_iscurrentindicator)
.Cascade
.SaveUpdate();
HasMany(x => x.ProductGroups).KeyColumn("ProductId").Fetch.Subselect().Where(_iscurrentindicator);
DynamicUpdate();
DynamicInsert();
BatchSize(500);
}
}
internal class ProductGroupMapping : ClassMap<ProductGroup>
{
public ProductGroupMapping()
{
Table("ProductGroup");
Id(x => x.ProductGroupId).Column("ProductGroupId").GeneratedBy.Identity();
References(x => x.Product).Column("ProductId").Not.Nullable();
References(x => x.Group).Column("GroupId").Not.Nullable();
//Where("IsCurrentIndicator=1");
}
}
internal class ProductAttributeMapping : ClassMap<ProductAttribute>
{
public ProductAttributeMapping()
{
Table("ProductAttribute");
LazyLoad();
Id(x => x.ProductAttributeId).GeneratedBy.Identity().Column("ProductAttributeID");
References(x => x.Product).Column("ProductID").Not.Nullable();
References(x => x.Attribute).Column("AttributeID").Not.Nullable().Fetch.Join();
Map(x => x.PositionNumber).Column("PositionNumber").Nullable();
Map(x => x.ValueText).Column("ValueText").Nullable();
Map(x => x.ValueBinary).Column("ValueBinary").Nullable();
Component(x => x.OperationalAuditHistory, m =>
{
Table("ProductAttribute");
m.Map(x => x.ExpirationDateTime).Column("ExpirationDateTime").Nullable();
m.Map(x => x.IsCurrent).Column("IsCurrentIndicator").Not.Nullable();
m.Map(x => x.OperationCode).Column("OperationCode").Nullable();
m.Map(x => x.OperationDateTime).Column("OperationDateTime").Nullable();
m.Map(x => x.OperationSystemName).Column("OperationSystemName").Nullable();
m.Map(x => x.OperationUserName).Column("OperationUserName").Nullable();
m.Map(x => x.LastUserPriority).Column("LastUserPriority").Nullable();
});
DynamicInsert();
BatchSize(50);
}
}
Unfortunately with .Future I still appear to get similar results. Here's a new trace; I've switched to Release, and x64 for the key projects, for the moment, so the times are lower, but the proportions are still pretty much the same; as well as with .Eager:
var products = ((HandleSession) _handleSession).Session.CreateCriteria(typeof (Product))
.SetFetchMode("ProductAttribute", FetchMode.Join)
.SetFetchMode("ProductGroup", FetchMode.Join)
.SetFetchMode("ProductType", FetchMode.Join)
.Future<Product>()
.AsEnumerable();
Generated SQL with .Eager and .Future in place:
SELECT this_.ProductID as
ProductID0_1_, this_.ProductNumber as
ProductN2_0_1_, this_.ProductName as
ProductN3_0_1_, this_.InsertedDateTime
as Inserted4_0_1_,
this_.UpdatedDateTime as
UpdatedD5_0_1_, this_.ProductGUID as
ProductG6_0_1_, this_.ProductTypeId as
ProductT7_0_1_,
producttyp2_.ProductTypeID as
ProductT1_6_0_,
producttyp2_.ProductTypeName as
ProductT2_6_0_ FROM Product this_
inner join ProductType producttyp2_ on
this_.ProductTypeId=producttyp2_.ProductTypeID;SELECT productatt0_.ProductId as
ProductId2_,
productatt0_.ProductAttributeID as
ProductA1_2_,
productatt0_.ProductAttributeID as
ProductA1_2_1_,
productatt0_.PositionNumber as
Position2_2_1_, productatt0_.ValueText
as ValueText2_1_,
productatt0_.ValueBinary as
ValueBin4_2_1_, productatt0_.ProductID
as ProductID2_1_,
productatt0_.AttributeID as
Attribut6_2_1_,
productatt0_.ExpirationDateTime as
Expirati7_2_1_,
productatt0_.IsCurrentIndicator as
IsCurren8_2_1_,
productatt0_.OperationCode as
Operatio9_2_1_,
productatt0_.OperationDateTime as
Operati10_2_1_,
productatt0_.OperationSystemName as
Operati11_2_1_,
productatt0_.OperationUserName as
Operati12_2_1_,
productatt0_.LastUserPriority as
LastUse13_2_1_,
attribute1_.AttributeId as
Attribut1_1_0_,
attribute1_.AttributeName as
Attribut2_1_0_,
attribute1_.DisplayName as
DisplayN3_1_0_,
attribute1_.DataTypeName as
DataType4_1_0_,
attribute1_.ConstraintText as
Constrai5_1_0_,
attribute1_.ConstraintMin as
Constrai6_1_0_,
attribute1_.ConstraintMax as
Constrai7_1_0_, attribute1_.ValuesMin
as ValuesMin1_0_,
attribute1_.ValuesMax as
ValuesMax1_0_, attribute1_.Precision
as Precision1_0_ FROM ProductAttribute
productatt0_ inner join Attribute
attribute1_ on
productatt0_.AttributeID=attribute1_.AttributeId
WHERE
(productatt0_.IsCurrentIndicator=1)
and productatt0_.ProductId in (select
this_.ProductID FROM Product this_
inner join ProductType producttyp2_ on
this_.ProductTypeId=producttyp2_.ProductTypeID)SELECT productgro0_.ProductId as
ProductId1_,
productgro0_.ProductGroupId as
ProductG1_1_,
productgro0_.ProductGroupId as
ProductG1_3_0_, productgro0_.ProductId
as ProductId3_0_, productgro0_.GroupId
as GroupId3_0_ FROM ProductGroup
productgro0_ WHERE
(productgro0_.IsCurrentIndicator=1)
and productgro0_.ProductId in (select
this_.ProductID FROM Product this_
inner join ProductType producttyp2_ on
this_.ProductTypeId=producttyp2_.ProductTypeID)
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(3)
1)序列化映射只会有助于减少构建SessionFactory所需的时间。如果上面的查询不是第一次访问数据库,那么它不会完成这方面的任何事情。
2) Set FetchMode 需要应用于子级,如下所示:
3) 如果我正确解释屏幕截图中的方法,这看起来像是一个 N+1 问题。您是否将查询结果中的
Products
转换为 ProductDTO 列表?如果是这样,子集合似乎是在循环内从数据库延迟加载的。编辑:
为了对抗 N+1 Select,我们必须告诉 NHibernate 预先加载所有内容,最好是使用 Futures。这是一个潜在的解决方案,基本上可以使用一些 Select 语句从数据库中获取所有数据。我没有包含任何Where 条件。您必须相应地添加那些内容。
1) A serialized mapping will only help to reduce the time required to build the SessionFactory. If the above query is not the first access to the database, it will not accomplish anything in that regard.
2) Set FetchMode needs to be applied to the children, like this:
3) This looks like a N+1 problem, if I interpret the methods in the Screenshots correctly. Are you transforming the
Products
in your query result to a list of ProductDTOs? If so, it seems as if the child collections are lazy loaded from the DB within a loop.Edit:
In order to combat the N+1 Select, we will have to tell NHibernate to load everything beforehand, preferably with Futures. Here is a potential solution that basically fetches all your data from the db with a handful of Select-statements. I did not include any Where-conditions. Those you would have to add accordingly.
一种选择是在集合上启用批量大小。我假设这些是惰性的,并且启用了批量大小,它会尝试在单次往返中获取多个实体的集合。
如果您使用一个集合获取 1 个实体,这不会产生什么影响,但如果您选择 1000 个实体,它们都具有一个集合,则可能会产生巨大的差异。使用 1000 的批量大小将导致 2 个查询而不是 1001 个。
尝试查找一些文档,但只找到了这个示例:
nhibernate 替代批量大小
在您的情况下使用连接策略会导致巨大的结果集,因此这不是一个好的选择。更好的选择是使用 FetchMode.Select ,它会显式强制您的集合在后续往返中加载。
另一件可以提高性能的事情是设置:
禁用示波器的自动刷新。如果您实际上所做的只是读取数据而不是修改数据,那么这非常有用。但是,您会看到对 IsDirty 的调用或调用堆栈中对脏对象的任何其他检查。
One option is to enable batch-size on your collections. I assume those are lazy, and with batch size enabled, it would try to fetch collections for multiple entities in a single roundtrip.
It doesn't make a difference if you fetch 1 entity with one collection, but can make a huge difference if you select 1000 entities which all has one collection. Using a batch-size of 1000 would result in 2 queries instead of 1001.
Tried to find some documentation, but only found this example:
nhibernate alternates batch size
Using join strategies in your case would result in gigantic resultsets so that is not a good option. A better option would be to use FetchMode.Select which would explicitly force your collections to be loaded in a subsequent roundtrip.
Another thing that could improve performance is setting:
Which disable automatic flushing of your scope. This is useful if all you actually do is reading data, not modifying it. However, you would see calls to IsDirty or any other check for dirty objects in your callstack.
如果您仅使用此会话进行报告,则必须使用无状态会话:
http://nhforge.org/blogs/nhibernate/archive/2008/10/30/bulk-data-operations-with-nhibernate-s-stateless-sessions.aspx
If you are using this session only for reporting, you have to use Stateless Sessions:
http://nhforge.org/blogs/nhibernate/archive/2008/10/30/bulk-data-operations-with-nhibernate-s-stateless-sessions.aspx