OutOfMemoryError：尝试使用 HSQLDB、DBUnit、unitils、Hibernate、Spring 优化 Maven 多模块单元测试
我有一个大型多模块 Maven 项目，包含数千个测试。 每个测试都使用 SpringApplicationContext 注释加载 DAO、服务等。 我的unitils 配置如下所示：
database.driverClassName=org.hsqldb.jdbcDriver
database.url=jdbc:hsqldb:file:mytestdb
database.schemaNames=PUBLIC
database.userName=sa
database.password=
database.dialect=hsqldb
unitils.modules=database,dbunit,hibernate,inject,spring
# custom version of HsqldbDbSupport which calls "SET REFERENTIAL_INTEGRITY FALSE"
org.unitils.core.dbsupport.DbSupport.implClassName.hsqldb=my.company.unitils.HsqldbDbSupport
org.dbunit.dataset.datatype.IDataTypeFactory.implClassName.hsqldb=org.dbunit.ext.hsqldb.HsqldbDataTypeFactory
org.unitils.dbmaintainer.structure.ConstraintsDisabler.implClassName=org.unitils.dbmaintainer.structure.impl.DefaultConstraintsDisabler
# custom version of CleanInsertLoadStrategy which calls "DatabaseUnitils.disableConstraints();"
DbUnitModule.DataSet.loadStrategy.default=my.company.unitils.DataSetLoadStrategy
HibernateModule.configuration.implClassName=org.hibernate.cfg.AnnotationConfiguration
dbMaintainer.disableConstraints.enabled=true
dbMaintainer.cleanDb.enabled=true
dbMaintainer.generateDataSetStructure.enabled=false
dataSetStructureGenerator.xsd.dirName=target/xsd
dbMaintainer.autoCreateExecutedScriptsTable=true
updateDataBaseSchema.enabled=false
dbMaintainer.fromScratchEnabled=true
我的Hibernate 配置如下所示：
hibernate.hbm2ddl.auto=update
hibernate.show_sql=false
hibernate.format_sql=false
hibernate.dialect=org.hibernate.dialect.HSQLDialect
hibernate.connection.autocommit=false
hibernate.cache.provider_class=org.hibernate.cache.NoCacheProvider
hibernate.cache.use_query_cache=false
hibernate.cache.use_second_level_cache=false
hibernate.cache.use_structured_entries=false
hibernate.generate_statistics=false
当我运行测试时，分配给堆的2GB 最终会填满。 Permgen 达到约 400MB。 查看堆转储，char、java.lang.String 和 java.lang.String[] 约占总数的 40%实例。
当我抽查其中一些实例时，GC 根追溯到：
- Spring 上下文管理器（以上下文 XML 提取的形式）
- SessionFactory（以查询的形式）
我想优化这两件事，并且我认为它们分别是由以下原因引起的：
- 每个测试都有一个@SpringApplicationContext注释
- 每个测试还使用unitils@DataSet注释来加载该测试的数据。
注意，我使用 jdbc:hsqldb:file 从堆中卸载一些数据（即而不是使用 jdbc:hsqldb:mem）
版本：
hsqldb 1.8.0.7
Spring 3.0.4.RELEASE
unitils 3.1
PS 评分超过 1500 的人可以用“unitils”标记此问题吗？
I have a big multi-module Maven project with thousands of tests.
Each test loads DAOs, services, etc using the SpringApplicationContext annotation.
My unitils configuration looks like this:
database.driverClassName=org.hsqldb.jdbcDriver
database.url=jdbc:hsqldb:file:mytestdb
database.schemaNames=PUBLIC
database.userName=sa
database.password=
database.dialect=hsqldb
unitils.modules=database,dbunit,hibernate,inject,spring
# custom version of HsqldbDbSupport which calls "SET REFERENTIAL_INTEGRITY FALSE"
org.unitils.core.dbsupport.DbSupport.implClassName.hsqldb=my.company.unitils.HsqldbDbSupport
org.dbunit.dataset.datatype.IDataTypeFactory.implClassName.hsqldb=org.dbunit.ext.hsqldb.HsqldbDataTypeFactory
org.unitils.dbmaintainer.structure.ConstraintsDisabler.implClassName=org.unitils.dbmaintainer.structure.impl.DefaultConstraintsDisabler
# custom version of CleanInsertLoadStrategy which calls "DatabaseUnitils.disableConstraints();"
DbUnitModule.DataSet.loadStrategy.default=my.company.unitils.DataSetLoadStrategy
HibernateModule.configuration.implClassName=org.hibernate.cfg.AnnotationConfiguration
dbMaintainer.disableConstraints.enabled=true
dbMaintainer.cleanDb.enabled=true
dbMaintainer.generateDataSetStructure.enabled=false
dataSetStructureGenerator.xsd.dirName=target/xsd
dbMaintainer.autoCreateExecutedScriptsTable=true
updateDataBaseSchema.enabled=false
dbMaintainer.fromScratchEnabled=true
My Hibernate configuration looks like this:
hibernate.hbm2ddl.auto=update
hibernate.show_sql=false
hibernate.format_sql=false
hibernate.dialect=org.hibernate.dialect.HSQLDialect
hibernate.connection.autocommit=false
hibernate.cache.provider_class=org.hibernate.cache.NoCacheProvider
hibernate.cache.use_query_cache=false
hibernate.cache.use_second_level_cache=false
hibernate.cache.use_structured_entries=false
hibernate.generate_statistics=false
When I run the tests, the 2GB allocated for heap eventually fills up. Permgen reaches about 400MB.
Looking at the heap dump, char, java.lang.String and java.lang.String[] takes up about 40% of the total instances.
When I spot-check a few of these instances, the GC root traces back to:
- the Spring context manager (in the form of context XML extracts)
- the SessionFactory (in the form of queries)
I would like to optimise these 2 things, and I think they are respectively caused by:
- Each test has a @SpringApplicationContext annotation
- Each test also uses the unitils @DataSet annotation to load up data for that test.
Note, I use jdbc:hsqldb:file to offload some data off of the heap (i.e. instead of using jdbc:hsqldb:mem)
Versions:
hsqldb 1.8.0.7
Spring 3.0.4.RELEASE
unitils 3.1
PS can someone with a 1500+ rating please tag this question with "unitils"?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
如果内存被数据库中存储的数据占用，则可以使数据库使用基于磁盘的表来减少内存使用。可以在数据库 URL 上指定：
database.url=jdbc:hsqldb:file:mytestdb;hsqldb.default_table_type=cached
If the memory is taken by the data stored in the database, you can make the database use disk based tables to reduce memory use. This can be specified on the database URL:
database.url=jdbc:hsqldb:file:mytestdb;hsqldb.default_table_type=cached