检测到线程饥饿或时钟跳跃(管家 delta=*):Spring Batch
我已经面临这个问题有一段时间了。让我尝试解释一下问题到底是什么。 当我使用 Drools 运行带有大量文件的 Spring 批处理应用程序时,经过 30-40 分钟的处理后,我面临以下问题。我拥有的数据库是
oracle11g
。请找到下面的日志:
2022-03-23 18:24:39.242 ERROR 21708 --- [ main] o.s.batch.core.step.AbstractStep : Encountered an error executing step readProvisions in job processProvisions
java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.Hashtable.addEntry(Hashtable.java:435) ~[na:1.8.0_92]
at java.util.Hashtable.put(Hashtable.java:476) ~[na:1.8.0_92]
at java.util.Properties.setProperty(Properties.java:166) ~[na:1.8.0_92]
at org.springframework.batch.item.file.transform.DefaultFieldSet.getProperties(DefaultFieldSet.java:748) ~[spring-batch-infrastructure-4.3.1.jar:4.3.1]
当我添加以下参数时
-XX:+HeapDumpOnOutOfMemoryError
在运行配置中获取内存转储时出现以下错误:
2022-03-22 00:26:15.640 DEBUG 25460 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Add connection elided, waiting 0, queue 1
2022-03-22 00:27:15.997 DEBUG 25460 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Add connection elided, waiting 0, queue 3
2022-03-22 00:27:46.013 DEBUG 25460 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Add connection elided, waiting 0, queue 4
2022-03-22 00:28:15.993 DEBUG 25460 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Add connection elided, waiting 0, queue 5
2022-03-22 00:36:35.396 DEBUG 25460 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Fill pool skipped, pool is at sufficient level.
2022-03-22 00:36:35.396 DEBUG 25460 --- [HikariPool-1 connection adder] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Cannot acquire connection from data source
java.sql.SQLRecoverableException: IO Error: Connection reset by peer: socket write error
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:682) ~[ojdbc7-12.1.0.1.0.jar:12.1.0.1.0]
at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:715) ~[ojdbc7-12.1.0.1.0.jar:12.1.0.1.0]
at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:385) ~[ojdbc7-12.1.0.1.0.jar:12.1.0.1.0]
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:30) ~[ojdbc7-12.1.0.1.0.jar:12.1.0.1.0]
现在,当我评论 Drools 功能时,我能够在单次运行中快速处理 70 个文件,并且没有任何错误。我在 POM.xml 中看到以下 drools 依赖项:
<!-- https://mvnrepository.com/artifact/org.drools/drools-core -->
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-core</artifactId>
<version>6.4.0.Final</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.drools/drools-compiler -->
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-compiler</artifactId>
<version>6.4.0.Final</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.drools/drools-spring -->
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-spring</artifactId>
<version>5.6.0.Final</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.drools/drools-decisiontables -->
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-decisiontables</artifactId>
<version>6.4.0.Final</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.drools/drools-templates -->
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-templates</artifactId>
<version>6.4.0.Final</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.kie/kie-api -->
<dependency>
<groupId>org.kie</groupId>
<artifactId>kie-api</artifactId>
<version>6.4.0.Final</version>
<scope>provided</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/org.kie/kie-internal -->
<dependency>
<groupId>org.kie</groupId>
<artifactId>kie-internal</artifactId>
<version>6.4.0.Final</version>
</dependency>
这是配置代码:
@Bean
public KieSession getKieSession() {
KieFileSystem kieFileSystem = kieServices.newKieFileSystem();
kieFileSystem.write(ResourceFactory.newClassPathResource("rules/RE21DataConformRules.xls"));
kieFileSystem.write(ResourceFactory.newClassPathResource("rules/RE21DataSystemRules.xls"));
kieFileSystem.write(ResourceFactory.newClassPathResource("rules/CoveragesMappingRules.xls"));
kieFileSystem.write(ResourceFactory.newClassPathResource("rules/AS400ARiskIDMappingForCDBRules.xls"));
kieFileSystem.write(ResourceFactory.newClassPathResource("rules/HSBCoveragesTableforRE21.xls"));
kieFileSystem.write(ResourceFactory.newClassPathResource("rules/UmbrellaILFRules.xls"));
KieBuilder kieBuilder = kieServices.newKieBuilder(kieFileSystem);
kieBuilder.buildAll();
KieModule kieModule = kieBuilder.getKieModule();
KieContainer kContainer = kieServices.newKieContainer(kieModule.getReleaseId());
return kContainer.newKieSession();
}
任何人都可以帮助解决此问题吗?
提前致谢。
I am facing this issue for quite sometime. Let me try to explain what exactly the problem is.
When I am running Spring batch application with large no of files with Drools I face following issue after 30-40mins of processing. The database I am having is
oracle11g
. Please find the logs below:
2022-03-23 18:24:39.242 ERROR 21708 --- [ main] o.s.batch.core.step.AbstractStep : Encountered an error executing step readProvisions in job processProvisions
java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.Hashtable.addEntry(Hashtable.java:435) ~[na:1.8.0_92]
at java.util.Hashtable.put(Hashtable.java:476) ~[na:1.8.0_92]
at java.util.Properties.setProperty(Properties.java:166) ~[na:1.8.0_92]
at org.springframework.batch.item.file.transform.DefaultFieldSet.getProperties(DefaultFieldSet.java:748) ~[spring-batch-infrastructure-4.3.1.jar:4.3.1]
When I add following parameter
-XX:+HeapDumpOnOutOfMemoryError
in the run Configuration to get the memory dump I get following error:
2022-03-22 00:26:15.640 DEBUG 25460 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Add connection elided, waiting 0, queue 1
2022-03-22 00:27:15.997 DEBUG 25460 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Add connection elided, waiting 0, queue 3
2022-03-22 00:27:46.013 DEBUG 25460 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Add connection elided, waiting 0, queue 4
2022-03-22 00:28:15.993 DEBUG 25460 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Add connection elided, waiting 0, queue 5
2022-03-22 00:36:35.396 DEBUG 25460 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Fill pool skipped, pool is at sufficient level.
2022-03-22 00:36:35.396 DEBUG 25460 --- [HikariPool-1 connection adder] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Cannot acquire connection from data source
java.sql.SQLRecoverableException: IO Error: Connection reset by peer: socket write error
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:682) ~[ojdbc7-12.1.0.1.0.jar:12.1.0.1.0]
at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:715) ~[ojdbc7-12.1.0.1.0.jar:12.1.0.1.0]
at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:385) ~[ojdbc7-12.1.0.1.0.jar:12.1.0.1.0]
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:30) ~[ojdbc7-12.1.0.1.0.jar:12.1.0.1.0]
Now when I commented Drools functionality I was able to process 70 files in single run, quickly and without any error. I see following drools dependencies in POM.xml:
<!-- https://mvnrepository.com/artifact/org.drools/drools-core -->
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-core</artifactId>
<version>6.4.0.Final</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.drools/drools-compiler -->
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-compiler</artifactId>
<version>6.4.0.Final</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.drools/drools-spring -->
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-spring</artifactId>
<version>5.6.0.Final</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.drools/drools-decisiontables -->
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-decisiontables</artifactId>
<version>6.4.0.Final</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.drools/drools-templates -->
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-templates</artifactId>
<version>6.4.0.Final</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.kie/kie-api -->
<dependency>
<groupId>org.kie</groupId>
<artifactId>kie-api</artifactId>
<version>6.4.0.Final</version>
<scope>provided</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/org.kie/kie-internal -->
<dependency>
<groupId>org.kie</groupId>
<artifactId>kie-internal</artifactId>
<version>6.4.0.Final</version>
</dependency>
And here is the Configuration code:
@Bean
public KieSession getKieSession() {
KieFileSystem kieFileSystem = kieServices.newKieFileSystem();
kieFileSystem.write(ResourceFactory.newClassPathResource("rules/RE21DataConformRules.xls"));
kieFileSystem.write(ResourceFactory.newClassPathResource("rules/RE21DataSystemRules.xls"));
kieFileSystem.write(ResourceFactory.newClassPathResource("rules/CoveragesMappingRules.xls"));
kieFileSystem.write(ResourceFactory.newClassPathResource("rules/AS400ARiskIDMappingForCDBRules.xls"));
kieFileSystem.write(ResourceFactory.newClassPathResource("rules/HSBCoveragesTableforRE21.xls"));
kieFileSystem.write(ResourceFactory.newClassPathResource("rules/UmbrellaILFRules.xls"));
KieBuilder kieBuilder = kieServices.newKieBuilder(kieFileSystem);
kieBuilder.buildAll();
KieModule kieModule = kieBuilder.getKieModule();
KieContainer kContainer = kieServices.newKieContainer(kieModule.getReleaseId());
return kContainer.newKieSession();
}
Can any one help here to resolve this issue?
thanks in advance.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论