检测到线程饥饿或时钟跳跃(管家 delta=*):Spring Batch

发布于 2025-01-16 11:12:58 字数 5684 浏览 3 评论 0原文

我已经面临这个问题有一段时间了。让我尝试解释一下问题到底是什么。 当我使用 Drools 运行带有大量文件的 Spring 批处理应用程序时,经过 30-40 分钟的处理后,我面临以下问题。我拥有的数据库是

oracle11g

。请找到下面的日志:

2022-03-23 18:24:39.242 ERROR 21708 --- [           main] o.s.batch.core.step.AbstractStep         : Encountered an error executing step readProvisions in job processProvisions

java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.Hashtable.addEntry(Hashtable.java:435) ~[na:1.8.0_92]
at java.util.Hashtable.put(Hashtable.java:476) ~[na:1.8.0_92]
at java.util.Properties.setProperty(Properties.java:166) ~[na:1.8.0_92]
at org.springframework.batch.item.file.transform.DefaultFieldSet.getProperties(DefaultFieldSet.java:748) ~[spring-batch-infrastructure-4.3.1.jar:4.3.1]

当我添加以下参数时

-XX:+HeapDumpOnOutOfMemoryError

在运行配置中获取内存转储时出现以下错误:

2022-03-22 00:26:15.640 DEBUG 25460 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Add connection elided, waiting 0, queue 1
2022-03-22 00:27:15.997 DEBUG 25460 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Add connection elided, waiting 0, queue 3
2022-03-22 00:27:46.013 DEBUG 25460 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Add connection elided, waiting 0, queue 4
2022-03-22 00:28:15.993 DEBUG 25460 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Add connection elided, waiting 0, queue 5
2022-03-22 00:36:35.396 DEBUG 25460 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Fill pool skipped, pool is at sufficient level.
2022-03-22 00:36:35.396 DEBUG 25460 --- [HikariPool-1 connection adder] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Cannot acquire connection from data source

java.sql.SQLRecoverableException: IO Error: Connection reset by peer: socket write error
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:682) ~[ojdbc7-12.1.0.1.0.jar:12.1.0.1.0]
at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:715) ~[ojdbc7-12.1.0.1.0.jar:12.1.0.1.0]
at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:385) ~[ojdbc7-12.1.0.1.0.jar:12.1.0.1.0]
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:30) ~[ojdbc7-12.1.0.1.0.jar:12.1.0.1.0]

现在,当我评论 Drools 功能时,我能够在单次运行中快速处理 70 个文件,并且没有任何错误。我在 POM.xml 中看到以下 drools 依赖项:

<!-- https://mvnrepository.com/artifact/org.drools/drools-core -->
    <dependency>
        <groupId>org.drools</groupId>
        <artifactId>drools-core</artifactId>
        <version>6.4.0.Final</version>
    </dependency>

    <!-- https://mvnrepository.com/artifact/org.drools/drools-compiler -->
    <dependency>
        <groupId>org.drools</groupId>
        <artifactId>drools-compiler</artifactId>
        <version>6.4.0.Final</version>
    </dependency>

    <!-- https://mvnrepository.com/artifact/org.drools/drools-spring -->
    <dependency>
        <groupId>org.drools</groupId>
        <artifactId>drools-spring</artifactId>
        <version>5.6.0.Final</version>
    </dependency>

    <!-- https://mvnrepository.com/artifact/org.drools/drools-decisiontables -->
    <dependency>
        <groupId>org.drools</groupId>
        <artifactId>drools-decisiontables</artifactId>
        <version>6.4.0.Final</version>
    </dependency>

    <!-- https://mvnrepository.com/artifact/org.drools/drools-templates -->
    <dependency>
        <groupId>org.drools</groupId>
        <artifactId>drools-templates</artifactId>
        <version>6.4.0.Final</version>
    </dependency>

    <!-- https://mvnrepository.com/artifact/org.kie/kie-api -->
    <dependency>
        <groupId>org.kie</groupId>
        <artifactId>kie-api</artifactId>
        <version>6.4.0.Final</version>
        <scope>provided</scope>
    </dependency>

    <!-- https://mvnrepository.com/artifact/org.kie/kie-internal -->
    <dependency>
        <groupId>org.kie</groupId>
        <artifactId>kie-internal</artifactId>
        <version>6.4.0.Final</version>
    </dependency>

这是配置代码:

@Bean
public KieSession getKieSession() {

    KieFileSystem kieFileSystem = kieServices.newKieFileSystem();

    kieFileSystem.write(ResourceFactory.newClassPathResource("rules/RE21DataConformRules.xls"));
    kieFileSystem.write(ResourceFactory.newClassPathResource("rules/RE21DataSystemRules.xls"));
    kieFileSystem.write(ResourceFactory.newClassPathResource("rules/CoveragesMappingRules.xls"));
    kieFileSystem.write(ResourceFactory.newClassPathResource("rules/AS400ARiskIDMappingForCDBRules.xls"));
    kieFileSystem.write(ResourceFactory.newClassPathResource("rules/HSBCoveragesTableforRE21.xls"));
    kieFileSystem.write(ResourceFactory.newClassPathResource("rules/UmbrellaILFRules.xls"));

    KieBuilder kieBuilder = kieServices.newKieBuilder(kieFileSystem);
    kieBuilder.buildAll();
    KieModule kieModule = kieBuilder.getKieModule();

    KieContainer kContainer = kieServices.newKieContainer(kieModule.getReleaseId());

    return kContainer.newKieSession();
}

任何人都可以帮助解决此问题吗?

提前致谢。

I am facing this issue for quite sometime. Let me try to explain what exactly the problem is.
When I am running Spring batch application with large no of files with Drools I face following issue after 30-40mins of processing. The database I am having is

oracle11g

. Please find the logs below:

2022-03-23 18:24:39.242 ERROR 21708 --- [           main] o.s.batch.core.step.AbstractStep         : Encountered an error executing step readProvisions in job processProvisions

java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.Hashtable.addEntry(Hashtable.java:435) ~[na:1.8.0_92]
at java.util.Hashtable.put(Hashtable.java:476) ~[na:1.8.0_92]
at java.util.Properties.setProperty(Properties.java:166) ~[na:1.8.0_92]
at org.springframework.batch.item.file.transform.DefaultFieldSet.getProperties(DefaultFieldSet.java:748) ~[spring-batch-infrastructure-4.3.1.jar:4.3.1]

When I add following parameter

-XX:+HeapDumpOnOutOfMemoryError

in the run Configuration to get the memory dump I get following error:

2022-03-22 00:26:15.640 DEBUG 25460 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Add connection elided, waiting 0, queue 1
2022-03-22 00:27:15.997 DEBUG 25460 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Add connection elided, waiting 0, queue 3
2022-03-22 00:27:46.013 DEBUG 25460 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Add connection elided, waiting 0, queue 4
2022-03-22 00:28:15.993 DEBUG 25460 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Add connection elided, waiting 0, queue 5
2022-03-22 00:36:35.396 DEBUG 25460 --- [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Fill pool skipped, pool is at sufficient level.
2022-03-22 00:36:35.396 DEBUG 25460 --- [HikariPool-1 connection adder] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Cannot acquire connection from data source

java.sql.SQLRecoverableException: IO Error: Connection reset by peer: socket write error
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:682) ~[ojdbc7-12.1.0.1.0.jar:12.1.0.1.0]
at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:715) ~[ojdbc7-12.1.0.1.0.jar:12.1.0.1.0]
at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:385) ~[ojdbc7-12.1.0.1.0.jar:12.1.0.1.0]
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:30) ~[ojdbc7-12.1.0.1.0.jar:12.1.0.1.0]

Now when I commented Drools functionality I was able to process 70 files in single run, quickly and without any error. I see following drools dependencies in POM.xml:

<!-- https://mvnrepository.com/artifact/org.drools/drools-core -->
    <dependency>
        <groupId>org.drools</groupId>
        <artifactId>drools-core</artifactId>
        <version>6.4.0.Final</version>
    </dependency>

    <!-- https://mvnrepository.com/artifact/org.drools/drools-compiler -->
    <dependency>
        <groupId>org.drools</groupId>
        <artifactId>drools-compiler</artifactId>
        <version>6.4.0.Final</version>
    </dependency>

    <!-- https://mvnrepository.com/artifact/org.drools/drools-spring -->
    <dependency>
        <groupId>org.drools</groupId>
        <artifactId>drools-spring</artifactId>
        <version>5.6.0.Final</version>
    </dependency>

    <!-- https://mvnrepository.com/artifact/org.drools/drools-decisiontables -->
    <dependency>
        <groupId>org.drools</groupId>
        <artifactId>drools-decisiontables</artifactId>
        <version>6.4.0.Final</version>
    </dependency>

    <!-- https://mvnrepository.com/artifact/org.drools/drools-templates -->
    <dependency>
        <groupId>org.drools</groupId>
        <artifactId>drools-templates</artifactId>
        <version>6.4.0.Final</version>
    </dependency>

    <!-- https://mvnrepository.com/artifact/org.kie/kie-api -->
    <dependency>
        <groupId>org.kie</groupId>
        <artifactId>kie-api</artifactId>
        <version>6.4.0.Final</version>
        <scope>provided</scope>
    </dependency>

    <!-- https://mvnrepository.com/artifact/org.kie/kie-internal -->
    <dependency>
        <groupId>org.kie</groupId>
        <artifactId>kie-internal</artifactId>
        <version>6.4.0.Final</version>
    </dependency>

And here is the Configuration code:

@Bean
public KieSession getKieSession() {

    KieFileSystem kieFileSystem = kieServices.newKieFileSystem();

    kieFileSystem.write(ResourceFactory.newClassPathResource("rules/RE21DataConformRules.xls"));
    kieFileSystem.write(ResourceFactory.newClassPathResource("rules/RE21DataSystemRules.xls"));
    kieFileSystem.write(ResourceFactory.newClassPathResource("rules/CoveragesMappingRules.xls"));
    kieFileSystem.write(ResourceFactory.newClassPathResource("rules/AS400ARiskIDMappingForCDBRules.xls"));
    kieFileSystem.write(ResourceFactory.newClassPathResource("rules/HSBCoveragesTableforRE21.xls"));
    kieFileSystem.write(ResourceFactory.newClassPathResource("rules/UmbrellaILFRules.xls"));

    KieBuilder kieBuilder = kieServices.newKieBuilder(kieFileSystem);
    kieBuilder.buildAll();
    KieModule kieModule = kieBuilder.getKieModule();

    KieContainer kContainer = kieServices.newKieContainer(kieModule.getReleaseId());

    return kContainer.newKieSession();
}

Can any one help here to resolve this issue?

thanks in advance.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。
列表为空,暂无数据
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文