ignite查询出现内存溢出

Question

@李玉珏    您好！！！

今天查询ignite出现了一个异常，内存溢出，我们的ignite 堆内内存已经设置为20g，堆外也是20g，数据大概是160万条，占用50%左右的堆内存， 麻烦帮忙看看，谢谢。

ORDER BY 5, 4 LIMIT 1 [90108-195]

        at org.h2.message.DbException.getJdbcSQLException(DbException.java:345)

        at org.h2.message.DbException.get(DbException.java:168)

        at org.h2.message.DbException.convert(DbException.java:289)

        at org.h2.command.Command.executeQuery(Command.java:213)

        at org.h2.jdbc.JdbcPreparedStatement.executeQuery(JdbcPreparedStatement.java:111)

        at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeSqlQuery(IgniteH2Indexing.java:1139)

        ... 26 more

Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded

        at java.lang.Long.valueOf(Long.java:840)

        at org.h2.value.ValueLong.getObject(ValueLong.java:194)

        at org.apache.ignite.internal.processors.query.h2.opt.GridH2KeyValueRowOnheap.getValue0(GridH2KeyValueRowOnheap.java:122)

        at org.apache.ignite.internal.processors.query.h2.opt.GridH2KeyValueRowOnheap.getValue(GridH2KeyValueRowOnheap.java:106)

        at org.h2.table.TableFilter.getValue(TableFilter.java:1083)

        at org.h2.expression.ExpressionColumn.getValue(ExpressionColumn.java:186)

        at org.h2.expression.Alias.getValue(Alias.java:36)

        at org.h2.command.dml.Select$LazyResultQueryFlat.fetchNextRow(Select.java:1459)

        at org.h2.result.LazyResult.hasNext(LazyResult.java:79)

        at org.h2.result.LazyResult.next(LazyResult.java:59)

        at org.h2.command.dml.Select.queryFlat(Select.java:519)

        at org.h2.command.dml.Select.queryWithoutCache(Select.java:625)

        at org.h2.command.dml.Query.queryWithoutCacheLazyCheck(Query.java:114)

        at org.h2.command.dml.Query.query(Query.java:352)

        at org.h2.command.dml.Query.query(Query.java:333)

        at org.h2.command.CommandContainer.query(CommandContainer.java:113)

        at org.h2.command.Command.executeQuery(Command.java:201)

        at org.h2.jdbc.JdbcPreparedStatement.executeQuery(JdbcPreparedStatement.java:111)

        at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeSqlQuery(IgniteH2Indexing.java:1139)

        at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeSqlQueryWithTimer(IgniteH2Indexing.java:1207)

        at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeSqlQueryWithTimer(IgniteH2Indexing.java:1185)

        at org.apache.ignite.internal.processors.query.h2.twostep.GridMapQueryExecutor.onQueryRequest0(GridMapQueryExecutor.java:683)

        at org.apache.ignite.internal.processors.query.h2.twostep.GridMapQueryExecutor.onQueryRequest(GridMapQueryExecutor.java:527)

        at org.apache.ignite.internal.processors.query.h2.twostep.GridMapQueryExecutor.onMessage(GridMapQueryExecutor.java:218)

        at org.apache.ignite.internal.processors.query.h2.twostep.GridReduceQueryExecutor$1.applyx(GridReduceQueryExecutor.java:153)

        at org.apache.ignite.internal.processors.query.h2.twostep.GridReduceQueryExecutor$1.applyx(GridReduceQueryExecutor.java:151)

        at org.apache.ignite.internal.util.lang.IgniteInClosure2X.apply(IgniteInClosure2X.java:38)

        at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.send(IgniteH2Indexing.java:2648)

        at org.apache.ignite.internal.processors.query.h2.twostep.GridReduceQueryExecutor.send(GridReduceQueryExecutor.java:1418)

        at org.apache.ignite.internal.processors.query.h2.twostep.GridReduceQueryExecutor.query(GridReduceQueryExecutor.java:731)

        at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$8.iterator(IgniteH2Indexing.java:1397)

        at org.apache.ignite.internal.processors.cache.QueryCursorImpl.iterator(QueryCursorImpl.java:95)

[09:48:31,468][WARNING][grid-timeout-worker-#87][diagnostic] Found long running cache future [startTime=09:45:35.674, curTime=09:48:31.430, fut=GridNearAtomicSingleUpdateFuture [reqState=Primary [id=d272a480-8a0d-4cb4-a9d6-dcc5d376ff8f, opRes=false, expCnt=-1, rcvdCnt=0, primaryRes=false, done=false, waitFor=null, rcvd=null], super=GridNearAtomicAbstractUpdateFuture [remapCnt=100, topVer=AffinityTopologyVersion [topVer=7, minorTopVer=0], remapTopVer=null, err=null, futId=258864, super=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null, hash=1326538027]]]]

[09:48:31,475][WARNING][grid-timeout-worker-#87][diagnostic] Found long running cache future [startTime=09:45:35.674, curTime=09:48:31.430, fut=GridDhtAtomicSingleUpdateFuture [allUpdated=true, super=GridDhtAtomicAbstractUpdateFuture [futId=438170, resCnt=0, addedReader=false, dhtRes={af946716-d2df-4902-a9cd-8d6a85a866a2=[res=false, size=1, nearSize=0]}]]]

 

Answer 1

我们的ignite是最新版本的，总是提示Possible too long JVM pause，然后脱离ignite集群。但是实际上JVM在当时却没有发生GC停顿，请问是为什么？

Answer 2

只存储了100多万的数据，大概100多M，原来值设置了10g的heap，后面调到20g，依然不行。

Answer 3

ignite高并发下执行SQL为何很耗CPU

Answer 4

我采用最新2.7版本，没有使用堆内内存，运行几十分钟的并发压力测试之后，堆内几乎耗尽。 当我再停掉压力测试之后，堆内内存并没有释放，我采用的jvm参数是官方的，这是为什么？

Answer 5

你内存中存储太多数据了，导致GC时间过长，就可能抛出这个错误。

你要么调整JVM参数，避免这个错误，要么不要使用堆内内存。

Ignite新版已经默认使用堆外内存了，不会出现这样的问题，除非必要，已经不建议使用堆内内存了。