ibatis：提高插入性能
我正在使用 ibatis 进行 sql insert stmt。在我的代码中，我正在逐行解析文件夹中的文件。匹配条件的每一行都需要插入到数据库中。 单次运行程序中的插入总数可以是 200k 左右。
SqlSession sess = null;
this.sess = sf.openSession(ExecutorType.BATCH, false);
for (each file) {
for( each line matching criteria ){
this.sess.insert("com.logs.util.insertFileInfo", fileData);
insertcount++;
if(insert count == 10)
this.sess.commit();
}
}
if(insert count > 0){
this.sess.commit();
}
}
这种风格会慢慢占用大量内存，并且一段时间后会抛出 OutOfMemory 异常。 我怎样才能提高这里的性能？
I am using ibatis for my sql insert stmt. In my code I am parsing files line by line from a folder. Each line that matches criteria, need to be inserted into database.
Total number of insert in a single run of program can be any where along 200k.
SqlSession sess = null;
this.sess = sf.openSession(ExecutorType.BATCH, false);
for (each file) {
for( each line matching criteria ){
this.sess.insert("com.logs.util.insertFileInfo", fileData);
insertcount++;
if(insert count == 10)
this.sess.commit();
}
}
if(insert count > 0){
this.sess.commit();
}
}
This style slowly takes up lot of memory and after some times throws OutOfMemory exception.
How can I improve performance here?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
您打算每 10 次插入后提交一次吗？看起来您只在前 10 次插入后才这样做。我想您需要类似的东西
那些未提交的更改必须存储在某个地方。我对 Ibatis 不熟悉，但如果未提交的更改存储在 Ibatis 分配的缓冲区中，那么如果您不提交更改，最终将耗尽内存。
Is it your intention to commit after every 10 inserts? It looks like you only do so after the first 10 inserts. I guess that you would need something like
Those uncommitted changes have to be stored somewhere. I'm not familiar with Ibatis but if the uncommitted changes are being stored in a buffer allocated by Ibatis then you will eventually run out of memory if you don't commit the changes.