对离线 Solr/Lucene 索引调用 commit 的频率是多少?

发布于 2024-10-31 19:40:59 字数 379 浏览 1 评论 0原文

我知道存在一些半类似的问题,但在这种情况下,我正在构建一个离线索引,直到构建完成。我正在从头开始构建两个核心,一个拥有大约 300k 条记录,包含大量引文信息和大块全文(这是文档索引),另一个核心拥有大约 660 万条记录,包含全文(这是页面索引) )。

鉴于该索引是离线构建的,唯一真正的性能问题是构建速度。任何人都不应该查询这些数据。

如果我停止添加项目 50 秒,自动提交显然会触发?我不这样做。我一次添加十个,并且每隔几秒添加一次。

那么,我应该更频繁地承诺吗?我觉得运行的时间越长,速度就越慢,至少在我要索引的 6k 文档的测试用例中是这样。

如果没有人搜索该索引,那么有人会建议我提交多久一次?

应该说我正在使用 Solr 3.1 和 SolrNet。

I know there have been some semi-similar questions, but in this case, I am building an index which is offline, until build is complete. I am building from scratch two cores, one has about 300k records with alot of citation information and large blocks of full text (this is the document index) and another core which has about 6.6 Million records, with full text (this is the page index).

Given this index is being built offline, the only real performance issue is speed of building. Noone should be querying this data.

The auto-commit would apparently fire if I stop adding items for 50 seconds? Which I don't do. I am adding ten at a time and they are added every couple seconds.

So, should I commit more often? I feel like the longer this runs the slower it gets, at least in my test case of 6k documents to index.

With noone searching this index, how often would anyone suggest I commit?

Should say I am using Solr 3.1 and SolrNet.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

鼻尖触碰 2024-11-07 19:40:59

尽管提交需要花费时间,但您可能需要考虑考虑提交频率之外的其他调整。

索引核心是否也进行搜索,或者在索引结束后将其复制到其他地方?如果是后者,那么关闭缓存可能会对性能产生非常明显的影响(solr 重建缓存每次你提交)。

Although it's commits that are taking time for you, you might want to consider looking into other tweaking than commit frequency.

Is it the indexing core that also does searching, or is it replicated somewhere else after indexing concludes? If the latter is the case, then turning off caches might have a very noticeable impact on performance (solr rebuilds caches every time you commit).

ˉ厌 2024-11-07 19:40:59

您还可以考虑使用 Solr 的 autoCommit 或 commitWith 功能。

You could also look into using the autoCommit or commitWith features of Solr.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文