Sunspot-solr 占用了太多内存？有什么方便的方法可以减少内存使用？

Question

所以看来 Sunspot-Solr 正在消耗大量内存。这可能是由于我作为程序员分配正确使用它的不当行为。

首先，我不允许太阳黑子自行重新索引。

searchable :auto_index => false do

我认为仅此一点就阻止了它重新索引。相反，我在每天清晨运行一次的 cron-tab 上运行重新索引过程。

我开始这样做的原因是，随着应用程序的扩展，数据似乎在机器上堆积得如此之多，以至于仅仅加载主页就需要相当长的时间。

我做错了什么导致下面的错误？我能做得更好吗？

问题中的错误：

ActionView::TemplateError (Solr Response: Lock_obtain_timed_out_NativeFSLocktmpindexlucenede61b2c77401967646cf8916982a09a0writelock__orgapachelucenestoreLockObtainFailedException_Lock_obtain_timed_out_NativeFSLocktmpindexlucenede61b2c77401967646cf8916982a09a0writelock__at_orgapachelucenestoreLockobtainLockjava85__at_orgapacheluceneindexIndexWriterinitIndexWriterjava1545__at_orgapacheluceneindexIndexWriterinitIndexWriterjava1402__at_orgapachesolrupdateSolrIndexWriterinitSolrIndexWriterjava190__at_orgapachesolrupdateUpdateHandlercreateMainIndexWriterUpdateHandlerjava98__at_orgapachesolrupdateDirectUpdateHandler2openWriterDirectUpdateHandler2java173__at_orgapachesolrupdateDirectUpdateHandler2addDocDirectUpdateHandler2java220__at_orgapachesolrupdateprocessorRunUpdateProcessorprocessAddRunUpdateProcessorFactoryjava61__at_orgapachesolrhandlerXMLLoaderprocessUpdateXMLLoaderjava139__at_orgapachesolrhandlerXMLLoaderloadXMLLoaderjava69__at_orgapachesolrhandlerContentStreamHandlerBasehandleRequestBodyContentStreamHandlerBasejava54__at_orgapachesolrhandlerRequestHandlerBasehandleRequestRequestHandlerBasejava131__at_orgapachesolrcoreSolrCoreexecuteSolrCorejava1316__at_orgapachesolrservletSolrDispatchFilterexecuteSolrDispatchFilterjava338__at_orgapachesolrservletSolrDispatchFilterdoFilterSolrDispatchFilterjava241__at_orgmortbayjettyservletServletHandler$CachedChaindoFilterServletHandlerjava1089__at_orgmortbayjettyservletServletHandlerhandleServletHandlerjava365__at_orgmortbayjettysecuritySecurityHandlerhandleSecurityHandlerjava216__at_orgmortbayjettyservletSessionHandlerhandleSessionHandlerjava181__at_orgmortbayjettyhandlerContextHandlerhandleContextHandlerjava712__at_orgmortbayjettywebappWebAppContexthandleWebAppContextjava405__at_orgmortbayjettyhandlerContextHandlerCollectionhandleContextHandlerCollectionjava211__at_orgmortbayjettyhandlerHand) on line #24 of app/views/main/_main_nav.html.haml:
21:         %br
22:         community calendars
23: 
24:   - if (current_user.blank? || current_user.card_signup.blank?)
25:     %li
26:       - if current_user.blank?
27:         = link_to 'Get Your HQcard', signup_path, :title => "Signup for your free HQcard and redeem local deals and promotions."

    rsolr (0.12.1) [v] lib/rsolr/connection/requestable.rb:39:in `request'
    rsolr (0.12.1) [v] lib/rsolr/client.rb:34:in `request_without_rails_logging'
    /usr/lib/ruby/gems/1.8/gems/sunspot_rails-1.2.1/lib/sunspot/rails/solr_logging.rb:25:in `request'
    /usr/lib/ruby/gems/1.8/gems/sunspot_rails-1.2.1/lib/sunspot/rails/solr_logging.rb:24:in `request'
    rsolr (0.12.1) [v] lib/rsolr/client.rb:22:in `update'
    rsolr (0.12.1) [v] lib/rsolr/client.rb:46:in `add'
    sunspot (1.2.1) lib/sunspot/indexer.rb:101:in `add_documents'
    sunspot (1.2.1) lib/sunspot/indexer.rb:26:in `add'
    sunspot (1.2.1) lib/sunspot/session.rb:91:in `index'
    sunspot (1.2.1) lib/sunspot/session_proxy/abstract_session_proxy.rb:11:in `index'
    sunspot (1.2.1) lib/sunspot.rb:175:in `index'
    /usr/lib/ruby/gems/1.8/gems/sunspot_rails-1.2.1/lib/sunspot/rails/searchable.rb:349:in `solr_index'
    /usr/lib/ruby/gems/1.8/gems/sunspot_rails-1.2.1/lib/sunspot/rails/searchable.rb:405:in `maybe_auto_index'
    vendor/gems/binarylogic-authlogic-2.1.1/lib/authlogic/acts_as_authentic/session_maintenance.rb:73:in `save_without_session_maintenance'
    vendor/gems/binarylogic-authlogic-2.1.1/lib/authlogic/session/callbacks.rb:83:in `save_record'
    vendor/gems/binarylogic-authlogic-2.1.1/lib/authlogic/session/priority_record.rb:30:in `save_record'
    vendor/gems/binarylogic-authlogic-2.1.1/lib/authlogic/session/persistence.rb:60:in `persisting?'
    vendor/gems/binarylogic-authlogic-2.1.1/lib/authlogic/session/persistence.rb:39:in `find'
    app/controllers/application_controller.rb:23:in `current_user_session'
    app/controllers/application_controller.rb:28:in `current_user'
    (eval):2:in `send'
    (eval):2:in `current_user'
    app/views/main/_main_nav.html.haml:24:in `_run_haml_app47views47main47_main_nav46html46haml_locals_main_nav_object'
    haml (2.2.2) [v] lib/haml/helpers/action_view_mods.rb:11:in `render'
    haml (2.2.2) [v] lib/haml/helpers.rb:96:in `non_haml'
    haml (2.2.2) [v] lib/haml/helpers/action_view_mods.rb:11:in `render'
    app/views/main/index.html.haml:2:in `_run_haml_app47views47main47index46html46haml'
    haml (2.2.2) [v] lib/haml/helpers/action_view_mods.rb:13:in `render'
    haml (2.2.2) [v] lib/haml/helpers/action_view_mods.rb:13:in `render'
    haml (2.2.2) [v] rails/./lib/sass/plugin/rails.rb:19:in `process'
    lib/flash_session_cookie_middleware.rb:14:in `call'
    vendor/gems/hoptoad_notifier-2.2.2/lib/hoptoad_notifier/rack.rb:27:in `call'

Answer 1

您可以根据当前环境在 config/sunspot.yml 文件中指定最小和最大内存分配，例如

development:
  solr:
    min_memory: 512M
    max_memory: 1G
    solr_jar: /path/to/start.jar
    log_level: DEBUG

文档：http://outoftime.github.com/sunspot/rails/docs/classes/Sunspot/Rails/Configuration.html

Answer 2

max_memory 和 min_memory 已更改为 memory rel="nofollow noreferrer">https://github.com/sunspot/sunspot/commit/74ea3881872398bd14cf6e96481826141ee82c93

所以当前的答案应该是：

development:
  solr:
    memory: 1G
    solr_jar: /path/to/start.jar
    log_level: DEBUG

Answer 3

我反转了 Lucene 引擎的执行并强制它使用 64 位 JVM，这允许它完全访问我的机器上的所有可用内存...

为什么 rake 在 64 位 Windows 上使用 32 位 JVM 超出了我的范围...

我现在使用 sunspot:solr 运行脚本手动执行：

cd \RailsInstaller\Ruby1.9.3\lib\ruby\gems\1.9.1\gems\sunspot_solr-2.1.0\solr\

java -d64 -Djetty.port=8982
-Dsolr.data.dir=C:/[您的项目目录]/solr/data/development -Dsolr.solr.home=solr -Djava.util.logging.config.file=C:/Users/[用户-名称]/AppData/Local/Temp/logging.properties20140117
-jar启动.jar