克服 appengine 搜索的条目大小限制?
我正在尝试对博客条目使用可搜索模型,它在开发平台上运行良好,但是当我尝试在云中添加条目时,我收到错误:
Traceback (most recent call last):
File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/init.py", line 517, in call
handler.post(*groups) File
"/base/data/home/apps/smart-fast/1.348228399174418277/admin.py", line 76, in post
article.put()
File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/db/init.py", line 895, in put
return datastore.Put(self._entity, config=config)
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/datastore.py", line 404, in Put
return _GetConnection().async_put(config, entities, extra_hook).get_result()
File "/base/python_runtime/python_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 601, in get_result
self.check_success()
File "/base/python_runtime/python_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 572, in check_success
rpc.check_success()
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 502, in check_success
self.__rpc.CheckSuccess()
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/apiproxy_rpc.py", line 126, in CheckSuccess
raise self.exception
ApplicationError:
ApplicationError: 1 Too many indexed > properties for entity: app:
"smart-fast",path < Element { type: "Article", id: 2002 }> This index put
it over the limit: entity_type: "Article",ancestor: false,
Property { name: "searchable_text_index", direction: ASCENDING},
Property { name: "searchable_text_index", direction: ASCENDING},
Property { name: "date", direction: ASCENDING}
这些条目不是那么大(<500 个字)限制这么低?我认为解决这个问题的唯一方法是将条目存储为不可搜索的模型,并将条目文本分解为更小的可搜索模型,每个模型都引用主条目。 非常感谢任何帮助
I am trying to use a searchable model for blog entries and it worked fine on the development platform but when I try to add an entry in the cloud I get an error:
Traceback (most recent call last):
File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/init.py", line 517, in call
handler.post(*groups) File
"/base/data/home/apps/smart-fast/1.348228399174418277/admin.py", line 76, in post
article.put()
File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/db/init.py", line 895, in put
return datastore.Put(self._entity, config=config)
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/datastore.py", line 404, in Put
return _GetConnection().async_put(config, entities, extra_hook).get_result()
File "/base/python_runtime/python_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 601, in get_result
self.check_success()
File "/base/python_runtime/python_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 572, in check_success
rpc.check_success()
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 502, in check_success
self.__rpc.CheckSuccess()
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/apiproxy_rpc.py", line 126, in CheckSuccess
raise self.exception
ApplicationError:
ApplicationError: 1 Too many indexed > properties for entity: app:
"smart-fast",path < Element { type: "Article", id: 2002 }> This index put
it over the limit: entity_type: "Article",ancestor: false,
Property { name: "searchable_text_index", direction: ASCENDING},
Property { name: "searchable_text_index", direction: ASCENDING},
Property { name: "date", direction: ASCENDING}
These entries aren't that large(<500 words) is the limit that low? The only way I can think to get around this is to store the entry as a non-searchable model and also break the entry text up into smaller searchable models, that each reference the main entry.
Any help is greatly appreciated
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
您似乎遇到的情况称为爆炸索引。您可以在此处了解有关它们的更多信息:http://code.google .com/appengine/docs/python/datastore/queries.html#Big_Entities_and_Exploding_Indexes
基本上,您无法使用 AppEngine 创建全文引擎,除非您的数据集非常小。您要么会遇到索引爆炸的问题,要么会遇到其他问题(即合并连接超时)。我建议您研究一个名为 IndexTank 的服务,这是一个非常好的全文搜索服务。它有完整的 REST API 和 Python 客户端,因此可以轻松使用 AppEngine。
What you seem to be running into are called Exploding Indexes. You can read more about them here: http://code.google.com/appengine/docs/python/datastore/queries.html#Big_Entities_and_Exploding_Indexes
Basically you can't use AppEngine to create a full-text engine, unless you have a very small dataset. It's either you'll run into exploding indexes or you'll run into other issues (i.e. merge join timeouts). I recommend that you look into a service called IndexTank which is a very good full-text search service. It has a full REST api and python client so it's easy to get going on AppEngine.