如何限制 AppEngine 上的 appengine-mapreduce?
http://code.google.com/p/appengine-mapreduce/ 提到它可以控制执行速度,但我不知道如何控制。为映射器作业创建一个任务队列并从那里控制速度是有意义的,但我不知道如何指定要使用哪个队列。
http://code.google.com/p/appengine-mapreduce/ mentions that it can control the speed of execution, but I can't figure out how. It would make sense to create a task queue for a mapper job and control the speed from there, but I don't see how to specify which queue to use.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(3)
(感谢 Chris 的指点,但之后需要进行一些挖掘才能找到确切的参数。)
来自 http://code.google.com/p/appengine-mapreduce/wiki/UserGuideJava 将此添加到您的 mapreduce.xml 配置元素中,以限制每秒处理的实体数量:
(Thanks to Chris for the pointer, but it took a bit of digging after that to find the exact parameter.)
From http://code.google.com/p/appengine-mapreduce/wiki/UserGuideJava add this to your mapreduce.xml configuration element to limit the number of entities processed per second:
我自己不使用 Java 版本,但 Python 版本有一个
processing_rate
参数,您可以将其传递给映射器规范。在 Java 源代码中快速搜索会发现 MAPPER_INPUT_PROCESSING_RATE_KEY 配置键,希望它能为您指明正确的方向。
I don't use the Java version myself, but the Python version has a
processing_rate
param that you pass to the mapper spec.A quick search in the Java source reveals a MAPPER_INPUT_PROCESSING_RATE_KEY config key, which hopefully points you in the right direction.
对于使用 python 的用户,您可能需要在 model.py 中搜索名为“_DEFAULT_PROCESSING_RATE_PER_SEC”的变量,以提高每秒的默认处理速率。这对我有用!
For users using python, you may want to search for this variable known as "_DEFAULT_PROCESSING_RATE_PER_SEC" in model.py to increase the default processing rate per second. That worked for me!