Amazon Auto Scaling 服务可以与 Elastic Map Reduce 服务配合使用吗？
由于亚马逊网络服务需要付费，所以在我开始使用它之前想问问谁曾参与过它，并确认一些关于它的知识。
问题一： 在亚马逊自动扩展服务中，它表示可以扩展和缩小实例。 这是什么意思？
这是否意味着更改实例类型？或者可以根据定义的条件启动/停止更多/更少的实例？
问题二： 自动缩放框架可以与MapReduce一起使用吗？
例如，如果我有一个极端的情况，我将有无穷无尽的任务，并且完成任务的时间会有所不同，并且可能会非常计算密集。 那么我可以使用自动缩放服务来缩放地图缩小吗？
谢谢
since amazon web service need to pay, so just wanna ask ppl who had worked on it before i jump into it, and confirm some knowledge about it.
Question one:
In Amazon auto scaling service, it says can scale up and down instances.
that does this mean?
does it mean changing the type of instance? or can start/stop more/less instance base on the condition define?
Question two:
can the auto scaling framework work with map reduce?
for example, if i have a extreme case, i will have endless tasks, and the time to finish the task will be vary, and might be very computation intensive.
so can i use auto scaling service to scale the map reduce?
Thanks
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
关于。问题一：“纵向扩展和横向缩减”意味着他们可以根据需要添加更多实例，然后在不再需要时减少实例数量（即可以关闭虚拟机实例）。
关于。问题二：我认为自动缩放对于 MapReduce 来说并没有真正意义，至少如果您正在考虑 Hadoop：Map/Reduce 背后的主要思想之一Hadoop 的特点是数据被分散在许多节点中，然后节点可以处理距离它们很近的数据 - 理想情况下在它们的本地磁盘上或至少在靠近（物理）的另一台机器上接近。启动新实例当然是可能的，但只有当您不必将要处理的数据复制到新实例时，这才有意义。希望这个解释对您有意义...无论如何，请查看 Amazon 的 Elastic Map Red 产品，至少如果我正确理解你的问题的话，它会满足你的要求。
re. question one: "scale up and scale down" means they can add more instances as needed and the number of instances can then be reduced (i.e. virtual machine instances can be shut down) when they're no longer required.
re. question two: I would argue that auto-scaling does not really make sense for map reduce, at least if you're thinking about Hadoop: One of the main ideas behind Map/Reduce & Hadoop is that the data is split up among a lot of nodes and the nodes can then process the data that is close to them - ideally on their local disk or at least on another machine in close (physical) proximity. Starting a new instance would of course be possible, but that makes sense only as long as you do not have to copy the data you want to process over to the new instance. Hope this explanation makes sense to you... In any case, check out Amazon's Elastic Map Reduce offering, which does what you want, at least if I understood your question correctly.
您可以查看 Themis，这是 Atlassian 开发的 EMR 自动缩放框架。当前的功能包括反应式自动缩放（基于当前集群负载）以及主动式自动缩放（基于预定义的计划），它配备了 Web UI，并且该工具非常易于配置。
（很抱歉在旧线程中发帖，但对于发现该线程的读者来说，答案可能仍然很有趣。）
You could take a look at Themis, an EMR autoscaling framework developed at Atlassian. Current features include reactive autoscaling (based on current cluster load) as well as proactive autoscaling (based on predefined schedules), it comes with a Web UI, and the tool is very easy to configure.
(Apologies for posting in an old thread, but the answer may still be interesting for readers discovering this thread.)