哪种类型的工作负载适合在 Amazon EC2 Spot 实例上使用?
Amazon 刚刚宣布为其基于 EC2 的基础设施推出“Spot 实例”。我想知道什么样的工作负载适合这样的服务?
Spot 实例使您能够竞价 未使用的 Amazon EC2 容量。实例 按现货价格收取 Amazon EC2,波动 根据供应情况定期 Spot 实例的数量和需求 容量。
JIT 的理念很简单:库存就是浪费。
编辑:
我想知道是否有应用程序可以通过利用大量这些 Spot 实例来维持自身(阅读:可行)。想一想:想象一下,您平均以 1 个实例的价格获得 10 个实例……当然不能保证,但在没有可用的 Spot 实例的情况下,可能会踢掉许多“正常”实例。
Amazon just announced "Spot Instances" for their EC2 based infrastructure. I was wondering what sort of workloads would be appropriate for such service?
Spot Instances enable you to bid for
unused Amazon EC2 capacity. Instances
are charged the Spot Price set by
Amazon EC2, which fluctuates
periodically depending on the supply
of and demand for Spot Instance
capacity.
The philosophy of JIT is simple: inventory is waste.
EDITED:
I wonder if there are applications that could sustain themselves (read: be viable) just by leveraging a large volume of those Spot instances. Think about it: imagine you get 10 instances for the price of 1 on average... of course there wouldn't be guarantees but in the case that no Spot instances are available, a number of "normal" instances could be kicked-of.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(4)
显然,这适用于任何不需要实时的工作负载。
让我们以较小的规模来说,这如何应用于 stackoverflow?例如,该网站上的许多徽章都不是实时计算的。有一个定期的流程来评估资格,无论每天凌晨 4 点还是下午 4 点运行,只要运行即可。凌晨 4 点做可能会便宜 5 美分。 (显然他们根本不使用 EC2)
规模更大?针对大量数据的搜索引擎可能需要巨大的计算能力来构建索引。如果您每天对新数据建立索引,并且需要花费 2 小时在数百台服务器上对它们建立索引,那么您可以在一夜之间完成,并且每天可能节省数千美元。
通过全天候分散工作量,亚马逊可以最大限度地利用其资源,从而提供市场上最便宜的价格。
Obviously this is for any workload that doesn't need to be real-time.
Let's say on smaller scale, how this could apply to stackoverflow? For example, many badges on this site are not calculated in real-time. There is periodical process that will evaluate eligibility and it doesn't matter whether it runs at 4am or 4pm everyday as long as it runs. Doing it at 4am could be 5 cents cheaper. (obviously they don't use EC2 at all for this)
Larger scale? Search engine over large set of data might need huge computing capacity to build its indexes. If you index new data once a day and it takes 2 hours to index them on hundreds of servers, you can do it overnight and save perhaps thousands of dollars every day.
By spreading workload around the clock helps Amazon maximize utilization of their resources and therefore provide the cheapest prices on the market.
Amazon 只能想到这些工作负载:
现货实例让我想起“双价电表”,即当需求较少时,您支付的能源费用会更少。我认为这是一个非常有趣的概念,也是对云的一个意想不到的介绍,但它可能很难应用于传统的业务问题。
Amazon could only think of these workloads:
Spot Instances remind me of "double tariff electricity meters", where you pay less for energy when the demand is less. I think it is a very interesting concept, and quite an unexpected introduction to the cloud, but it will probably be difficult to apply to conventional business problems.
我正在考虑建立一个灵活的集群(例如 HADOOP),其主干在常规实例上运行,并以较低的现货价格运行几组附加实例。随着价格下降,更多实例可用于处理工作单元。如果价格上涨,节点将被关闭。集群通过将工作单元重新发布到其他节点来处理此问题,就像在节点发生故障时一样。
显然,这是一个相当敌对的环境,因此需要进行一些调整。如果您对全局文件系统使用标准的三倍复制,并且包含该块的三个节点同时关闭,那么您就失败了。分散现货实例价格可以降低一举失去许多实例的可能性。增加复制因子将减少影响,并且无论如何,实例的磁盘空间都是空闲的,因此这不会成为一个因素。这够了吗?我们拭目以待。
I am considering setting up a flexible cluster (say HADOOP) with a backbone that runs on regular instances and a few sets of additional instances at decreasing spot prices. As the price drops, additional instances become available to process work units. If the price increases, nodes will be shut down. The cluster handles this by re-issuing the work units to other nodes, just as it would in case of node failure.
Obviously this is a rather hostile environment so some adjustments need to be made. If you work with standard 3-fold replication for the global filesystem and the three nodes containing the block are shut down at the same time, you lose. Spreading the spot instance prices decreases the likelihood of losing many in one fell swoop. Increasing the replication factor will reduce the impact, and disk space is free with the instance anyway so that won't be a factor. Is this enough? We'll see.
有一些明显的用例,例如批处理或不需要 24/7 运行的任务。
其他有趣的实现是为了额外的容量。您可以混合使用按需实例和现货实例来运行您的网站。按需实例将作为您的“核心”。如果您的现货实例在这里或那里停机了几个小时,您的按需实例可能会工作得更困难一些,但您的网站仍然可以访问。
There are obvious use cases such as batch processing or tasks that don't need to be running 24/7.
Other interesting implementations are for extra capacity. You could use a mix of on-demand and spot instance to run your website. The on-demand instances would serve as your 'core'. If your spot instances are down for a few hours here or there your on-demand instance may work a little harder but your website would still be accessible.