在 Google 应用引擎上运行 hadoop?
是否可以在 Google 应用程序引擎上运行地图缩减作业?
任何参考或教程都会有帮助
谢谢
Is it possible to run map reduce jobs on Google app engine?
Any reference or tutorial would help
Thanks
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(4)
有点像。
您无法使用实际的 MapReduce 框架 - 该架构与 AppEngine 太不兼容。
不过,有一个专为 GAE 构建的等效系统 - appengine-mapreduce。该站点有点令人困惑,因为代码的第一个版本仅支持映射器,而没有后续的减少步骤 - 最近他们发布了一个具有完整映射器支持的版本,但一些文档仍然引用了早期的仅映射器版本。
最好的介绍是 GoogleIO Mike Aizatskyi 的演讲。
Sort of.
You can't use the actual MapReduce framework - the architecture is too incompatible with AppEngine.
However, there is an equivalent system built specficially for GAE - appengine-mapreduce. That site is a bit confusing, as the first version of the code only supported mappers, without the subsequent reduce step - recently they released a version with full mapreduce support, but some of the documentation still referes to the earlier mapper-only one.
The best introduction is the GoogleIO talk from Mike Aizatskyi.
您无法在 Appengine 上运行 Hadoop(也无法访问文件系统)。
您可能需要检查 AWS ElasticMapreduce。它是一个基于云的平台,用于运行 Mapreduce 作业。
ElasticMapreduce
You cannot run Hadoop on Appengine (No filesystem access as well).
You may want to check AWS ElasticMapreduce. Its a cloud based platform for running Mapreduce jobs.
ElasticMapreduce
以下是完整文档:https://developers.google.com/appengine/docs /python/数据处理/概述
Here is the full documentation: https://developers.google.com/appengine/docs/python/dataprocessing/overview
看到这个 Google Cloud Platform 广告:
Google Compute Engine 虚拟机上的 Hadoop
https://cloud.google.com/solutions/hadoop
Saw this Google Cloud Platform advertisement:
Hadoop on Google Compute Engine virtual machines
https://cloud.google.com/solutions/hadoop