HBase MapReduce 是否支持组合器阶段?如果是这样,怎么办?
Hadoop MapReduce 支持组合器阶段。但是,我在 HBase MapReduce 包中找不到类似的功能。它存在吗?
Hadoop map reduce supports a combiner stage. However, I can't find a similar capability in the HBase MapReduce package. Does it exist?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(2)
如果您正在运行使用 HBase 作为输入格式和非 hbase 输出格式的 MapReduce 作业,则情况是相同的。
组合器是一种仅对单个映射器的输出进行操作的缩减器。只要你只在你的reducer中做幂等的事情,那么你可以通过简单地说:job.setCombiner(.class);将你的reducer放入combiner槽中。
If you are running a MapReduce job with HBase as an input format and non-hbase output formats it is the same.
A combiner is a reducer that only operates on the output of a single mapper. As long as you only do things that are idempotent in your reducer, than you can put your reducer in the combiner slot by simply saying: job.setCombiner(.class);
您可以使用 job.setCombiner(.class);在MapReduce中使用Combiner类的方法。减速器也将充当组合器
You can use job.setCombiner(.class); method to use combiner class in map reduce. Reducer will aslso acts as an combiner