MapReduceBase 和 Mapper 已弃用

发布于 2024-12-07 15:12:09 字数 1390 浏览 0 评论 0原文

public static class Map extends MapReduceBase implements Mapper

MapReduceBase、Mapper 和 JobConf 在 Hadoop 0.20.203 中已弃用。

我们现在应该用什么？

编辑 1 - 对于 Mapper 和 MapReduceBase，我发现我们只需要扩展 Mapper

public static class Map extends Mapper
            <LongWritable, Text, Text, IntWritable> {
  private final static IntWritable one = new IntWritable(1);
  private Text word = new Text();

  public void map(LongWritable key, Text value, 
         OutputCollector<Text, IntWritable> output, 
         Reporter reporter) throws IOException {
    String line = value.toString();
    StringTokenizer tokenizer = new StringTokenizer(line);
    while (tokenizer.hasMoreTokens()) {
      word.set(tokenizer.nextToken());
      output.collect(word, one);
    }
  }
}

编辑 2 - 对于 JobConf< /code> 我们应该使用这样的配置：

public static void main(String[] args) throws Exception {
        Configuration conf = new Configuration();
        Job job = new Job(conf);
        job.setMapperClass(WordCount.Map.class);
    }

编辑 3 - 我根据新的 API 找到了一个很好的教程：http://sonerbalkir.blogspot.com/2010/01/new-hadoop -api-020x.html

原文

public static class Map extends MapReduceBase implements Mapper

MapReduceBase, Mapper and JobConf are deprecated in Hadoop 0.20.203.

What should we use now?

Edit 1 - for the Mapper and the MapReduceBase, I found that we just need to extends the Mapper

public static class Map extends Mapper
            <LongWritable, Text, Text, IntWritable> {
  private final static IntWritable one = new IntWritable(1);
  private Text word = new Text();

  public void map(LongWritable key, Text value, 
         OutputCollector<Text, IntWritable> output, 
         Reporter reporter) throws IOException {
    String line = value.toString();
    StringTokenizer tokenizer = new StringTokenizer(line);
    while (tokenizer.hasMoreTokens()) {
      word.set(tokenizer.nextToken());
      output.collect(word, one);
    }
  }
}

Edit 2 - For JobConf we should use configuration like this:

public static void main(String[] args) throws Exception {
        Configuration conf = new Configuration();
        Job job = new Job(conf);
        job.setMapperClass(WordCount.Map.class);
    }

Edit 3 - I found a good tutorial according to the new API : http://sonerbalkir.blogspot.com/2010/01/new-hadoop-api-020x.html

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

千仐 2024-12-14 15:12:09

Javadoc 包含该废弃类的使用信息：

例如 http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred/JobConf.html

 Deprecated. Use Configuration instead

编辑：当您使用maven和开放类声明（F3）时maven可以自动下载源代码，你会看到javadoc注释的内容和解释。

Javadoc contains info what to use instaed of this depraceated classes:

e.g. http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred/JobConf.html

 Deprecated. Use Configuration instead

Edit: When you use maven and open class declaration (F3) maven can automatically download source code and you'll see content of javadoc comments with explanations.

回复收藏 0 原文