自定义 MapReduce 输入格式 - 找不到构造函数

Question

我正在为 Hadoop 0.20.2 编写一个自定义 InputFormat，但遇到了无法摆脱的 NoSuchMethodException。我一开始是：

public class ConnectionInputFormat extends FileInputFormat<Text, Connection> {

    @Override
    public RecordReader<Text, Connection> createRecordReader(InputSplit split, TaskAttemptContext context) throws IOException, InterruptedException {
        return new ConnectionRecordReader();
    }
}

运行时出现此错误：

Exception in thread "main" java.lang.RuntimeException: java.lang.NoSuchMethodException: testingground.TestInputJob$ConnectionInputFormat.<init>()
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:882)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:779)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
at testingground.TestInputJob.run(TestInputJob.java:141)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at testingground.TestInputJob.main(TestInputJob.java:156)
Caused by: java.lang.NoSuchMethodException: testingground.TestInputJob$ConnectionInputFormat.<init>()
at java.lang.Class.getConstructor0(Class.java:2706)
at java.lang.Class.getDeclaredConstructor(Class.java:1985)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:109)
... 8 more
Java Result: 1

在最初收到错误并在线研究后，我认为可能是我没有零参数构造函数，所以我添加了一个：

public class ConnectionInputFormat extends FileInputFormat<Text, Connection> {

    public ConnectionInputFormat() {
        System.out.println("NetflowInputFormat Constructor");
    }

    @Override
    public RecordReader<Text, Connection> createRecordReader(InputSplit split, TaskAttemptContext context) throws IOException, InterruptedException {
        return new ConnectionRecordReader();
    }
}

这也不起作用，所以我添加了第二个构造函数，它接受任意数量的对象：

public class ConnectionInputFormat extends FileInputFormat<Text, Connection> {

    public ConnectionInputFormat() {
        System.out.println("NetflowInputFormat Constructor");
    }

    public ConnectionInputFormat(Object... o) {
        System.out.println("NetflowInputFormat Constructor");
    }

    @Override
    public RecordReader<Text, Connection> createRecordReader(InputSplit split, TaskAttemptContext context) throws IOException, InterruptedException {
        return new ConnectionRecordReader();
    }
}

仍然遇到相同的错误，并且到目前为止尚未成功找到解决方案。

当前完整源代码：http://pastebin.com/2XyW5ZSS

Answer 1

您的 ConnectionInputFormat 类应该是静态的。非静态嵌套类在每个构造函数中添加了隐含的“this”。因此，您的无参数构造函数实际上有一个不可见的参数，除非该类被声明为静态。