Apache Flink -Java.lang.noclassdeffounderror:org/apache/flink/api/connector/connector/sink2/sink2/sink

发布于 2025-01-27 18:42:31 字数 3377 浏览 2 评论 0原文

我写了一条flink管道,该管道写入数据流以镶木式格式提交。我使用sinkto方法将输出写入文件。当应用程序启动时,我会得到以下例外。

java.lang.RuntimeException: Could not look up the main(String[]) method from the class com.fk.logs.StreamingJob: org/apache/flink/api/connector/sink2/Sink
    at org.apache.flink.client.program.PackagedProgram.hasMainMethod(PackagedProgram.java:315)
    at org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:161)
    at org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:65)
    at org.apache.flink.client.program.PackagedProgram$Builder.build(PackagedProgram.java:691)
    at org.apache.flink.client.cli.CliFrontend.buildProgram(CliFrontend.java:851)
    at org.apache.flink.client.cli.CliFrontend.getPackagedProgram(CliFrontend.java:271)
    at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:245)
    at org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1054)
    at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1132)
    at org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:28)
    at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1132)
Caused by: java.lang.NoClassDefFoundError: org/apache/flink/api/connector/sink2/Sink
    at java.base/java.lang.Class.getDeclaredMethods0(Native Method)
    at java.base/java.lang.Class.privateGetDeclaredMethods(Class.java:3166)
    at java.base/java.lang.Class.getMethodsRecursive(Class.java:3307)
    at java.base/java.lang.Class.getMethod0(Class.java:3293)
    at java.base/java.lang.Class.getMethod(Class.java:2106)
    at org.apache.flink.client.program.PackagedProgram.hasMainMethod(PackagedProgram.java:307)
    ... 10 more
Caused by: java.lang.ClassNotFoundException: org.apache.flink.api.connector.sink2.Sink
    at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:476)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
    at org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:64)
    at org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:65)
    at org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)

参考法截面的代码

DataStream<String> stream = env.fromSource(getSource(parser), WatermarkStrategy.noWatermarks(),
                "event source").uid("log file event source").filter(f -> !f.isEmpty());
        readLogFile(parser, stream)
                .flatMap((value, out) -> value.lines()
                        .forEach(out::collect), TypeInformation.of(String.class))
                .map(StreamingJob::parseLine).filter(Optional::isPresent)
                .map(Optional::get, mapType)
                .map(lm -> {
                    if (lm.containsKey("msg")) {
                        parseLine(lm.get("msg").toString()).ifPresent(mf -> lm.putAll(mf));
                    }
                    return lm;
                }, mapType)
                .keyBy(m -> m.computeIfAbsent("appid", k -> "unknownapp"))
                .map(m -> newRecord(m))
                .sinkTo(getSink());

使用了1.15版本。

I wrote a Flink pipeline that writes the data stream to file in parquet format. I use sinkTo method to write the output to the file. I get the below exception when the application starts.

java.lang.RuntimeException: Could not look up the main(String[]) method from the class com.fk.logs.StreamingJob: org/apache/flink/api/connector/sink2/Sink
    at org.apache.flink.client.program.PackagedProgram.hasMainMethod(PackagedProgram.java:315)
    at org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:161)
    at org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:65)
    at org.apache.flink.client.program.PackagedProgram$Builder.build(PackagedProgram.java:691)
    at org.apache.flink.client.cli.CliFrontend.buildProgram(CliFrontend.java:851)
    at org.apache.flink.client.cli.CliFrontend.getPackagedProgram(CliFrontend.java:271)
    at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:245)
    at org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1054)
    at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1132)
    at org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:28)
    at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1132)
Caused by: java.lang.NoClassDefFoundError: org/apache/flink/api/connector/sink2/Sink
    at java.base/java.lang.Class.getDeclaredMethods0(Native Method)
    at java.base/java.lang.Class.privateGetDeclaredMethods(Class.java:3166)
    at java.base/java.lang.Class.getMethodsRecursive(Class.java:3307)
    at java.base/java.lang.Class.getMethod0(Class.java:3293)
    at java.base/java.lang.Class.getMethod(Class.java:2106)
    at org.apache.flink.client.program.PackagedProgram.hasMainMethod(PackagedProgram.java:307)
    ... 10 more
Caused by: java.lang.ClassNotFoundException: org.apache.flink.api.connector.sink2.Sink
    at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:476)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
    at org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:64)
    at org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:65)
    at org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)

Code for reference

DataStream<String> stream = env.fromSource(getSource(parser), WatermarkStrategy.noWatermarks(),
                "event source").uid("log file event source").filter(f -> !f.isEmpty());
        readLogFile(parser, stream)
                .flatMap((value, out) -> value.lines()
                        .forEach(out::collect), TypeInformation.of(String.class))
                .map(StreamingJob::parseLine).filter(Optional::isPresent)
                .map(Optional::get, mapType)
                .map(lm -> {
                    if (lm.containsKey("msg")) {
                        parseLine(lm.get("msg").toString()).ifPresent(mf -> lm.putAll(mf));
                    }
                    return lm;
                }, mapType)
                .keyBy(m -> m.computeIfAbsent("appid", k -> "unknownapp"))
                .map(m -> newRecord(m))
                .sinkTo(getSink());

Flink 1.15 version was used.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

三生池水覆流年 2025-02-03 18:42:31

问题似乎在我下载并使用我的应用程序的1.15版本中。我可以清理安装并重新安装以正确运行它。

The problem seems to be in the 1.15 version I downloaded and ran my application with. I have clean up with installation and reinstall to run it properly.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文