在使用rowkey中阅读hbase时问题

发布于 2025-02-09 09:57:24 字数 2296 浏览 1 评论 0原文

Rowkey的东西读取HBase

Error:
Exception in thread "main" java.lang.IncompatibleClassChangeError: Class org.apache.hadoop.hbase.client.RegionCoprocessorRpcChannel does not implement the requested interface com.google.protobuf.BlockingRpcChannel
    at org.apache.hadoop.hbase.protobuf.generated.AuthenticationProtos$AuthenticationService$BlockingStub.getAuthenticationToken(AuthenticationProtos.java:4512)
    at org.apache.hadoop.hbase.security.token.TokenUtil.obtainToken(TokenUtil.java:97)
    at org.apache.hadoop.hbase.security.token.TokenUtil$1.run(TokenUtil.java:137)
    at org.apache.hadoop.hbase.security.token.TokenUtil$1.run(TokenUtil.java:134)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
    at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:313)
    at org.apache.hadoop.hbase.security.token.TokenUtil.obtainToken(TokenUtil.java:134)
    at org.apache.hadoop.hbase.security.token.TokenUtil.addTokenForJob(TokenUtil.java:288)
    at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initCredentials(TableMapReduceUtil.java:533)
    at org.apache.hadoop.hbase.spark.HBaseContext.<init>(HBaseContext.scala:71)


Error thrown while creating HbaseContext in below line of code:

    var hb:org.apache.hadoop.hbase.spark.HBaseContext =new HBaseContext(sc, HbaseConfig)

目的:使用我试图解决上述问题的 : 我相信上述问题是因为命名的普通类:jar文件中的authentication protos HBase Shaded-Client&amp;造成冲突的HBase-protocol。 与罐子冲突的Maven依赖性:

    <dependency>
            <artifactId>hbase-shaded-client</artifactId>
            <groupId>org.apache.hbase</groupId>
            <version>2.1.0-cdh6.3.4</version>
        </dependency>

<dependency>
            <artifactId>hbase-protocol</artifactId>
            <groupId>org.apache.hbase</groupId>
            <version>2.1.0-cdh6.3.4</version>
        </dependency>

如果HBase Shaded-Client被排除在pom.xml中,则问题解决方案。

但是,就我而言,我不能像已经在项目中所说的那样永久删除HBase Shaded-Client依赖。 我的问题是“这两个依赖关系可以一起使用HBase-protocol和HBase Shaded-Client吗?”

Aim: To read Hbase using Spark using rowkey

Error:
Exception in thread "main" java.lang.IncompatibleClassChangeError: Class org.apache.hadoop.hbase.client.RegionCoprocessorRpcChannel does not implement the requested interface com.google.protobuf.BlockingRpcChannel
    at org.apache.hadoop.hbase.protobuf.generated.AuthenticationProtos$AuthenticationService$BlockingStub.getAuthenticationToken(AuthenticationProtos.java:4512)
    at org.apache.hadoop.hbase.security.token.TokenUtil.obtainToken(TokenUtil.java:97)
    at org.apache.hadoop.hbase.security.token.TokenUtil$1.run(TokenUtil.java:137)
    at org.apache.hadoop.hbase.security.token.TokenUtil$1.run(TokenUtil.java:134)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
    at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:313)
    at org.apache.hadoop.hbase.security.token.TokenUtil.obtainToken(TokenUtil.java:134)
    at org.apache.hadoop.hbase.security.token.TokenUtil.addTokenForJob(TokenUtil.java:288)
    at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initCredentials(TableMapReduceUtil.java:533)
    at org.apache.hadoop.hbase.spark.HBaseContext.<init>(HBaseContext.scala:71)


Error thrown while creating HbaseContext in below line of code:

    var hb:org.apache.hadoop.hbase.spark.HBaseContext =new HBaseContext(sc, HbaseConfig)

Things that I tried to resolve above issue:
i believe above issue is because of common class named :AuthenticationProtos in jar files
hbase-shaded-client & hbase-protocol that creates a conflict.
maven dependencies for conflicting jars:

    <dependency>
            <artifactId>hbase-shaded-client</artifactId>
            <groupId>org.apache.hbase</groupId>
            <version>2.1.0-cdh6.3.4</version>
        </dependency>

<dependency>
            <artifactId>hbase-protocol</artifactId>
            <groupId>org.apache.hbase</groupId>
            <version>2.1.0-cdh6.3.4</version>
        </dependency>

if hbase-shaded-client is excluded from pom.xml , issue resolves .

However in my case i cant remove hbase-shaded-client dependency permanently as its already declared in project.
my question is "Can these 2 dependencies hbase-protocol and hbase-shaded-client be used together ?"

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。
列表为空,暂无数据
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文