当前位置： 文江博客 话题详情

连接并持久化到 HBase

发布于 2024-12-04 09:29:03 字数 9201 浏览 1 评论 0 原文

我只是尝试使用 java 客户端连接到 hbase，它是 cloudera-vm 的一部分。

（192.168.56.102是虚拟机的inet ip）

我使用仅主机网络设置的虚拟盒。

所以我可以访问hbase master的webUI @ http://192.168.56.102:60010/master.jsp< /a>

我的 java 客户端（在虚拟机本身上运行良好）也建立了与192.168.56.102:2181

但是当它调用 getMaster 时，我得到 连接被拒绝 查看日志：

11/09/14 11:19:30 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=192.168.56.102:2181 sessionTimeout=180000 watcher=hconnection
11/09/14 11:19:30 INFO zookeeper.ClientCnxn: Opening socket connection to server /192.168.56.102:2181
11/09/14 11:19:30 INFO zookeeper.ClientCnxn: Socket connection established to cloudera-vm/192.168.56.102:2181, initiating session
11/09/14 11:19:30 INFO zookeeper.ClientCnxn: Session establishment complete on server cloudera-vm/192.168.56.102:2181, sessionid = 0x13267157f930009, negotiated timeout = 40000
11/09/14 11:19:32 INFO client.HConnectionManager$HConnectionImplementation: getMaster attempt 0 of 10 failed; retrying after sleep of 1000
java.net.ConnectException: Connection refused: no further information
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
    at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
    at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:404)
    at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:311)
    at org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:865)
    at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:732)
    at org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
    at $Proxy9.getProtocolVersion(Unknown Source)
    at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419)
    at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393)
    at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:358)
    at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:90)
    at org.datanucleus.store.hbase.HBaseUtils$1.run(HBaseUtils.java:432)
    at java.security.AccessController.doPrivileged(Native Method)
    at org.datanucleus.store.hbase.HBaseUtils.createSchemaForClass(HBaseUtils.java:428)
    at org.datanucleus.store.hbase.HBaseMetaDataListener.loaded(HBaseMetaDataListener.java:74)
    at org.datanucleus.store.hbase.HBaseStoreManager.<init>(HBaseStoreManager.java:76)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
    at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:677)
    at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:287)
    at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:453)
    at org.datanucleus.NucleusContext.initialise(NucleusContext.java:264)
    at org.datanucleus.api.jpa.JPAEntityManagerFactory.initialiseNucleusContext(JPAEntityManagerFactory.java:746)
    at org.datanucleus.api.jpa.JPAEntityManagerFactory.<init>(JPAEntityManagerFactory.java:422)
    at org.datanucleus.api.jpa.PersistenceProviderImpl.createEntityManagerFactory(PersistenceProviderImpl.java:91)
    at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:150)
    at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:70)
    at de.syrtec.jpa2hbase.start.TestDAO.main(TestDAO.java:13)

hbase-site.xml:

<configuration>
    <property>
        <name>hbase.zookeeper.quorum</name>
        <value>192.168.56.102</value>
    </property>
    <property>
        <name>hbase.zookeeper.property.clientPort</name>
        <value>2181</value>
    </property>
    <property>
        <name>hbase.master</name>
        <value>192.168.56.102:60010</value>
    </property>
</configuration>

persistence.xml:

<persistence
    xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd"
    version="1.0">

    <persistence-unit name="hbase-addressbook"
        transaction-type="RESOURCE_LOCAL">

        <properties>
            <property name="datanucleus.ConnectionURL" value="hbase:192.168.56.102:60010" />
            <property name="datanucleus.ConnectionUserName" value="" />
            <property name="datanucleus.ConnectionPassword" value="" />
            <property name="datanucleus.autoCreateSchema" value="true" />
            <property name="datanucleus.validateTables" value="false" />
            <property name="datanucleus.Optimistic" value="false" />
            <property name="datanucleus.validateConstraints" value="false" />
        </properties>
    </persistence-unit>
</persistence>

TestDAO.java:

public class TestDAO {
    public static void main(String[] args) {
        EntityManagerFactory emf = Persistence.createEntityManagerFactory("hbase-addressbook");
        EntityManager em = emf.createEntityManager();
        EntityTransaction tx = null;

        Account a2 = new Account("myPre3", "mySur3", 2, new Login("e", "f"));

        tx = em.getTransaction();
        tx.begin(); 
            em.persist(a2);
        tx.commit();
    }
}

只是添加VM 上运行的网络服务 (netstat -ntpl)：

tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      563/sshd        
tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN      909/cupsd       
tcp        0      0 0.0.0.0:8088            0.0.0.0:*               LISTEN      2004/python2.6  
tcp        0      0 127.0.0.1:8001          0.0.0.0:*               LISTEN      2345/python2.6  
tcp6       0      0 :::39087                :::*                    LISTEN      1327/java       
tcp6       0      0 :::54162                :::*                    LISTEN      1411/java       
tcp6       0      0 127.0.0.1:8020          :::*                    LISTEN      1411/java       
tcp6       0      0 127.0.0.1:8021          :::*                    LISTEN      1327/java       
tcp6       0      0 :::50070                :::*                    LISTEN      1411/java       
tcp6       0      0 :::22                   :::*                    LISTEN      563/sshd        
tcp6       0      0 ::1:631                 :::*                    LISTEN      909/cupsd       
tcp6       0      0 :::11000                :::*                    LISTEN      2077/java       
tcp6       0      0 :::43738                :::*                    LISTEN      858/java        
tcp6       0      0 :::50010                :::*                    LISTEN      858/java        
tcp6       0      0 :::51258                :::*                    LISTEN      858/java        
tcp6       0      0 :::50075                :::*                    LISTEN      858/java        
tcp6       0      0 :::60030                :::*                    LISTEN      1726/java       
tcp6       0      0 127.0.0.1:44446         :::*                    LISTEN      1642/java       
tcp6       0      0 127.0.1.1:41246         :::*                    LISTEN      1726/java       
tcp6       0      0 :::8002                 :::*                    LISTEN      2009/java       
tcp6       0      0 :::8003                 :::*                    LISTEN      2009/java       
tcp6       0      0 127.0.1.1:58915         :::*                    LISTEN      1726/java       
tcp6       0      0 :::59203                :::*                    LISTEN      1491/java       
tcp6       0      0 :::50020                :::*                    LISTEN      858/java        
tcp6       0      0 127.0.0.1:8005          :::*                    LISTEN      2077/java       
tcp6       0      0 :::2181                 :::*                    LISTEN      1726/java       
tcp6       0      0 :::9290                 :::*                    LISTEN      1327/java       
tcp6       0      0 :::60010                :::*                    LISTEN      1726/java       
tcp6       0      0 :::50090                :::*                    LISTEN      1491/java       
tcp6       0      0 :::10090                :::*                    LISTEN      1411/java       
tcp6       0      0 :::50060                :::*                    LISTEN      1642/java       
tcp6       0      0 :::50030                :::*                    LISTEN      1327/java
原文

I just tried to connect to hbase which is part of the cloudera-vm using a java client.

(192.168.56.102 is the inet ip of the vm)

I use virtual box with host only network setting.

So I can access the webUI of the hbase master @ http://192.168.56.102:60010/master.jsp

Also my java client (worked well on the vm itself) established connection to 192.168.56.102:2181

But when it calls getMaster I get connection refused see log:

11/09/14 11:19:30 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=192.168.56.102:2181 sessionTimeout=180000 watcher=hconnection
11/09/14 11:19:30 INFO zookeeper.ClientCnxn: Opening socket connection to server /192.168.56.102:2181
11/09/14 11:19:30 INFO zookeeper.ClientCnxn: Socket connection established to cloudera-vm/192.168.56.102:2181, initiating session
11/09/14 11:19:30 INFO zookeeper.ClientCnxn: Session establishment complete on server cloudera-vm/192.168.56.102:2181, sessionid = 0x13267157f930009, negotiated timeout = 40000
11/09/14 11:19:32 INFO client.HConnectionManager$HConnectionImplementation: getMaster attempt 0 of 10 failed; retrying after sleep of 1000
java.net.ConnectException: Connection refused: no further information
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
    at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
    at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:404)
    at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:311)
    at org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:865)
    at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:732)
    at org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
    at $Proxy9.getProtocolVersion(Unknown Source)
    at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419)
    at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393)
    at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:358)
    at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:90)
    at org.datanucleus.store.hbase.HBaseUtils$1.run(HBaseUtils.java:432)
    at java.security.AccessController.doPrivileged(Native Method)
    at org.datanucleus.store.hbase.HBaseUtils.createSchemaForClass(HBaseUtils.java:428)
    at org.datanucleus.store.hbase.HBaseMetaDataListener.loaded(HBaseMetaDataListener.java:74)
    at org.datanucleus.store.hbase.HBaseStoreManager.<init>(HBaseStoreManager.java:76)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
    at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:677)
    at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:287)
    at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:453)
    at org.datanucleus.NucleusContext.initialise(NucleusContext.java:264)
    at org.datanucleus.api.jpa.JPAEntityManagerFactory.initialiseNucleusContext(JPAEntityManagerFactory.java:746)
    at org.datanucleus.api.jpa.JPAEntityManagerFactory.<init>(JPAEntityManagerFactory.java:422)
    at org.datanucleus.api.jpa.PersistenceProviderImpl.createEntityManagerFactory(PersistenceProviderImpl.java:91)
    at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:150)
    at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:70)
    at de.syrtec.jpa2hbase.start.TestDAO.main(TestDAO.java:13)

hbase-site.xml:

<configuration>
    <property>
        <name>hbase.zookeeper.quorum</name>
        <value>192.168.56.102</value>
    </property>
    <property>
        <name>hbase.zookeeper.property.clientPort</name>
        <value>2181</value>
    </property>
    <property>
        <name>hbase.master</name>
        <value>192.168.56.102:60010</value>
    </property>
</configuration>

persistence.xml:

<persistence
    xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd"
    version="1.0">

    <persistence-unit name="hbase-addressbook"
        transaction-type="RESOURCE_LOCAL">

        <properties>
            <property name="datanucleus.ConnectionURL" value="hbase:192.168.56.102:60010" />
            <property name="datanucleus.ConnectionUserName" value="" />
            <property name="datanucleus.ConnectionPassword" value="" />
            <property name="datanucleus.autoCreateSchema" value="true" />
            <property name="datanucleus.validateTables" value="false" />
            <property name="datanucleus.Optimistic" value="false" />
            <property name="datanucleus.validateConstraints" value="false" />
        </properties>
    </persistence-unit>
</persistence>

TestDAO.java:

public class TestDAO {
    public static void main(String[] args) {
        EntityManagerFactory emf = Persistence.createEntityManagerFactory("hbase-addressbook");
        EntityManager em = emf.createEntityManager();
        EntityTransaction tx = null;

        Account a2 = new Account("myPre3", "mySur3", 2, new Login("e", "f"));

        tx = em.getTransaction();
        tx.begin(); 
            em.persist(a2);
        tx.commit();
    }
}

just an addition of the network services running on the VM (netstat -ntpl):

tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      563/sshd        
tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN      909/cupsd       
tcp        0      0 0.0.0.0:8088            0.0.0.0:*               LISTEN      2004/python2.6  
tcp        0      0 127.0.0.1:8001          0.0.0.0:*               LISTEN      2345/python2.6  
tcp6       0      0 :::39087                :::*                    LISTEN      1327/java       
tcp6       0      0 :::54162                :::*                    LISTEN      1411/java       
tcp6       0      0 127.0.0.1:8020          :::*                    LISTEN      1411/java       
tcp6       0      0 127.0.0.1:8021          :::*                    LISTEN      1327/java       
tcp6       0      0 :::50070                :::*                    LISTEN      1411/java       
tcp6       0      0 :::22                   :::*                    LISTEN      563/sshd        
tcp6       0      0 ::1:631                 :::*                    LISTEN      909/cupsd       
tcp6       0      0 :::11000                :::*                    LISTEN      2077/java       
tcp6       0      0 :::43738                :::*                    LISTEN      858/java        
tcp6       0      0 :::50010                :::*                    LISTEN      858/java        
tcp6       0      0 :::51258                :::*                    LISTEN      858/java        
tcp6       0      0 :::50075                :::*                    LISTEN      858/java        
tcp6       0      0 :::60030                :::*                    LISTEN      1726/java       
tcp6       0      0 127.0.0.1:44446         :::*                    LISTEN      1642/java       
tcp6       0      0 127.0.1.1:41246         :::*                    LISTEN      1726/java       
tcp6       0      0 :::8002                 :::*                    LISTEN      2009/java       
tcp6       0      0 :::8003                 :::*                    LISTEN      2009/java       
tcp6       0      0 127.0.1.1:58915         :::*                    LISTEN      1726/java       
tcp6       0      0 :::59203                :::*                    LISTEN      1491/java       
tcp6       0      0 :::50020                :::*                    LISTEN      858/java        
tcp6       0      0 127.0.0.1:8005          :::*                    LISTEN      2077/java       
tcp6       0      0 :::2181                 :::*                    LISTEN      1726/java       
tcp6       0      0 :::9290                 :::*                    LISTEN      1327/java       
tcp6       0      0 :::60010                :::*                    LISTEN      1726/java       
tcp6       0      0 :::50090                :::*                    LISTEN      1491/java       
tcp6       0      0 :::10090                :::*                    LISTEN      1411/java       
tcp6       0      0 :::50060                :::*                    LISTEN      1642/java       
tcp6       0      0 :::50030                :::*                    LISTEN      1327/java

如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论， 你可以免费 注册 一个本站的账号。

评论（1）

好久不见√ 2024-12-11 09:29:03

经过许多天乏味的研究，我可以提供一个解决方案..可以让其他人免于很多麻烦..

问题的根源是Ubuntu的IPV6支持（我在这里使用）。
但仅仅在网络管理中停用 IPV6 还不够。

相反，您必须将以下内容附加到文件中： /etc/sysctl.conf

#disable ipv6
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1

重新启动后，您应该通过以下方式验证 IPV6 是否确实关闭：

cat /proc/sys/net/ipv6/conf/all/disable_ipv6

(0 = IPV6 on ; 1 = IPV6 off)

非常感谢

http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/#disabling-ipv6

这给了我关键的提示。

After many tedious days of research I can provide a solution..could save others from a lot of headaches..

The root of the problem is the IPV6 support of Ubuntu (I'm using here).
But just deactivating IPV6 in the network administration wasn't enough.

Instead you have to append the following to the file: /etc/sysctl.conf 

#disable ipv6
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1

After reboot you should validate that IPV6 is really off by:

cat /proc/sys/net/ipv6/conf/all/disable_ipv6

(0 = IPV6 on ; 1 = IPV6 off)

Thanks a lot to

http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/#disabling-ipv6

which gave me the crucial hint.

回复 原文
~没有更多了~

关于作者

风月客

暂无简介

0 文章
0 评论
23 人气
发私信

相关话题

更多

热门标签

操作系统 程序设计 IT运维 Linux系统管理 JavaScript 服务器应用 solaris C/C++ PHP Shell BSD Vue.js aix Oracle Python HTML 系统管理 HTML5 CSS 前端
更多

推荐作者

已经忘了多久

文章 0 评论 0

15867725375

文章 0 评论 0

LonelySnow

文章 0 评论 0

走过海棠暮

文章 0 评论 0

轻许诺言

文章 0 评论 0

信馬由缰

文章 0 评论 0

更多

友情链接

文江博客
    我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站，即表示您同意使用 Cookies 和您的相关数据。
    原文