bulk-load数据时异常问题

Question

bulk-load时，想把hdfs上的数据刷到hbase中

执行：hadoop jar /$HBASE_HOME/Hbase.jar importtsv -Dimporttsv.columns=HBASE_ROW_KEY,f1 t8  test时报下列错误，请高手指点下

Warning: $HADOOP_HOME is deprecated.

 

12/11/27 17:33:11 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.3-1240972, built on 02/06/2012 10:48 GMT

12/11/27 17:33:11 INFO zookeeper.ZooKeeper: Client environment:host.name=localhost

12/11/27 17:33:11 INFO zookeeper.ZooKeeper: Client environment:java.version=1.7.0_05

12/11/27 17:33:11 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation

12/11/27 17:33:11 INFO zookeeper.ZooKeeper: Client environment:java.home=/usr/jdk/jdk7/jre

12/11/27 17:33:11 INFO zookeeper.ZooKeeper: Client environment:java.class.path=/usr/hadoop/hadoop-1.0.3/libexec/../conf:/usr/jdk/jdk7/lib/tools.jar:/usr/hadoop/hadoop-1.0.3/libexec/..:/usr/hadoop/hadoop-1.0.3/libexec/../hadoop-core-1.0.3.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/asm-3.2.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/aspectjrt-1.6.5.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/aspectjtools-1.6.5.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/commons-beanutils-1.7.0.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/commons-beanutils-core-1.8.0.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/commons-cli-1.2.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/commons-codec-1.4.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/commons-collections-3.2.1.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/commons-configuration-1.6.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/commons-daemon-1.0.1.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/commons-digester-1.8.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/commons-el-1.0.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/commons-httpclient-3.0.1.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/commons-io-2.1.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/commons-lang-2.4.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/commons-logging-1.1.1.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/commons-logging-api-1.0.4.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/commons-math-2.1.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/commons-net-1.4.1.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/core-3.1.1.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/guava-11.0.2.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/hadoop-capacity-scheduler-1.0.3.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/hadoop-fairscheduler-1.0.3.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/hadoop-thriftfs-1.0.3.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/hbase-0.94.1.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/hsqldb-1.8.0.10.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/jackson-core-asl-1.8.8.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/jackson-mapper-asl-1.8.8.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/jasper-compiler-5.5.12.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/jasper-runtime-5.5.12.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/jdeb-0.8.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/jersey-core-1.8.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/jersey-json-1.8.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/jersey-server-1.8.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/jets3t-0.6.1.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/jetty-6.1.26.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/jetty-util-6.1.26.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/jsch-0.1.42.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/junit-4.5.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/kfs-0.2.2.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/log4j-1.2.15.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/mockito-all-1.8.5.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/oro-2.0.8.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/servlet-api-2.5-20081211.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/slf4j-api-1.4.3.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/slf4j-log4j12-1.4.3.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/xmlenc-0.52.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/zookeeper-3.4.3.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/jsp-2.1/jsp-2.1.jar:/usr/hadoop/hadoop-1.0.3/libexec/../lib/jsp-2.1/jsp-api-2.1.jar

12/11/27 17:33:11 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/usr/hadoop/hadoop-1.0.3/libexec/../lib/native/Linux-i386-32

12/11/27 17:33:11 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp

12/11/27 17:33:11 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>

12/11/27 17:33:11 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux

12/11/27 17:33:11 INFO zookeeper.ZooKeeper: Client environment:os.arch=i386

12/11/27 17:33:11 INFO zookeeper.ZooKeeper: Client environment:os.version=2.6.32-279.el6.i686

12/11/27 17:33:11 INFO zookeeper.ZooKeeper: Client environment:user.name=hbase

12/11/27 17:33:11 INFO zookeeper.ZooKeeper: Client environment:user.home=/home/hbase

12/11/27 17:33:11 INFO zookeeper.ZooKeeper: Client environment:user.dir=/usr/hadoop/hadoop-1.0.3

12/11/27 17:33:11 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=180000 watcher=hconnection

12/11/27 17:33:11 INFO zookeeper.ClientCnxn: Opening socket connection to server /0:0:0:0:0:0:0:1:2181

12/11/27 17:33:11 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 19818@wonders

12/11/27 17:33:11 INFO client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.

12/11/27 17:33:11 INFO zookeeper.ClientCnxn: Socket connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session

12/11/27 17:33:11 INFO zookeeper.ClientCnxn: Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x13b40c8ce570023, negotiated timeout = 40000

Exception in thread "main" java.lang.reflect.InvocationTargetException

     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

     at java.lang.reflect.Method.invoke(Method.java:601)

     at org.apache.hadoop.hbase.mapreduce.Driver.main(Driver.java:51)

     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

     at java.lang.reflect.Method.invoke(Method.java:601)

     at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

Caused by: java.lang.NoClassDefFoundError: com/google/protobuf/Message

     at org.apache.hadoop.hbase.io.HbaseObjectWritable.<clinit>(HbaseObjectWritable.java:263)

     at org.apache.hadoop.hbase.ipc.Invocation.write(Invocation.java:138)

     at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.sendParam(HBaseClient.java:537)

     at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:899)

     at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)

     at $Proxy5.getProtocolVersion(Unknown Source)

     at org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:183)

     at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:335)

     at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:312)

     at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:364)

     at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:665)

     at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:109)

     at org.apache.hadoop.hbase.mapreduce.ImportTsv.main(ImportTsv.java:352)

     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

     at java.lang.reflect.Method.invoke(Method.java:601)

     at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)

     at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)

     ... 10 more

Caused by: java.lang.ClassNotFoundException: com.google.protobuf.Message

     at java.net.URLClassLoader$1.run(URLClassLoader.java:366)

     at java.net.URLClassLoader$1.run(URLClassLoader.java:355)

     at java.security.AccessController.doPrivileged(Native Method)

     at java.net.URLClassLoader.findClass(URLClassLoader.java:354)

     at java.lang.ClassLoader.loadClass(ClassLoader.java:423)

     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)

     at java.lang.ClassLoader.loadClass(ClassLoader.java:356)

     ... 29 more

怎么依赖com.google.protobuf.Message这个包，如何解决？谢谢

Answer 1

在hbase的安装目录下lib文件夹中找到protobuf-java-2.4.0a.jar，然后在eclipse中添加该jar文件就可以了