sqoop 从mysql数据迁移到hdfs中

Question

我在连接数据库的时候显示都能连接上，list表的时候也能显示所有的表。但是我在导入数据的时候走到：

guoyachao@master :~/software/hbase-0.94.11$ sqoop list-tables --connect  jdbc:mysql://192.168.1.100:3306/test --username root --password root
Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
14/08/11 06:59:34 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
14/08/11 06:59:34 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
test_hbase
guoyachao@master :~/software/hbase-0.94.11$ sqoop import --connect  jdbc:mysql://192.168.1.100:3306/test --username root --password root --table test_hbase -m 1
Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
14/08/11 07:02:11 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
14/08/11 07:02:11 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
14/08/11 07:02:11 INFO tool.CodeGenTool: Beginning code generation
14/08/11 07:02:12 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `test_hbase` AS t LIMIT 1
14/08/11 07:02:13 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `test_hbase` AS t LIMIT 1
14/08/11 07:02:13 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /home/guoyachao/software/hadoop-1.0.4
注意：/tmp/sqoop-guoyachao/compile/2acdd51525ffead97c25d28580b4b077/test_hbase.java 使用或覆盖了已过时的 API。
注意：要了解详细信息，请使用 -Xlint:deprecation 重新编译。
14/08/11 07:02:19 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-guoyachao/compile/2acdd51525ffead97c25d28580b4b077/test_hbase.jar
14/08/11 07:02:20 WARN manager.MySQLManager: It looks like you are importing from mysql.
14/08/11 07:02:20 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
14/08/11 07:02:20 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
14/08/11 07:02:20 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
14/08/11 07:02:20 INFO mapreduce.ImportJobBase: Beginning import of test_hbase

Answer 1

可以连接数据库，但是导入的时候出错，求解。。。谢谢

Answer 2

你好，你用的是那个版本的，我从mysql导入数据时出错SqlManager: Error reading from database: java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@6a78afa0 is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries.

java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@6a78afa0 is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries.

请问你可以分享你的环境搭建的步骤吗？

Answer 3

查看拉hadoop的日志，发现我的hadoop处于安全模式，造成不能修改和新增，所以停到这里不走啦，最后在hdfs-site.xml文件下配置

        <property>

                <name>dfs.safemode.threshold.pct</name>

                <value>0</value>

<final>true</final>

        </property>就OK啦