Sqoop import using ojdbc6 connector

2019-07-25 03:18发布

问题:

I am using sqoop to import data from oracle 11g, as i do not have the permission to put the ojdbc jar in sqoop's lib on cluster i am explicitly providing the jar using -libjars but it is throwing exception.The code I have used is : sqoop eval -libjars /root/shared_folder/ojdbc6.jar --driver oracle.jdbc.OracleDriver --connect jdbc:oracle:thin:@127.0.0.1:1521:XE --username srivastavaaman --password manager --query 'SELECT * from TestTable1'

The output that follows is : Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. 17/04/05 02:48:40 INFO sqoop.Sqoop: Running Sqoop version: 1.4.4.2.1.1.0-385 17/04/05 02:48:41 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 17/04/05 02:48:41 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time. 17/04/05 02:48:41 INFO manager.SqlManager: Using default fetchSize of 1000 17/04/05 02:48:41 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: oracle.jdbc.OracleDriver java.lang.RuntimeException: Could not load db driver class: oracle.jdbc.OracleDriver at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:797) at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52) at org.apache.sqoop.tool.EvalSqlTool.run(EvalSqlTool.java:64) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231) at org.apache.sqoop.Sqoop.main(Sqoop.java:240)

If the same ojdbc6.jar is placed in the sqoop's lib in local machine it is working properly.

回答1:

The -libjars argument is not typically used with Sqoop, but is added as part of Hadoop’s internal argument-parsing system.

Append the path of Jar file to the $HADOOP_CLASSPATH variable.

export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/root/shared_folder/ojdbc6.jar
sqoop eval --driver oracle.jdbc.OracleDriver --connect jdbc:oracle:thin:@127.0.0.1:1521:XE --username ****** --password ***** --query 'SELECT * from TestTable1'