SQOOP SQLSERVER Failed to load driver “ appropriat

Question

I downloaded sqljdbc4.jar. I'm invoking sqoop like so from the folder (where the jar is stored):

sqoop list-tables --driver com.microsoft.jdbc.sqlserver.SQLServerDriver --connect jdbc:sqlserver://localhost:1433;user=me;password=myPassword; -libjars=./sqljdbc4.jar

I'm getting the following warning & error:

13/10/25 18:38:13 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.

13/10/25 18:38:13 INFO manager.SqlManager: Using default fetchSize of 1000
13/10/25 18:38:13 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.microsoft.jdbc.sqlserver.SQLServerDriver
java.lang.RuntimeException: Could not load db driver class: com.microsoft.jdbc.sqlserver.SQLServerDriver
    at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:727)
    at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
    at org.apache.sqoop.manager.SqlManager.listTables(SqlManager.java:418)
    at org.apache.sqoop.tool.ListTablesTool.run(ListTablesTool.java:49)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
    at org.apache.sqoop.Sqoop.main(Sqoop.java:238)

UPDATE

I changed the command line to reflect the comments below, I get the same error:

sqoop list-databases -libjars=<ABSOLUTE_PATH>/jars/sqljdbc4.jar --connect jdbc:sqlserver://localhost:1433;user=me;password=password

13/10/28 17:00:33 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.microsoft.sqlserver.jdbc.SQLServerDriver
java.lang.RuntimeException: Could not load db driver class: com.microsoft.sqlserver.jdbc.SQLServerDriver
    at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:727)
    at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
    at org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:57)
    at org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:49)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
    at org.apache.sqoop.Sqoop.main(Sqoop.java:238)

When I look at the listing of sqljdbc4.jar, I do see the class in that path... Is it possible that libjars option isn't doing what I think it is supposed to do?

Answer 1

In vast majority of cases using parameter --driver is not required and even more will lead to an undesirable behaviour. I would strongly recommend dropping this argument entirely from your command line. Check out Connectors vs Drivers blog post for more details.

Also in addition you are specifying a nonexistent JDBC Driver class. The correct one is:

com.microsoft.sqlserver.jdbc.SQLServerDriver

You can see it in the official docs, whereas you are specifying

com.microsoft.jdbc.sqlserver.SQLServerDriver

Notice the different order of jdbc and sqlserver packages. This is one of the reasons why it's recommended to not use the --driver option at all.

Answer 2

You need to put sqljdbc4.jar in $SQOOP_HOME/lib and also add sqoop-1.4.4.jar or whatever version you are using along with sqljdbc4.jar to $HADOOP_HOME/lib.

I'm using Hadoop-2.2.0, so i put it inside $HADOOP_HOME/share/hadoop/common/lib directory, and use the following command to do the import:

export HCAT_HOME=/home/Kuntal/BIG_DATA/hive-0.12.0/hcatalog

(sometimes HCatlog of Hive needs to be exported or set.)

./sqoop-import --connect "jdbc:sqlserver://IP\INSTANCE;port=1433;username=USERNAME;password=PASSWORD;database=DATABASE_NAME" --table TABLE_NAME --target-dir hdfs://localhost:50315/sqoop --m 1

Sometimes you have to specify the port, otherwise default works. Hope you find it useful.

Answer 3

According to this sqoop documentation, generic options like -libjars must come before tool-specific options:

Generic Hadoop command-line arguments:
(must preceed any tool-specific arguments)
...
-libjars <comma separated list of jars> specify comma separated jar files to include in the classpath.