Hbase connection about zookeeper error

Environment : Ubuntu 14.04 , hadoop-2.2.0 , hbase-0.98.7

when i start hadoop and hbase(single node mode), both all success (I also check the website 8088 for hadoop, 60010 for hbase)

jps
4507 SecondaryNameNode
5350 HRegionServer
4197 NameNode
4795 NodeManager
3948 QuorumPeerMain
5209 HMaster
4678 ResourceManager
5831 Jps
4310 DataNode

but when i check hbase-hadoop-master-localhost.log, i found a information following

    2014-10-23 14:16:11,392 INFO  [main-SendThread(localhost:2181)] zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2014-10-23 14:16:11,426 INFO  [main-SendThread(localhost:2181)] zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:2181, initiating session

i have google lot of website for that unknown error problem, but i can't solve this problem... Following is my hadoop and hbase configuration

Hadoop :

salves content : localhost

core-site.xml

<configuration>
    <property>
         <name>fs.defaultFS</name>
         <value>hdfs://localhost:8020</value>
     </property>
</configuration>

yarn-site.xml

<configuration>
  <property>
    <name>yarn.resourcemanager.resource-tracker.address</name>
    <value>localhost:9001</value>
    <description>host is the hostname of the resource manager and 
    port is the port on which the NodeManagers contact the Resource Manager.
    </description>
  </property>

  <property>
    <name>yarn.resourcemanager.scheduler.address</name>
    <value>localhost:9002</value>
    <description>host is the hostname of the resourcemanager and port is the port
    on which the Applications in the cluster talk to the Resource Manager.
    </description>
  </property>

  <property>
    <name>yarn.resourcemanager.scheduler.class</name>
    <value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler</value>
    <description>In case you do not want to use the default scheduler</description>
  </property>

  <property>
    <name>yarn.resourcemanager.address</name>
    <value>localhost:9003</value>
    <description>the host is the hostname of the ResourceManager and the port is the port on
    which the clients can talk to the Resource Manager. </description>
  </property>

  <property>
    <name>yarn.nodemanager.local-dirs</name>
    <value></value>
    <description>the local directories used by the nodemanager</description>
  </property>

  <property>
    <name>yarn.nodemanager.address</name>
    <value>localhost:9004</value>
    <description>the nodemanagers bind to this port</description>
  </property>  

  <property>
    <name>yarn.nodemanager.resource.memory-mb</name>
    <value>10240</value>
    <description>the amount of memory on the NodeManager in GB</description>
  </property>

  <property>
    <name>yarn.nodemanager.remote-app-log-dir</name>
    <value>/app-logs</value>
    <description>directory on hdfs where the application logs are moved to </description>
  </property>

   <property>
    <name>yarn.nodemanager.log-dirs</name>
    <value></value>
    <description>the directories used by Nodemanagers as log directories</description>
  </property>

  <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
    <description>shuffle service that needs to be set for Map Reduce to run </description>
  </property>
</configuration>

Hbase:

hbase-env.sh :

..
export JAVA_HOME="/usr/lib/jvm/java-7-oracle"
..
export HBASE_MANAGES_ZK=true
..

hbase-site.xml

<configuration>
    <property>
        <name>hbase.rootdir</name>
        <value>hdfs://localhost:8020/hbase</value>
    </property>
    <property> 
        <name>hbase.cluster.distributed</name> 
        <value>true</value> 
    </property> 
    <property>
        <name>hbase.zookeeper.property.clientPort</name>
        <value>2181</value> 
    </property>
</configuration>

regionserver content : localhost

my /etc/hosts content:

127.0.0.1       localhost
#127.0.1.1      localhost

# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

I try lots of methods to solve it, but all fail, please help me to solve it, i really need to know how to solve.

Originally, i run a mapreuce program and when map 67% reduce 0%, it print out some INFO and some of INFO is following:

14/10/23 15:50:41 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=60000 watcher=org.apache.hadoop.hbase.client.HConnectionManager$ClientZKWatcher@ce1472
14/10/23 15:50:41 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
14/10/23 15:50:41 INFO zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:2181, initiating session
14/10/23 15:50:41 INFO zookeeper.ClientCnxn: Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x1493be510380007, negotiated timeout = 40000
14/10/23 15:50:43 INFO mapred.LocalJobRunner: map > sort
14/10/23 15:50:46 INFO mapred.LocalJobRunner: map > sort

then it crash.. I think program maybe in dead lock and that is what i want to solve zookeeper problem above.

If want another configuration file i set in hadoop or hbase or others, just tell me, i'll post up. thanks!

回答1:

I don't think zookeeper is your problem. You should look at your other logs for more information about your map/reduce job status. Check the datanode and namenode logs for errors along with the yarn log messages through the yarn job tracker ui.

ZooKeeper Messages

Those messages are from zookeeper trying to connect with the Zookeeper sasl client. If sasl is not configured the client will still be able to connect but the connection won't be authenticated.

Error message comes from this file ZooKeeperSaslClient.java

150                 // The user did not override the default context. It might be that they just don't intend to use SASL,
151                 // so log at INFO, not WARN, since they don't expect any SASL-related information.
152                 String msg = "Will not attempt to authenticate using SASL ";
153                 if (runtimeException != null) {
154                     msg += "(" + runtimeException + ")";
155                 } else {
156                     msg += "(unknown error)";
157                 }
158                 this.configStatus = msg;
159                 this.isSASLConfigured = false;
160             }

If you want to get rid of the error you will have to configure zookeeper to use sasl. Sorry don't have any experience with configuration sasl for zookeeper.

ZooKeeper SASL Configuration

回答2:

Add follwing properties in hbase-site.xml file

 <property>
 <name>hbase.zookeeper.quorum</name>
 <value>192.168.56.101</value>                       #this is my server ip
 </property>
 <property>
 <name>hbase.zookeeper.property.clientPort</name>
 <value>2181</value>
 </property>

restart ./start-hbase.sh

回答3:

This is how I solved it - after I tried including hbase-site.xml in classpath and giving the zookeeper quorum value as -Dhbase.zookeeper.quorum - and they did not work. I copied hbase-site.xml to the same folder as my jar and then did jar uf myjar.jar hbase-site.xml

And then I ran hadoop jar myjar.jar Blah

This fixed the problem