I am following this tutorial to install Hadoop in my computer. After finishing the installation, when I try to launch Hadoop using this command ./start-dfs.sh, it returns me the following:
U:sbin U$ ./start-dfs.sh
Starting namenodes on [localhost]
localhost: U@localhost: Permission denied (publickey,password,keyboard-interactive).
Starting datanodes
localhost: U@localhost: Permission denied (publickey,password,keyboard-interactive).
Starting secondary namenodes [U.local]
U.local: U@pc.local: Permission denied (publickey,password,keyboard-interactive).
2018-02-25 14:52:15,505 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
I tried un-installing and installing it several times to re-check if I missed something but still I keep getting this error at the end. After looking in some online forums I came to find that the last warning : WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform is not a big deal because it gives the error when we run Hadoop in a 64 bit machine. Will you please let me know what the other two error mean and how to fix them ? I have tried many solutions posted in the internet.
Problem is when you are trying to ssh to a server (in this case localhost) it tries to authenticate you using your credential. And stores that info. But here password-less authentication is not configured, so each time you try to ssh, it will ask you for your password, which is a problem if machines try to communicate with each other using ssh. So to setup passwordless ssh, we need to add user machine's public key to server machines ~/.ssh/authorized_keys file. In this case, both the system are same machines.
So Long story short run the following command.
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
Proceed with the following steps:
Generate new keygen.
ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
Register key gen:
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
I did the following 3 steps to create the password less login
- ssh-keygen -t rsa (Press enter for each line)
- cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
- chmod og-wx ~/.ssh/authorized_keys
Those who are still struggling with this error, my answer could help them.
If you have done everything right and have already added keys in authorized_keys as well, then all you need to do is to remove your id_rsa and id_rsa.pub (whatever names you have used for the keypair file) and empty your authorized_keys, in short just take a rollback, because you might have given a password while generating rsa key.
So just do one thing, create again the RSA key by:
ssh-keygen -t rsa
Give the name of the file(when prompted): < your_filename >
And then do not give it a passphrase and rather just simply hit Enter, i.e, leave it blank and thus you will never see the permission denied error (It worked in my case.)
cd hadoop/etc/hadoop
nano hadoop-env.sh
And paste this line in hadoop-env.sh
export HADOOP_SSH_OPTS="-p 22"
