Hadoop : how to start my first project

Question

I'm starting to work with Hadoop but I don't know where and how do it. I'm working on OS X and I follow some tutorial to install Hadoop, it's done and it's work but now I don't know what to do.

Is there an IDE to install (maybe eclipse)? I find some codes but nothing works and I don't know what I have to add in my project etc ...

Can you give me some informations or guide me to a complete tutorial ?

Answer 1

I would recommend you install the Cloudera pseudo distributed example on a virtual machine, the latest LTS Ubuntu. That way, you don't messed up with your laptop and it would be a environment closer to anything you would do in production. Have you checked vagrantup.com?

When you have it installed, you could choose on work directly on Java or chose a framework like MrJob (python) to execute some custom programs.

Best, Alvaro.

Answer 2

If you want to learn Hadoop framework then i recomend to just start with installing Cloudera QuickStart virtual machine on your OSX system provided your system has all the prerequisites:

http://www.cloudera.com/downloads/quickstart_vms/5-8.html

Cloudera QuickStart virtual machines include everything you need to try Hadoop, MapReduce, Hive, Pig, Impala, etc. and Eclipse IDE as well.

Above will do perfect if you are interested in perusing career as Hadoop Developer however, if you are interested in Hadoop systems administrator then follow the @Alvaro recommendation.

Then there is a intro to Hadoop and MapReduce course on Udacity would be a good start for beginners: https://www.udacity.com/course/intro-to-hadoop-and-mapreduce--ud617

Hadoop: The Definitive Guide By Tom White could be a great comprehensive book to refer: http://shop.oreilly.com/product/0636920033448.do