Installing Apache Spark using yum

问题:

I am in the process of installing spark in my organization's HDP box. I run yum install spark and it installs Spark 1.4.1. How do I install Spark 2.0? Please help!

回答1:

Spark 2 is supported (as a technical preview) in HDP 2.5. You can get the specific HDP 2.5 repo added to your yum repo directory and then install the same. Spark 1.6.2 is the version default in HDP 2.5.

wget http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.5.0.0/hdp.repo

sudo cp hdp.repo /etc/yum.repos.d/hdp.repo

sudo yum install spark2-master

or sudo yum install spark2 (also seems to be doing the same when i tried)

see whats new in HDP 2.5 http://hortonworks.com/products/data-center/hdp/

For full list of repos see https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/bk_release-notes/content/download-links-250.html