stkim1
Posts: 5
Joined: Thu Jun 04, 2015 2:24 am
Contact: Website

Apache Spark/Hadoop on Raspberry Pi 2

Thu Jun 04, 2015 10:47 pm

Hi,

I've been running Apache Spark/Hadoop on six RPI 2 for about three months so far, and it serves me quite well. The cluster is a bit more of investment than six instances of cloud node, but it runs deadly quiet and works quite well. On top of that, it runs on Raspberry PI for God's sake!

Image

You can start running Apache Spark/Hadoop cluster with two RPI 2, and one PC with following items.

Oracle Java 8
Scala 2.11
Hadoop 2.6
Spark 1.3.1
I am building pocket-sized BigData clusters. Come check out my blog.

pegasis
Posts: 87
Joined: Sat Mar 02, 2013 2:47 pm

Re: Apache Spark/Hadoop on Raspberry Pi 2

Thu Aug 20, 2015 12:47 am

what were the steps to install and set it up?

stkim1
Posts: 5
Joined: Thu Jun 04, 2015 2:24 am
Contact: Website

Re: Apache Spark/Hadoop on Raspberry Pi 2

Wed Oct 21, 2015 11:43 am

pegasis wrote:what were the steps to install and set it up?
@pegasis,

Sorry for late reply.

In order to install you'd need go through few steps; assign ip address, setup login credential, install java and Hadoop packages, then finally install spark to run.

I can write it out on the forum but I'd rather give you an utility to do so in a few mouse clicks. In fact, I am working on it and here's a screen shot.

Image
full size
I am building pocket-sized BigData clusters. Come check out my blog.

stkim1
Posts: 5
Joined: Thu Jun 04, 2015 2:24 am
Contact: Website

Re: Apache Spark/Hadoop on Raspberry Pi 2

Sat Dec 05, 2015 12:04 pm

I've built an OSX application for easy deployment. It's now updated to support Spark 1.5.2.

Check out the blog and Youtube. :)

Apache Spark 1.5.2 on Raspberry PI 2 cluster

Youtube
I am building pocket-sized BigData clusters. Come check out my blog.

Return to “Networking and servers”