Spark Setting up Spark Dev Environment using SBT and Eclipse

############################# Video Source: www.youtube.com/watch?v=qrPjAyIapFY

Setting up Apache Spark Development Environment using SBT and Eclipse • Spark: Setting up Dev Environment (Spark : SBT + Eclipse) • -------------------------------------------------- • STEP 1(Prereq): Install /Setup Software • Install Scala: http://www.scala-lang.org/download/ • Install SBT: http://www.scala-sbt.org/download.html • Install Eclipse (/Scala IDE): http://scala-ide.org/download/sdk.html • Install Spark • STEP 2: • Create folder structure and add a build.sbt file • mkdir -p src/{main,test}/{java,resources,scala} • mkdir lib project target • touch build.sbt • touch project/plugins.sbt • Create plugins.sbt in project folder (for sbt eclipse and sbt assembly) • addSbtPlugin( com.eed3si9n % sbt-assembly % 0.14.3 ) • addSbtPlugin( com.typesafe.sbteclipse % sbteclipse-plugin % 4.0.0 ) • STEP 3: • Run sbt, sbt eclipse • STEP 4: • Import project into Workspace • Add code for Spark

#############################

New on site