To start, I'm following along the itversity "Develop Spark apps with Scala IDE" -- the instructions are for Linux, but were easy to translate to Windows -- with the following notes:
- Prereqs
 - Set up Scala & Spark yesterday.
 - Installed Scala IDE for Eclipse
 - Installed sbt
 - Installed sbteclipse plugin (note to self; on Windows the home directory is the \Users\<username> directory)
 - In the section "Create Spark Scala project using sbt"
 - I created a project folder C:\scala-projects\simple-scala-spark with
 - The step to run sbt package creates project and target folders and downloads the dependencies defined in build.sbt
 - The step to run sbt eclipse creates the project definitions needed to bring the project into eclipse
 - (note to self: These would need to be run for every new project)
 - In the section "Import into Scala IDE for Eclipse"
 - For the first step, from the Eclipse menus, choose File > Import..., then General > Existing Projects into Workspace. I set the search directory to C:\scala-projects and it found "simple-spark" just fine.
 - When creating the new Scala object, it will direct you to the src\main\scala folder as the Source Folder.
 - In the section "Run the Spark Application using Scala IDE"
 - You need to select Scala Application > New configuration in the Run Configurations window.
 - Type SimpleApp as the main class
 - In the Program Arguments, I used local c:\temp\simpleappoutput.
 - (note to self: do not create the \simpleappoutput directory in advance, otherwise SimpleApp chokes)
 
No comments:
Post a Comment