To start, I'm following along the itversity "Develop Spark apps with Scala IDE" -- the instructions are for Linux, but were easy to translate to Windows -- with the following notes:
- Prereqs
- Set up Scala & Spark yesterday.
- Installed Scala IDE for Eclipse
- Installed sbt
- Installed sbteclipse plugin (note to self; on Windows the home directory is the \Users\<username> directory)
- In the section "Create Spark Scala project using sbt"
- I created a project folder C:\scala-projects\simple-scala-spark with
- The step to run sbt package creates project and target folders and downloads the dependencies defined in build.sbt
- The step to run sbt eclipse creates the project definitions needed to bring the project into eclipse
- (note to self: These would need to be run for every new project)
- In the section "Import into Scala IDE for Eclipse"
- For the first step, from the Eclipse menus, choose File > Import..., then General > Existing Projects into Workspace. I set the search directory to C:\scala-projects and it found "simple-spark" just fine.
- When creating the new Scala object, it will direct you to the src\main\scala folder as the Source Folder.
- In the section "Run the Spark Application using Scala IDE"
- You need to select Scala Application > New configuration in the Run Configurations window.
- Type SimpleApp as the main class
- In the Program Arguments, I used local c:\temp\simpleappoutput.
- (note to self: do not create the \simpleappoutput directory in advance, otherwise SimpleApp chokes)
No comments:
Post a Comment