Scala + GridGain = easy Functional Grid Computing

This short article illustrates how easy it is to leverage JVM-based frameworks using Scala. It focuses on the GridGain framework (a powerful open source grid platform for the JVM) which, combined with Scala, provides an elegant solution for executing some functional stuff on a grid of nodes.
Let's do it in 3 basic steps:

• Scalafy a distributed version of the classical "Hello world !"
• Gridify a basic functional example: execute a function composition using GridGain
• Scalafy the Split/Reduce process and apply it to another simple "Hello World" example (a Scala translation of the Java example coming with the GridGain installation)

We assume that the GridGain framework is installed on your system (e.g. on "C:\Program Files\gridgain-2.1.0") and that the Run/Debug eclipse configurations have been properly configured :

• Set the VM parameters to -ea "-javaagent:C:\Program Files\gridgain-2.1.0\libs\aspectjweaver-1.5.3.jar"
  
  
• Add C:\Program Files\gridgain-2.1.0\config\aop\aspectj folder in the classpath:
  
  

Gridify Scala

Let's do it whith annotations (GridGain allows you to choose between using annotations or go through the corresponding API).
If you run the following code snippet: import org.gridgain.grid._ import org.gridgain.grid.gridify._ @serializable object scalafyGridGain { def main(args: Array[String]): Unit = { GridFactory.start() try { sayHello } finally GridFactory.stop(true) } @Gridify def sayHello = println("Hi Scala Dudes") } then one of the nodes of your running GridGain topology should be chosen to print out a polite message.
That's it !!

Gridify Functional stuff

The gridified method can also take functions as parameters and return type. Let's adapt the previous example in order to gridify a method that basically does a function composition:
import org.gridgain.grid._ import org.gridgain.grid.gridify._ @serializable object scalafyGridGain { def main(args: Array[String]): Unit = { GridFactory.start() try { def f = (x:Int) => 2*x def g = (x:Int) => x*x val h = composeFunctions(f,f) println(">>> f(g(5))=" + h(5)) } finally GridFactory.stop(true) } @Gridify def composeFunctions(f1: Int => Int, f2: Int => Int): (Int =>Int) = { println(">>> composing functions ...") x => f1(f2(x)) } } You should see the two traces on two different nodes of the topology, showing that the composed function is built and evaluated by two different collaborating entities.

Split/Reduce

This last example shows the Split/Reduce pattern in action. It is basically a Scala translation of the GridGain HelloWorld example:
The string "Hi Dudes and Dudettes of the Scala Language !!!" is passed as an argument for execution of the GridGain task. As an outcome, the participating nodes will print out a single word from the string and return the number of chars within the printed word. In the end, the task returns the total number of chars ("reduce" phase).
Note that we do not use the annotations here, but the dedicated APIs. First, the main routine: import org.gridgain.grid._ import java.util._ import java.io._ object GridGainApp extends Application { GridFactory.start() try { val grid = GridFactory.getGrid() // Execute Hello World task. val future = grid.execute(classOf[HelloDudesTask].asSubclass(classOf[GridTask[Any,String]]), "Hello Dudes and Dudettes of the Scala Language !!!", 5000) // or using the class name directly: //val future : GridTaskFuture[String] = grid.execute("scaladudes.HelloDudesTask", // "Hello Dudes and Dudettes of the Scala Language !!!", 5000); // Wait for task completion. println("Char count=" + future.get()) } finally GridFactory.stop(true) } Then, the GridGain job implementation: /** * The job itself: it prints the passed in argument and returns its length */ class HelloDudesJob extends GridJobAdapter[String] { override def execute() = { val wrd = getArgument() // Prints the passed in argument println(">>> Printing '" + wrd + "' on this node from grid-enabled method.") // Return number of letters in the word. wrd.length() // note: the returned value must be Serializable } } Finally, the GridGain task implementation: /** * The task: * - it creates one job per word within the passed in phrase * - it returns the total number of chars */ class HelloDudesTask extends GridTaskSplitAdapter[String, Int] { override def split(gridSize: Int, args: String) = { // Split the passed in phrase into multiple words separated by spaces val words = args.split(" ") // The method return must be a Java Collection ( java.util.Collection[_ <:GridJob] ) val jobs = new ArrayList[GridJobAdapter[String]](words.length); words.foreach(word => { val newJob = new HelloDudesJob // Every job gets its own word as an argument. newJob.addArgument(word) jobs.add(newJob) }) jobs } /** * Reduce the task: determine the total number of letters * by summing the output of each job */ override def reduce(results: List[GridJobResult]) : Int = { var total:Int = 0; val iter = results.iterator while (iter.hasNext) total += iter.next.getData[Int]; total } } That's all for today !!

Next steps

Soon I'll show a more concrete example implementing the "GridTaskSplitAdapter.result(..)" method that acts on the reduce strategy (implying a WAIT, REDUCE or FAILOVER policy) ...

To be continued ...