You can include the MapreduceHelper into your class to give you standard self.map and self.reduce methods. You need only implement self.map_each and self.reduce_each methods which accept a single item (istead of an arrad)

Example Usage: This example is a bit contrived.

  class MapReduceTest
    include MapreduceHelper

    def self.run
      job = Skynet::Job.new(
        :mappers          => 2,
        :reducers         => 1,
        :map_reduce_class => self,
        :map_data         => ['http://www.geni.com'.'http://www.yahoo.com','http://www.cnet.com']
      )
      results = job.run
    end

    def self.map_each(url)
      SomeUrlSlurper.gather_results(url)  # returns an array of urls of sites that link to the given url
    end

    def self.reduce_each(linked_from_url)
      SomeUrlSluper.find_text("mysite", linked_from_url)   # finds all the times "mysite" appears in the given url, which we know links to the url given in the map_data
    end
  end

  MapReduceTest.run

Takes an array of map_data, iterates over that array calling self.map_each(item) for each item in that array. Catches exceptions in each iteration and continues processing.

Takes an array of post reduce_partitioned data, iterates over that array calling self.reduce_each(item) for each item in that array. Catches exceptions in each iteration and continues processing.