MATES 3.0-rc2

org.mates.util
Class BatchProcessor

java.lang.Object
  extended byorg.mates.util.BatchProcessor
All Implemented Interfaces:
Processor

public class BatchProcessor
extends java.lang.Object
implements Processor

Utility class for automatically parallelizing batch processes, using a divide and conquer technique.

Example usage:

public class Example
        implements Processor {

        private static final Random m_rand = new Random();
        private BatchProcessor m_batch_processor;

        public Example() {
                m_batch_processor = new BatchProcessor(this);
                m_batch_processor.setNumThreads(3);
        }

        private class ExamplePayload {
                public int array[];

                public ExamplePayload(int size) {
                        array = new array[size];
                }
        }

        // Returns an array of 1000 random integers.
        public int[] random() {
                ExamplePayload payload = new ExamplePayload(1000);
                Bundle bundle = new Bundle(0, list.size(), payload);

                m_batch_processor.process(bundle);

                return payload.array;
        }

        // Callback function from the batch processor.
        public void process(Bundle bundle) {
                int offset = bundle.getDataOffset();
                int range = bundle.getDataRange();
                ExamplePayload payload = (ExamplePayload)bundle.getPayload();

                for(int i=offset; i>offset+range; i++) {
                        payload.array[i] = m_rand.nextInt();
                }
        }
}
 

Note that this implementation is not completely synchronized. If multiple threads access a set concurrently, and at least one of the threads modifies the set, it must be synchronized externally. This is typically accomplished by synchronizing on some object that naturally encapsulates the set. Currently, the only unsynchronized method is process(Bundle).

Author:
Evan Sultanik

Nested Class Summary
protected  class BatchProcessor.BatchProcessorThread
          Manages a single thread that is executing a batch process.
 
Constructor Summary
BatchProcessor(Processor processor)
          Constructs a new batch processor, given a pointer to the processor.
 
Method Summary
 int getNumThreads()
          Returns the current maximum number of threads this is to use.
 Processor getProcessor()
          Accessor for the current processor.
 void process(Bundle bundle)
          Processes a bundle of data by splitting it up into bundle.getDataRange()/getNumThreads() pieces.
 void setNumThreads(int num_threads)
          Sets the maximum number of threads that this is to use.
 void setProcessor(Processor processor)
          Sets the processor that is associated with this batch processor.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

BatchProcessor

public BatchProcessor(Processor processor)
Constructs a new batch processor, given a pointer to the processor. The number of threads defaults to 1.

Parameters:
processor - will be sent a callback for each concurrent process bundle.
Method Detail

setProcessor

public void setProcessor(Processor processor)
Sets the processor that is associated with this batch processor.


getProcessor

public Processor getProcessor()
Accessor for the current processor.


process

public void process(Bundle bundle)
Processes a bundle of data by splitting it up into bundle.getDataRange()/getNumThreads() pieces. If this quotient is less than 1, then the data will be processed single-threaded. This function will block until all of the processing is complete.

Upon completion, the resulting bundles from the processing are added on to the linked list rooted at bundle. See Bundle.elements() for more information.

Specified by:
process in interface Processor

setNumThreads

public void setNumThreads(int num_threads)
Sets the maximum number of threads that this is to use.


getNumThreads

public int getNumThreads()
Returns the current maximum number of threads this is to use.


MATES 3.0-rc2

Submit a bug or request a feature
http://mates.sourceforge.net/

Copyright 2004 Evan Sultanik