org.apache.solr.hadoop

Class LineRandomizerMapper

java.lang.Object

org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

org.apache.solr.hadoop.LineRandomizerMapper

public class LineRandomizerMapper
extends org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

MR Mapper that randomizing a list of URLs. Mapper input is (offset, URL) pairs. Each such pair indicates a file to index. Mapper output is (randomPosition, URL) pairs. The reducer receives these pairs sorted by randomPosition.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Mapper

org.apache.hadoop.mapreduce.Mapper.Context

Constructor Summary

Constructors

Constructor and Description
LineRandomizerMapper()

Method Summary

Methods

Modifier and Type	Method and Description
protected void	map(org.apache.hadoop.io.LongWritable key, org.apache.hadoop.io.Text value, org.apache.hadoop.mapreduce.Mapper.Context context)
protected void	setup(org.apache.hadoop.mapreduce.Mapper.Context context)

Methods inherited from class org.apache.hadoop.mapreduce.Mapper

cleanup, run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LineRandomizerMapper

public LineRandomizerMapper()

Method Detail

setup

protected void setup(org.apache.hadoop.mapreduce.Mapper.Context context)
              throws IOException,
                     InterruptedException

Overrides:

setup in class org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

Throws:

IOException

InterruptedException

map

protected void map(org.apache.hadoop.io.LongWritable key,
       org.apache.hadoop.io.Text value,
       org.apache.hadoop.mapreduce.Mapper.Context context)
            throws IOException,
                   InterruptedException

Overrides:

map in class org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

Throws:

IOException

InterruptedException