org.apache.solr.hadoop.morphline

Class MorphlineMapRunner

java.lang.Object

org.apache.solr.hadoop.morphline.MorphlineMapRunner

@Beta
public final class MorphlineMapRunner
extends Object

Internal helper for MorphlineMapper and dryRun mode; This API is for *INTERNAL* use only and should not be considered public.

Field Summary

Fields

Modifier and Type	Field and Description
static String	DISABLE_FILE_OPEN Flag to disable reading of file contents if indexing just file metadata is sufficient.
static String	MORPHLINE_FIELD_PREFIX Headers, including MIME types, can also explicitly be passed by force from the CLI to Morphline, e.g: hadoop ...
static String	MORPHLINE_FILE_PARAM
static String	MORPHLINE_ID_PARAM
static String	MORPHLINE_VARIABLE_PARAM Morphline variables can be passed from the CLI to the Morphline, e.g.: hadoop ...

Constructor Summary

Constructors

Constructor and Description
MorphlineMapRunner(org.apache.hadoop.conf.Configuration configuration, DocumentLoader loader, String solrHomeDir)

Method Summary

Methods

Modifier and Type	Method and Description
void	cleanup()
protected org.kitesdk.morphline.api.Record	getRecord(PathParts parts)
void	map(String value, org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.mapreduce.Mapper.Context context) Extract content from the path specified in the value.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MORPHLINE_FILE_PARAM

public static final String MORPHLINE_FILE_PARAM

See Also:

Constant Field Values

MORPHLINE_ID_PARAM

public static final String MORPHLINE_ID_PARAM

See Also:

Constant Field Values

MORPHLINE_VARIABLE_PARAM

public static final String MORPHLINE_VARIABLE_PARAM

Morphline variables can be passed from the CLI to the Morphline, e.g.: hadoop ... -D morphlineVariable.zkHost=127.0.0.1:2181/solr

See Also:

Constant Field Values

MORPHLINE_FIELD_PREFIX

public static final String MORPHLINE_FIELD_PREFIX

Headers, including MIME types, can also explicitly be passed by force from the CLI to Morphline, e.g: hadoop ... -D morphlineField._attachment_mimetype=text/csv

See Also:

Constant Field Values

DISABLE_FILE_OPEN

public static final String DISABLE_FILE_OPEN

Flag to disable reading of file contents if indexing just file metadata is sufficient. This improves performance and confidentiality.

See Also:

Constant Field Values

Constructor Detail

MorphlineMapRunner

public MorphlineMapRunner(org.apache.hadoop.conf.Configuration configuration,
                  DocumentLoader loader,
                  String solrHomeDir)
                   throws IOException

Throws:

IOException

Method Detail

map

public void map(String value,
       org.apache.hadoop.conf.Configuration configuration,
       org.apache.hadoop.mapreduce.Mapper.Context context)
         throws IOException

Extract content from the path specified in the value. Key is useless.

Throws:

IOException

getRecord

protected org.kitesdk.morphline.api.Record getRecord(PathParts parts)

cleanup

public void cleanup()