Package org.apache.solr.update.processor

Class OpenNLPLangDetectUpdateProcessor

java.lang.Object

org.apache.solr.update.processor.UpdateRequestProcessor

org.apache.solr.update.processor.LanguageIdentifierUpdateProcessor

org.apache.solr.update.processor.OpenNLPLangDetectUpdateProcessor

All Implemented Interfaces:

Closeable, AutoCloseable, LangIdParams

public class OpenNLPLangDetectUpdateProcessor
extends LanguageIdentifierUpdateProcessor

Identifies the language of a set of input fields using Apache OpenNLP.

See "Language Detector" section of https://opennlp.apache.org/docs/1.8.3/manual/opennlp.html

Field Summary

Fields inherited from class org.apache.solr.update.processor.LanguageIdentifierUpdateProcessor

allMapFieldsSet, docIdField, enabled, enableMapping, enforceSchema, fallbackFields, fallbackValue, inputFields, langField, langPattern, langsField, langWhitelist, lcMap, mapFields, mapIndividual, mapIndividualFieldsSet, mapKeepOrig, mapLcMap, mapOverwrite, mapPattern, mapReplaceStr, maxFieldValueChars, maxTotalChars, overwrite, schema, threshold, tikaSimilarityPattern

Fields inherited from class org.apache.solr.update.processor.UpdateRequestProcessor

next

Fields inherited from interface org.apache.solr.update.processor.LangIdParams

DOCID_FIELD_DEFAULT, DOCID_LANGFIELD_DEFAULT, DOCID_LANGSFIELD_DEFAULT, DOCID_PARAM, DOCID_THRESHOLD_DEFAULT, ENFORCE_SCHEMA, FALLBACK, FALLBACK_FIELDS, FIELDS_PARAM, LANG_FIELD, LANG_WHITELIST, LANGS_FIELD, LANGUAGE_ID, LCMAP, MAP_ENABLE, MAP_FL, MAP_INDIVIDUAL, MAP_INDIVIDUAL_FL, MAP_KEEP_ORIG, MAP_LCMAP, MAP_OVERWRITE, MAP_PATTERN, MAP_PATTERN_DEFAULT, MAP_REPLACE, MAP_REPLACE_DEFAULT, MAX_FIELD_VALUE_CHARS, MAX_FIELD_VALUE_CHARS_DEFAULT, MAX_TOTAL_CHARS, MAX_TOTAL_CHARS_DEFAULT, OVERWRITE, THRESHOLD

Constructor Summary

Constructors

Constructor	Description
OpenNLPLangDetectUpdateProcessor(SolrQueryRequest req, SolrQueryResponse rsp, UpdateRequestProcessor next, opennlp.tools.langdetect.LanguageDetectorModel model)

Method Summary

Modifier and Type	Method	Description
protected List<DetectedLanguage>	detectLanguage(Reader solrDocReader)	Detects language(s) from a reader, typically based on some fields in SolrInputDocument Classes wishing to implement their own language detection module should override this method.

Methods inherited from class org.apache.solr.update.processor.LanguageIdentifierUpdateProcessor

concatFields, detectLanguage, getMappedField, isEnabled, normalizeLangCode, process, processAdd, resolveLanguage, resolveLanguage, setEnabled, solrDocReader

Methods inherited from class org.apache.solr.update.processor.UpdateRequestProcessor

close, doClose, finish, processCommit, processDelete, processMergeIndexes, processRollback

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OpenNLPLangDetectUpdateProcessor

public OpenNLPLangDetectUpdateProcessor(SolrQueryRequest req,
                                        SolrQueryResponse rsp,
                                        UpdateRequestProcessor next,
                                        opennlp.tools.langdetect.LanguageDetectorModel model)

Method Detail

detectLanguage

protected List<DetectedLanguage> detectLanguage(Reader solrDocReader)

Description copied from class: LanguageIdentifierUpdateProcessor

Detects language(s) from a reader, typically based on some fields in SolrInputDocument Classes wishing to implement their own language detection module should override this method.

Specified by:

detectLanguage in class LanguageIdentifierUpdateProcessor

Parameters:

solrDocReader - A reader serving the text from the document to detect

Returns:

List of detected language(s) according to RFC-3066