Package org.apache.solr.analysis
Class LowerCaseTokenizer
- java.lang.Object
-
- org.apache.lucene.util.AttributeSource
-
- org.apache.lucene.analysis.TokenStream
-
- org.apache.lucene.analysis.Tokenizer
-
- org.apache.solr.analysis.LowerCaseTokenizer
-
- All Implemented Interfaces:
Closeable
,AutoCloseable
@Deprecated public final class LowerCaseTokenizer extends org.apache.lucene.analysis.Tokenizer
Deprecated.UseLetterTokenizer
andLowerCaseFilter
LowerCaseTokenizer performs the function of LetterTokenizer and LowerCaseFilter together. It divides text at non-letters and converts them to lower case. While it is functionally equivalent to the combination of LetterTokenizer and LowerCaseFilter, there is a performance advantage to doing the two tasks at once, hence this (redundant) implementation.Note: this does a decent job for most European languages, but does a terrible job for some Asian languages, where words are not separated by spaces.
-
-
Field Summary
Fields Modifier and Type Field Description static int
DEFAULT_MAX_WORD_LEN
Deprecated.
-
Constructor Summary
Constructors Constructor Description LowerCaseTokenizer()
Deprecated.Construct a new LowerCaseTokenizer.LowerCaseTokenizer(org.apache.lucene.util.AttributeFactory factory)
Deprecated.Construct a new LowerCaseTokenizer using a givenAttributeFactory
.LowerCaseTokenizer(org.apache.lucene.util.AttributeFactory factory, int maxTokenLen)
Deprecated.Construct a new LowerCaseTokenizer using a givenAttributeFactory
.
-
Method Summary
All Methods Instance Methods Concrete Methods Deprecated Methods Modifier and Type Method Description void
end()
Deprecated.boolean
incrementToken()
Deprecated.void
reset()
Deprecated.-
Methods inherited from class org.apache.lucene.analysis.Tokenizer
close, correctOffset, setReader, setReaderTestPoint
-
Methods inherited from class org.apache.lucene.util.AttributeSource
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, endAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, removeAllAttributes, restoreState, toString
-
-
-
-
Field Detail
-
DEFAULT_MAX_WORD_LEN
public static final int DEFAULT_MAX_WORD_LEN
Deprecated.- See Also:
- Constant Field Values
-
-
Constructor Detail
-
LowerCaseTokenizer
public LowerCaseTokenizer()
Deprecated.Construct a new LowerCaseTokenizer.
-
LowerCaseTokenizer
public LowerCaseTokenizer(org.apache.lucene.util.AttributeFactory factory)
Deprecated.Construct a new LowerCaseTokenizer using a givenAttributeFactory
.- Parameters:
factory
- the attribute factory to use for thisTokenizer
-
LowerCaseTokenizer
public LowerCaseTokenizer(org.apache.lucene.util.AttributeFactory factory, int maxTokenLen)
Deprecated.Construct a new LowerCaseTokenizer using a givenAttributeFactory
.- Parameters:
factory
- the attribute factory to use for thisTokenizer
maxTokenLen
- maximum token length the tokenizer will emit. Must be greater than 0 and less than MAX_TOKEN_LENGTH_LIMIT (1024*1024)- Throws:
IllegalArgumentException
- if maxTokenLen is invalid.
-
-
Method Detail
-
incrementToken
public final boolean incrementToken() throws IOException
Deprecated.- Specified by:
incrementToken
in classorg.apache.lucene.analysis.TokenStream
- Throws:
IOException
-
end
public final void end() throws IOException
Deprecated.- Overrides:
end
in classorg.apache.lucene.analysis.TokenStream
- Throws:
IOException
-
reset
public void reset() throws IOException
Deprecated.- Overrides:
reset
in classorg.apache.lucene.analysis.Tokenizer
- Throws:
IOException
-
-