Package org.apache.solr.analysis

Class LowerCaseTokenizer

  • All Implemented Interfaces:
    Closeable, AutoCloseable
    @Deprecated
public final class LowerCaseTokenizer
extends org.apache.lucene.analysis.Tokenizer
    Deprecated.
    Use LetterTokenizer and LowerCaseFilter
    LowerCaseTokenizer performs the function of LetterTokenizer and LowerCaseFilter together. It divides text at non-letters and converts them to lower case. While it is functionally equivalent to the combination of LetterTokenizer and LowerCaseFilter, there is a performance advantage to doing the two tasks at once, hence this (redundant) implementation.

    Note: this does a decent job for most European languages, but does a terrible job for some Asian languages, where words are not separated by spaces.

    • Nested Class Summary

      • Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource

        org.apache.lucene.util.AttributeSource.State

    • Field Summary

      Fields 
      Modifier and Type Field Description
      static int DEFAULT_MAX_WORD_LEN
      Deprecated.
       

      • Fields inherited from class org.apache.lucene.analysis.Tokenizer

        input

      • Fields inherited from class org.apache.lucene.analysis.TokenStream

        DEFAULT_TOKEN_ATTRIBUTE_FACTORY

    • Constructor Summary

      Constructors 
      Constructor Description
      LowerCaseTokenizer()
      Deprecated.
      Construct a new LowerCaseTokenizer.
      LowerCaseTokenizer​(org.apache.lucene.util.AttributeFactory factory)
      Deprecated.
      Construct a new LowerCaseTokenizer using a given AttributeFactory.
      LowerCaseTokenizer​(org.apache.lucene.util.AttributeFactory factory, int maxTokenLen)
      Deprecated.
      Construct a new LowerCaseTokenizer using a given AttributeFactory.

    • Method Summary

      All Methods Instance Methods Concrete Methods Deprecated Methods 
      Modifier and Type Method Description
      void end()
      Deprecated.
       
      boolean incrementToken()
      Deprecated.
       
      void reset()
      Deprecated.
       

      • Methods inherited from class org.apache.lucene.analysis.Tokenizer

        close, correctOffset, setReader

      • Methods inherited from class org.apache.lucene.util.AttributeSource

        addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, endAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, removeAllAttributes, restoreState, toString

    • Field Detail

      • DEFAULT_MAX_WORD_LEN

        public static final int DEFAULT_MAX_WORD_LEN
        Deprecated.
        See Also:
        Constant Field Values

    • Constructor Detail

      • LowerCaseTokenizer

        public LowerCaseTokenizer()
        Deprecated.
        Construct a new LowerCaseTokenizer.

      • LowerCaseTokenizer

        public LowerCaseTokenizer​(org.apache.lucene.util.AttributeFactory factory)
        Deprecated.
        Construct a new LowerCaseTokenizer using a given AttributeFactory.
        Parameters:
        factory - the attribute factory to use for this Tokenizer

      • LowerCaseTokenizer

        public LowerCaseTokenizer​(org.apache.lucene.util.AttributeFactory factory,
                          int maxTokenLen)
        Deprecated.
        Construct a new LowerCaseTokenizer using a given AttributeFactory.
        Parameters:
        factory - the attribute factory to use for this Tokenizer
        maxTokenLen - maximum token length the tokenizer will emit. Must be greater than 0 and less than MAX_TOKEN_LENGTH_LIMIT (1024*1024)
        Throws:
        IllegalArgumentException - if maxTokenLen is invalid.

    • Method Detail

      • incrementToken

        public final boolean incrementToken()
                             throws IOException
        Deprecated.
        Specified by:
        incrementToken in class org.apache.lucene.analysis.TokenStream
        Throws:
        IOException

      • end

        public final void end()
               throws IOException
        Deprecated.
        Overrides:
        end in class org.apache.lucene.analysis.TokenStream
        Throws:
        IOException

      • reset

        public void reset()
           throws IOException
        Deprecated.
        Overrides:
        reset in class org.apache.lucene.analysis.Tokenizer
        Throws:
        IOException