Class TokenizerChain

  • All Implemented Interfaces:
    Closeable, AutoCloseable

    public final class TokenizerChain
    extends SolrAnalyzer
    An analyzer that uses a tokenizer and a list of token filters to create a TokenStream.

    It should probably be replaced with CustomAnalyzer.

    Since:
    3.1
    • Nested Class Summary

      • Nested classes/interfaces inherited from class org.apache.lucene.analysis.Analyzer

        org.apache.lucene.analysis.Analyzer.ReuseStrategy, org.apache.lucene.analysis.Analyzer.TokenStreamComponents
    • Field Summary

      • Fields inherited from class org.apache.lucene.analysis.Analyzer

        GLOBAL_REUSE_STRATEGY, PER_FIELD_REUSE_STRATEGY
    • Constructor Summary

      Constructors 
      Constructor Description
      TokenizerChain​(org.apache.lucene.analysis.CharFilterFactory[] charFilters, org.apache.lucene.analysis.TokenizerFactory tokenizer, org.apache.lucene.analysis.TokenFilterFactory[] filters)
      Creates a new TokenizerChain.
      TokenizerChain​(org.apache.lucene.analysis.custom.CustomAnalyzer customAnalyzer)
      Copy from CustomAnalyzer.
      TokenizerChain​(org.apache.lucene.analysis.TokenizerFactory tokenizer, org.apache.lucene.analysis.TokenFilterFactory[] filters)
      Creates a new TokenizerChain w/o any CharFilterFactories.
    • Constructor Detail

      • TokenizerChain

        public TokenizerChain​(org.apache.lucene.analysis.custom.CustomAnalyzer customAnalyzer)
        Copy from CustomAnalyzer.
      • TokenizerChain

        public TokenizerChain​(org.apache.lucene.analysis.TokenizerFactory tokenizer,
                              org.apache.lucene.analysis.TokenFilterFactory[] filters)
        Creates a new TokenizerChain w/o any CharFilterFactories.
        Parameters:
        tokenizer - Factory for the Tokenizer to use, must not be null.
        filters - Factories for the TokenFilters to use - if null, will be treated as if empty.
      • TokenizerChain

        public TokenizerChain​(org.apache.lucene.analysis.CharFilterFactory[] charFilters,
                              org.apache.lucene.analysis.TokenizerFactory tokenizer,
                              org.apache.lucene.analysis.TokenFilterFactory[] filters)
        Creates a new TokenizerChain.
        Parameters:
        charFilters - Factories for the CharFilters to use, if any - if null, will be treated as if empty.
        tokenizer - Factory for the Tokenizer to use, must not be null.
        filters - Factories for the TokenFilters to use if any- if null, will be treated as if empty.
    • Method Detail

      • getCharFilterFactories

        public org.apache.lucene.analysis.CharFilterFactory[] getCharFilterFactories()
        Returns:
        array of CharFilterFactories, may be empty but never null
      • getTokenizerFactory

        public org.apache.lucene.analysis.TokenizerFactory getTokenizerFactory()
        Returns:
        the TokenizerFactory in use, will never be null
      • getTokenFilterFactories

        public org.apache.lucene.analysis.TokenFilterFactory[] getTokenFilterFactories()
        Returns:
        array of TokenFilterFactories, may be empty but never null
      • initReaderForNormalization

        protected Reader initReaderForNormalization​(String fieldName,
                                                    Reader reader)
        Overrides:
        initReaderForNormalization in class org.apache.lucene.analysis.Analyzer
      • createComponents

        protected org.apache.lucene.analysis.Analyzer.TokenStreamComponents createComponents​(String fieldName)
        Specified by:
        createComponents in class org.apache.lucene.analysis.Analyzer
      • normalize

        protected org.apache.lucene.analysis.TokenStream normalize​(String fieldName,
                                                                   org.apache.lucene.analysis.TokenStream in)
        Overrides:
        normalize in class org.apache.lucene.analysis.Analyzer
      • getMultiTermAnalyzer

        public org.apache.lucene.analysis.Analyzer getMultiTermAnalyzer()