Package org.apache.solr.schema

Class FieldType

java.lang.Object
org.apache.solr.schema.FieldProperties
org.apache.solr.schema.FieldType
Direct Known Subclasses:
AbstractSpatialFieldType, AbstractSubTypeFieldType, BinaryField, CollationField, CurrencyFieldType, PrimitiveFieldType, RandomSortField, RankField, TextField
public abstract class FieldType extends FieldProperties
Base class for all field types used by an index schema.
Since:
3.1

  • Field Details

  • Constructor Details

    • FieldType

      public FieldType()

  • Method Details

    • isTokenized

      public boolean isTokenized()
      Returns true if fields of this type should be tokenized

    • isMultiValued

      public boolean isMultiValued()
      Returns true if fields can have multiple values

    • hasProperty

      protected boolean hasProperty(int p)
      Check if a property is set

    • isPolyField

      public boolean isPolyField()
      A "polyField" is a FieldType that can produce more than one IndexableField instance for a single value, via the createFields(org.apache.solr.schema.SchemaField, Object) method. This is useful when hiding the implementation details of a field from the Solr end user. For instance, a spatial point may be represented by multiple different fields.
      Returns:
      true if the createFields(org.apache.solr.schema.SchemaField, Object) method may return more than one field

    • isPointField

      public boolean isPointField()

    • isUtf8Field

      public boolean isUtf8Field()

    • useDocValuesAsStored

      public boolean useDocValuesAsStored()
      Returns true if the fields' docValues should be used for obtaining stored value

    • multiValuedFieldCache

      public boolean multiValuedFieldCache()
      Returns true if a single field value of this type has multiple logical values for the purposes of faceting, sorting, etc. Text fields normally return true since each token/word is a logical value.

    • init

      protected void init(IndexSchema schema, Map<String,String> args)
      subclasses should initialize themselves with the args provided and remove valid arguments. leftover arguments will cause an exception. Common boolean properties have already been handled.

    • write

      public boolean write(org.apache.solr.common.IteratorWriter.ItemWriter itemWriter)

    • setArgs

      protected void setArgs(IndexSchema schema, Map<String,String> args)
      Initializes the field type. Subclasses should usually override init(IndexSchema, Map) which is called by this method.

    • restrictProps

      protected void restrictProps(int props)
      :TODO: document this method

    • getTypeName

      public String getTypeName()
      The Name of this FieldType as specified in the schema file

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • createField

      public org.apache.lucene.index.IndexableField createField(SchemaField field, Object value)
      Used for adding a document when a field needs to be created from a type and a string.

      By default, the indexed value is the same as the stored value (taken from toInternal()). Having a different representation for external, internal, and indexed would present quite a few problems given the current Lucene architecture. An analyzer for adding docs would need to translate internal->indexed while an analyzer for querying would need to translate external->indexed.

      The only other alternative to having internal==indexed would be to have internal==external. In this case, toInternal should convert to the indexed representation, toExternal() should do nothing, and createField() should *not* call toInternal, but use the external value and set tokenized=true to get Lucene to convert to the internal(indexed) form. :TODO: clean up and clarify this explanation.

      See Also:

    • createField

      protected org.apache.lucene.index.IndexableField createField(String name, String val, org.apache.lucene.index.IndexableFieldType type)
      Create the field from native Lucene parts. Mostly intended for use by FieldTypes outputing multiple Fields per SchemaField
      Parameters:
      name - The name of the field
      val - The _internal_ value to index
      type - FieldType
      Returns:
      the IndexableField.

    • createFields

      public List<org.apache.lucene.index.IndexableField> createFields(SchemaField field, Object value)
      Given a SchemaField, create one or more IndexableField instances
      Parameters:
      field - the SchemaField
      value - The value to add to the field
      Returns:
      An array of IndexableField
      See Also:

    • toInternal

      public String toInternal(String val)
      Convert an external value (from XML update command or from query string) into the internal format for both storing and indexing (which can be modified by any analyzers).
      See Also:

    • toExternal

      public String toExternal(org.apache.lucene.index.IndexableField f)
      Convert the stored-field format to an external (string, human readable) value
      See Also:

    • toObject

      public Object toObject(org.apache.lucene.index.IndexableField f)
      Convert the stored-field format to an external object.
      Since:
      solr 1.3
      See Also:

    • toObject

      public Object toObject(SchemaField sf, org.apache.lucene.util.BytesRef term)

    • indexedToReadable

      public String indexedToReadable(String indexedForm)
      Given an indexed term, return the human readable representation

    • indexedToReadable

      public org.apache.lucene.util.CharsRef indexedToReadable(org.apache.lucene.util.BytesRef input, org.apache.lucene.util.CharsRefBuilder output)
      Given an indexed term, append the human readable representation

    • storedToReadable

      public String storedToReadable(org.apache.lucene.index.IndexableField f)
      Given the stored field, return the human readable representation

    • storedToIndexed

      public String storedToIndexed(org.apache.lucene.index.IndexableField f)
      Given the stored field, return the indexed form

    • readableToIndexed

      public String readableToIndexed(String val)
      Given the readable value, return the term value that will match it.

    • readableToIndexed

      public void readableToIndexed(CharSequence val, org.apache.lucene.util.BytesRefBuilder result)
      Given the readable value, return the term value that will match it. This method will modify the size and length of the result parameter and write from offset 0

    • setIsExplicitQueryAnalyzer

      public void setIsExplicitQueryAnalyzer(boolean isExplicitQueryAnalyzer)

    • isExplicitQueryAnalyzer

      public boolean isExplicitQueryAnalyzer()

    • setIsExplicitAnalyzer

      public void setIsExplicitAnalyzer(boolean explicitAnalyzer)

    • isExplicitAnalyzer

      public boolean isExplicitAnalyzer()

    • getClassArg

      public String getClassArg()
      Returns:
      the string used to specify the concrete class name in a serialized representation: the class arg. If the concrete class name was not specified via a class arg, returns getClass().getName().

    • getPrefixQuery

      public org.apache.lucene.search.Query getPrefixQuery(QParser parser, SchemaField sf, String termStr)
      Returns a Query instance for doing prefix searches on this field type. Also, other QueryParser implementations may have different semantics.

      Sub-classes should override this method to provide their own range query implementation.

      Parameters:
      parser - the QParser calling the method
      sf - the schema field
      termStr - the term string for prefix query, if blank then this query should match all docs with this field
      Returns:
      a Query instance to perform prefix search

    • getUninversionType

      public abstract UninvertingReader.Type getUninversionType(SchemaField sf)
      If DocValues is not enabled for a field, but it's indexed, docvalues can be constructed on the fly (uninverted, aka fieldcache) on the first request to sort, facet, etc. This specifies the structure to use.

      This method will not be used if the field is (effectively) uninvertible="false"

      Parameters:
      sf - field instance
      Returns:
      type to uninvert, or null (to disallow uninversion for the field)
      See Also:

    • getIndexAnalyzer

      public org.apache.lucene.analysis.Analyzer getIndexAnalyzer()
      Returns the Analyzer to be used when indexing fields of this type.

      This method may be called many times, at any time.

      See Also:

    • getQueryAnalyzer

      public org.apache.lucene.analysis.Analyzer getQueryAnalyzer()
      Returns the Analyzer to be used when searching fields of this type.

      This method may be called many times, at any time.

      See Also:

    • supportsAnalyzers

      protected boolean supportsAnalyzers()
      Returns true if this type supports index and query analyzers, false otherwise.

    • setIndexAnalyzer

      public final void setIndexAnalyzer(org.apache.lucene.analysis.Analyzer analyzer)
      Sets the Analyzer to be used when indexing fields of this type.

      Subclasses should override supportsAnalyzers() to enable this function.

      See Also:

    • setQueryAnalyzer

      public final void setQueryAnalyzer(org.apache.lucene.analysis.Analyzer analyzer)
      Sets the Analyzer to be used when querying fields of this type.

      Subclasses should override supportsAnalyzers() to enable this function.

      See Also:

    • getSimilarity

      public org.apache.lucene.search.similarities.Similarity getSimilarity()
      Gets the Similarity used when scoring fields of this type

      The default implementation returns null, which means this type has no custom similarity associated with it.

      NOTE: This API is for internal purposes only and might change in incompatible ways in the next release.

    • getSimilarityFactory

      public SimilarityFactory getSimilarityFactory()
      Gets the factory for the Similarity used when scoring fields of this type

      The default implementation returns null, which means this type has no custom similarity factory associated with it.

      NOTE: This API is for internal purposes only and might change in incompatible ways in the next release.

    • getNumberType

      public NumberType getNumberType()
      Return the numeric type of this field, or null if this field is not a numeric field.

    • setSimilarity

      public void setSimilarity(SimilarityFactory similarityFactory)
      Sets the Similarity used when scoring fields of this type
      NOTE: This API is for internal purposes only and might change in incompatible ways in the next release.

    • getPostingsFormat

      public String getPostingsFormat()

    • getDocValuesFormat

      public final String getDocValuesFormat()

    • write

      public abstract void write(TextResponseWriter writer, String name, org.apache.lucene.index.IndexableField f) throws IOException
      calls back to TextResponseWriter to write the field value

      Sub-classes should prefer using toExternal(IndexableField) or toObject(IndexableField) to get the writeable external value of f instead of directly using f.stringValue() or f.binaryValue()

      Throws:
      IOException

    • getSortField

      public abstract org.apache.lucene.search.SortField getSortField(SchemaField field, boolean top)
      Returns the SortField instance that should be used to sort fields of this type.
      See Also:

    • getSortField

      protected static org.apache.lucene.search.SortField getSortField(SchemaField field, org.apache.lucene.search.SortField.Type sortType, boolean reverse, Object missingLow, Object missingHigh)
      A Helper utility method for use by subclasses.

      This method deals with:

      • SchemaField.checkSortability()
      • Creating a SortField on field with the specified reverse & sortType
      • Setting the SortField.setMissingValue(java.lang.Object) to missingLow or missingHigh as appropriate based on the value of reverse and the sortMissingFirst & sortMissingLast properties of the field
      Parameters:
      field - The SchemaField to sort on. May use sortMissingFirst or sortMissingLast or neither.
      sortType - The sort Type of the underlying values in the field
      reverse - True if natural order of the sortType should be reversed
      missingLow - The missingValue to be used if the other params indicate that docs w/o values should sort as "low" as possible.
      missingHigh - The missingValue to be used if the other params indicate that docs w/o values should sort as "high" as possible.
      See Also:

    • getSortedSetSortField

      protected static org.apache.lucene.search.SortField getSortedSetSortField(SchemaField field, org.apache.lucene.search.SortedSetSelector.Type selector, boolean reverse, Object missingLow, Object missingHigh)
      Same as getSortField(org.apache.solr.schema.SchemaField, boolean) but using SortedSetSortField

    • getSortedNumericSortField

      protected static org.apache.lucene.search.SortField getSortedNumericSortField(SchemaField field, org.apache.lucene.search.SortField.Type sortType, org.apache.lucene.search.SortedNumericSelector.Type selector, boolean reverse, Object missingLow, Object missingHigh)
      Same as getSortField(org.apache.solr.schema.SchemaField, boolean) but using SortedNumericSortField.

    • getStringSort

      protected org.apache.lucene.search.SortField getStringSort(SchemaField field, boolean reverse)
      Utility usable by subclasses when they want to get basic String sorting using common checks.
      See Also:

    • getNumericSort

      protected org.apache.lucene.search.SortField getNumericSort(SchemaField field, NumberType type, boolean reverse)
      Utility usable by subclasses when they want to get basic Numeric sorting using common checks.
      See Also:

    • getValueSource

      public org.apache.lucene.queries.function.ValueSource getValueSource(SchemaField field, QParser parser)
      called to get the default value source (normally, from the Lucene FieldCache.)

    • getSingleValueSource

      public org.apache.lucene.queries.function.ValueSource getSingleValueSource(FieldType.MultiValueSelector choice, SchemaField field, QParser parser)
      Method for dynamically building a ValueSource based on a single value of a multivalued field.

      The default implementation throws an error except in the trivial case where this method is used on a SchemaField that is in fact not-multivalued, in which case it delegates to getValueSource(org.apache.solr.schema.SchemaField, org.apache.solr.search.QParser)

      See Also:

    • getDefaultMultiValueSelectorForSort

      public FieldType.MultiValueSelector getDefaultMultiValueSelectorForSort(SchemaField field, boolean reverse)
      Method for indicating which FieldType.MultiValueSelector (if any) should be used when sorting on a multivalued field of this type for the specified direction (asc/desc). The default implementation returns null (for all inputs).
      Parameters:
      field - The SchemaField (of this type) in question
      reverse - false if this is an ascending sort, true if this is a descending sort.
      Returns:
      the implicit selector to use for this direction, or null if implicit sorting on the specified direction is not supported and should return an error.
      See Also:

    • getRangeQuery

      public org.apache.lucene.search.Query getRangeQuery(QParser parser, SchemaField field, String part1, String part2, boolean minInclusive, boolean maxInclusive)
      Returns a Query instance for doing range searches on this field type. SolrQueryParser currently passes part1 and part2 as null if they are '*' respectively. minInclusive and maxInclusive are both true currently by SolrQueryParser but that may change in the future. Also, other QueryParser implementations may have different semantics.

      By default range queries with '*'s or nulls on either side are treated as existence queries and are created with getExistenceQuery(org.apache.solr.search.QParser, org.apache.solr.schema.SchemaField). If unbounded range queries should not be treated as existence queries for a certain fieldType, then treatUnboundedRangeAsExistence(org.apache.solr.schema.SchemaField) should be overridden.

      Sub-classes should override the getSpecializedRangeQuery(org.apache.solr.search.QParser, org.apache.solr.schema.SchemaField, java.lang.String, java.lang.String, boolean, boolean) method to provide their own range query implementation.

      Parameters:
      parser - the QParser calling the method
      field - the schema field
      part1 - the lower boundary of the range, nulls are allowed.
      part2 - the upper boundary of the range, nulls are allowe
      minInclusive - whether the minimum of the range is inclusive or not
      maxInclusive - whether the maximum of the range is inclusive or not
      Returns:
      a Query instance to perform range search according to given parameters

    • treatUnboundedRangeAsExistence

      protected boolean treatUnboundedRangeAsExistence(SchemaField field)
      Returns whether an unbounded range query should be treated the same as an existence query for the given field type.
      Parameters:
      field - the schema field
      Returns:
      whether unbounded range and existence are equivalent for the given field type.

    • getSpecializedRangeQuery

      protected org.apache.lucene.search.Query getSpecializedRangeQuery(QParser parser, SchemaField field, String part1, String part2, boolean minInclusive, boolean maxInclusive)
      Returns a Query instance for doing range searches on this field type. SolrQueryParser currently passes part1 and part2 as null if they are '*' respectively. minInclusive and maxInclusive are both true currently by SolrQueryParser but that may change in the future. Also, other QueryParser implementations may have different semantics.

      Sub-classes should override this method to provide their own range query implementation. They should strive to handle nulls in part1 and/or part2 as well as unequal minInclusive and maxInclusive parameters gracefully.

      This method does not, and should not, check for or handle existence queries, please look at getRangeQuery(org.apache.solr.search.QParser, org.apache.solr.schema.SchemaField, java.lang.String, java.lang.String, boolean, boolean) for that logic.

      Parameters:
      parser - the QParser calling the method
      field - the schema field
      part1 - the lower boundary of the range, nulls are allowed.
      part2 - the upper boundary of the range, nulls are allowed
      minInclusive - whether the minimum of the range is inclusive or not
      maxInclusive - whether the maximum of the range is inclusive or not
      Returns:
      a Query instance to perform range search according to given parameters

    • getExistenceQuery

      public org.apache.lucene.search.Query getExistenceQuery(QParser parser, SchemaField field)
      Returns a Query instance for doing existence searches for a field. If the field does not have docValues or norms, this method will call getSpecializedExistenceQuery(org.apache.solr.search.QParser, org.apache.solr.schema.SchemaField), which defaults to an unbounded rangeQuery.

      This method should only be overridden whenever a fieldType does not support FieldExistsQuery. If a fieldType does not support an unbounded rangeQuery as an existenceQuery (such as double or float fields), getSpecializedExistenceQuery(org.apache.solr.search.QParser, org.apache.solr.schema.SchemaField) should be overridden.

      Parameters:
      parser - The QParser calling the method
      field - The SchemaField of the field to search
      Returns:
      The Query instance.

    • getSpecializedExistenceQuery

      protected org.apache.lucene.search.Query getSpecializedExistenceQuery(QParser parser, SchemaField field)
      Returns a Query instance for doing existence searches for a field without certain options, such as docValues or norms.

      This method can be overridden to implement specialized existence logic for fieldTypes. The default query returned is an unbounded range query.

      Parameters:
      parser - The QParser calling the method
      field - The SchemaField of the field to search
      Returns:
      The Query instance.

    • getFieldQuery

      public org.apache.lucene.search.Query getFieldQuery(QParser parser, SchemaField field, String externalVal)
      Returns a Query instance for doing searches against a field.
      Parameters:
      parser - The QParser calling the method
      field - The SchemaField of the field to search
      externalVal - The String representation of the value to search
      Returns:
      The Query instance. This implementation returns a TermQuery but overriding queries may not

    • getFieldTermQuery

      public org.apache.lucene.search.Query getFieldTermQuery(QParser parser, SchemaField field, String externalVal)
      Returns a Query instance for doing a single term search against a field. This term will not be analyzed before searching.
      Parameters:
      parser - The QParser calling the method
      field - The SchemaField of the field to search
      externalVal - The String representation of the term value to search
      Returns:
      The Query instance.

    • getSetQuery

      public org.apache.lucene.search.Query getSetQuery(QParser parser, SchemaField field, Collection<String> externalVals)
      WARNING: This API is experimental and might change in incompatible ways in the next release.

    • getRewriteMethod

      public org.apache.lucene.search.MultiTermQuery.RewriteMethod getRewriteMethod(QParser parser, SchemaField field)
      Expert: Returns the rewrite method for multiterm queries such as wildcards.
      Parameters:
      parser - The QParser calling the method
      field - The SchemaField of the field to search
      Returns:
      A suitable rewrite method for rewriting multi-term queries to primitive queries.

    • checkSchemaField

      public void checkSchemaField(SchemaField field)
      Check's SchemaField instances constructed using this field type to ensure that they are valid.

      This method is called by the SchemaField constructor to check that its initialization does not violate any fundamental requirements of the FieldType. Subclasses may choose to throw a SolrException if invariants are violated by the SchemaField.

    • checkSupportsDocValues

      protected void checkSupportsDocValues()
      Called by checkSchemaField(SchemaField) if the field has docValues. By default none do.

    • enableDocValuesByDefault

      protected boolean enableDocValuesByDefault()
      Returns whether this field type should enable docValues by default for schemaVersion >= 1.7. This should not be enabled for fields that did not have docValues implemented by Solr 9.7, as users may have indexed documents without docValues (since they weren't supported). Flipping the default docValues values when they upgrade to a new version will break their index compatibility.

      New field types can enable this without issue, as long as they support docValues.

    • getNamedPropertyValues

      public org.apache.solr.common.util.SimpleOrderedMap<Object> getNamedPropertyValues(boolean showDefaults)
      Get a map of property name -> value for this field type.
      Parameters:
      showDefaults - if true, include default properties.

    • getNonFieldPropertyArgs

      protected Map<String,String> getNonFieldPropertyArgs()
      Returns args to this field type that aren't standard field properties

    • getAnalyzerProperties

      protected static org.apache.solr.common.util.SimpleOrderedMap<Object> getAnalyzerProperties(org.apache.lucene.analysis.Analyzer analyzer)
      Returns a description of the given analyzer, by either reporting the Analyzer class name (and optionally luceneMatchVersion) if it's not a TokenizerChain, or if it is, querying each analysis factory for its name and args.

    • toNativeType

      public Object toNativeType(Object val)
      Converts any Object to a java Object native to this field type

    • marshalSortValue

      public Object marshalSortValue(Object value)
      Convert a value used by the FieldComparator for this FieldType's SortField into a marshalable value for distributed sorting.

    • unmarshalSortValue

      public Object unmarshalSortValue(Object value)
      Convert a value marshaled via marshalSortValue(java.lang.Object) back into a value usable by the FieldComparator for this FieldType's SortField

    • marshalStringSortValue

      protected static Object marshalStringSortValue(Object value)
      Marshals a string-based field value.

    • unmarshalStringSortValue

      protected static Object unmarshalStringSortValue(Object value)
      Unmarshals a string-based field value.

    • marshalBase64SortValue

      protected static Object marshalBase64SortValue(Object value)
      Marshals a binary field value.

    • unmarshalBase64SortValue

      protected static Object unmarshalBase64SortValue(Object value)
      Unmarshals a binary field value.