org.apache.solr.util

Class SolrPluginUtils

java.lang.Object

org.apache.solr.util.SolrPluginUtils

public class SolrPluginUtils
extends Object

Utilities that may be of use to RequestHandlers.

Many of these functions have code that was stolen/mutated from StandardRequestHandler.

:TODO: refactor StandardRequestHandler to use these utilities

:TODO: Many "standard" functionality methods are not cognisant of default parameter settings.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	SolrPluginUtils.DisjunctionMaxQueryParser A subclass of SolrQueryParser that supports aliasing fields for constructing DisjunctionMaxQueries.
static class	SolrPluginUtils.IdentityRegenerator A CacheRegenerator that can be used whenever the items in the cache are not dependant on the current searcher.

Field Summary

Fields

Modifier and Type	Field and Description
protected static String	UNKNOWN_VALUE

Constructor Summary

Constructors

Constructor and Description
SolrPluginUtils()

Method Summary

Modifier and Type	Method and Description
static void	copyNamedListIntoArrayByDocPosInResponse(NamedList namedList, Map<Object,ShardDoc> resultIds, Map.Entry<String,Object>[] destArr) Copies the given namedList assumed to have doc uniqueKey keyed data into destArr at the position of the document in the response.
static SolrDocumentList	docListToSolrDocumentList(DocList docs, SolrIndexSearcher searcher, Set<String> fields, Map<SolrDocument,Integer> ids) Convert a DocList to a SolrDocumentList The optional param "ids" is populated with the lucene document id for each SolrDocument.
static DocList	doSimpleQuery(String sreq, SolrQueryRequest req, int start, int limit) Executes a basic query
static NamedList	doStandardDebug(SolrQueryRequest req, String userQuery, Query query, DocList results, boolean dbgQuery, boolean dbgResults) Returns a NamedList containing many "standard" pieces of debugging information.
static void	doStandardQueryDebug(SolrQueryRequest req, String userQuery, Query query, boolean dbgQuery, NamedList dbg)
static void	doStandardResultsDebug(SolrQueryRequest req, Query query, DocList results, boolean dbgResults, NamedList dbg)
static NamedList<NamedList<Object>>	explanationsToNamedLists(NamedList<Explanation> explanations)
static NamedList<Object>	explanationToNamedList(Explanation e)
static void	flattenBooleanQuery(BooleanQuery.Builder to, BooleanQuery from) Recursively walks the "from" query pulling out sub-queries and adding them to the "to" query.
static Set<String>	getDebugInterests(String[] params, ResponseBuilder rb)
static NamedList<Explanation>	getExplanations(Query query, DocList docs, SolrIndexSearcher searcher, IndexSchema schema) Generates an NamedList of Explanations for each item in a list of docs.
static String	getRequestPurpose(Integer reqPurpose) Given the integer purpose of a request generates a readable value corresponding the request purposes (there can be more than one on a single request).
static Sort	getSort(SolrQueryRequest req) Determines the correct Sort based on the request parameter "sort"
static void	invokeSetters(Object bean, Iterable<Map.Entry<String,Object>> initArgs)
static int	numDocs(SolrIndexSearcher s, Query q, Query f) SolrIndexSearch.numDocs(Query,Query) freaks out if the filtering query is null, so we use this workarround.
static void	optimizePreFetchDocs(ResponseBuilder rb, DocList docs, Query query, SolrQueryRequest req, SolrQueryResponse res) Pre-fetch documents into the index searcher's document cache.
static Map<String,Float>	parseFieldBoosts(String in) Given a string containing fieldNames and boost info, converts it to a Map from field name to boost info.
static Map<String,Float>	parseFieldBoosts(String[] fieldLists) Like parseFieldBoosts(String), but parses all the strings in the provided array (which may be null).
static List<FieldParams>	parseFieldBoostsAndSlop(String[] fieldLists, int wordGrams, int defaultSlop) /** Like parseFieldBoosts(java.lang.String), but allows for an optional slop value prefixed by "~".
static List<Query>	parseQueryStrings(SolrQueryRequest req, String[] queries) Turns an array of query strings into a List of Query objects.
static CharSequence	partialEscape(CharSequence s) Escapes all special characters except '"', '-', and '+'
static <T> NamedList<T>	removeNulls(Map.Entry<String,T>[] entries, NamedList<T> dest) Adds to dest all the not-null elements of entries that have non-null names
static void	setDefaults(SolrQueryRequest req, SolrParams defaults, SolrParams appends, SolrParams invariants) Set default-ish params on a SolrQueryRequest.
static void	setDefaults(SolrRequestHandler handler, SolrQueryRequest req, SolrParams defaults, SolrParams appends, SolrParams invariants)
static void	setMinShouldMatch(BooleanQuery.Builder q, String spec)
static void	setMinShouldMatch(BooleanQuery.Builder q, String spec, boolean mmAutoRelax) Checks the number of optional clauses in the query, and compares it with the specification string to determine the proper value to use.
static BooleanQuery	setMinShouldMatch(BooleanQuery q, String spec)
static BooleanQuery	setMinShouldMatch(BooleanQuery q, String spec, boolean mmAutoRelax)
static String[]	split(String value) Split a value that may contain a comma, space of bar separated list.
static CharSequence	stripIllegalOperators(CharSequence s) Strips operators that are used illegally, otherwise returns its input.
static CharSequence	stripUnbalancedQuotes(CharSequence s) Returns its input if there is an even (ie: balanced) number of '"' characters -- otherwise returns a String in which all '"' characters are striped out.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

UNKNOWN_VALUE

protected static final String UNKNOWN_VALUE

See Also:

Constant Field Values

Constructor Detail

SolrPluginUtils

public SolrPluginUtils()

Method Detail

setDefaults

public static void setDefaults(SolrQueryRequest req,
                               SolrParams defaults,
                               SolrParams appends,
                               SolrParams invariants)

Set default-ish params on a SolrQueryRequest. RequestHandlers can use this method to ensure their defaults and overrides are visible to other components such as the response writer

Parameters:

req - The request whose params we are interested i

defaults - values to be used if no values are specified in the request params

appends - values to be appended to those from the request (or defaults) when dealing with multi-val params, or treated as another layer of defaults for singl-val params.

invariants - values which will be used instead of any request, or default values, regardless of context.

setDefaults

public static void setDefaults(SolrRequestHandler handler,
                               SolrQueryRequest req,
                               SolrParams defaults,
                               SolrParams appends,
                               SolrParams invariants)

numDocs

public static int numDocs(SolrIndexSearcher s,
                          Query q,
                          Query f)
                   throws IOException

SolrIndexSearch.numDocs(Query,Query) freaks out if the filtering query is null, so we use this workarround.

Throws:

IOException

split

public static String[] split(String value)

Split a value that may contain a comma, space of bar separated list.

optimizePreFetchDocs

public static void optimizePreFetchDocs(ResponseBuilder rb,
                                        DocList docs,
                                        Query query,
                                        SolrQueryRequest req,
                                        SolrQueryResponse res)
                                 throws IOException

Pre-fetch documents into the index searcher's document cache. This is an entirely optional step which you might want to perform for the following reasons:

• Locates the document-retrieval costs in one spot, which helps detailed performance measurement
• Determines a priori what fields will be needed to be fetched by various subtasks, like response writing and highlighting. This minimizes the chance that many needed fields will be loaded lazily. (it is more efficient to load all the field we require normally).

If lazy field loading is disabled, this method does nothing.

Throws:

IOException

getDebugInterests

public static Set<String> getDebugInterests(String[] params,
                                            ResponseBuilder rb)

doStandardDebug

public static NamedList doStandardDebug(SolrQueryRequest req,
                                        String userQuery,
                                        Query query,
                                        DocList results,
                                        boolean dbgQuery,
                                        boolean dbgResults)
                                 throws IOException

Returns a NamedList containing many "standard" pieces of debugging information.

• rawquerystring - the 'q' param exactly as specified by the client
• querystring - the 'q' param after any preprocessing done by the plugin
• parsedquery - the main query executed formated by the Solr QueryParsing utils class (which knows about field types)
• parsedquery_toString - the main query executed formatted by its own toString method (in case it has internal state Solr doesn't know about)
• explain - the list of score explanations for each document in results against query.
• otherQuery - the query string specified in 'explainOther' query param.
• explainOther - the list of score explanations for each document in results against 'otherQuery'

Parameters:

req - the request we are dealing with

userQuery - the users query as a string, after any basic preprocessing has been done

query - the query built from the userQuery (and perhaps other clauses) that identifies the main result set of the response.

results - the main result set of the response

Returns:

The debug info

Throws:

IOException - if there was an IO error

doStandardQueryDebug

public static void doStandardQueryDebug(SolrQueryRequest req,
                                        String userQuery,
                                        Query query,
                                        boolean dbgQuery,
                                        NamedList dbg)

doStandardResultsDebug

public static void doStandardResultsDebug(SolrQueryRequest req,
                                          Query query,
                                          DocList results,
                                          boolean dbgResults,
                                          NamedList dbg)
                                   throws IOException

Throws:

IOException

explanationToNamedList

public static NamedList<Object> explanationToNamedList(Explanation e)

explanationsToNamedLists

public static NamedList<NamedList<Object>> explanationsToNamedLists(NamedList<Explanation> explanations)

getExplanations

public static NamedList<Explanation> getExplanations(Query query,
                                                     DocList docs,
                                                     SolrIndexSearcher searcher,
                                                     IndexSchema schema)
                                              throws IOException

Generates an NamedList of Explanations for each item in a list of docs.

Parameters:

query - The Query you want explanations in the context of

docs - The Documents you want explained relative that query

Throws:

IOException

doSimpleQuery

public static DocList doSimpleQuery(String sreq,
                                    SolrQueryRequest req,
                                    int start,
                                    int limit)
                             throws IOException

Executes a basic query

Throws:

IOException

parseFieldBoosts

public static Map<String,Float> parseFieldBoosts(String in)

Given a string containing fieldNames and boost info, converts it to a Map from field name to boost info.

Doesn't care if boost info is negative, you're on your own.

Doesn't care if boost info is missing, again: you're on your own.

Parameters:

in - a String like "fieldOne^2.3 fieldTwo fieldThree^-0.4"

Returns:

Map of fieldOne => 2.3, fieldTwo => null, fieldThree => -0.4

parseFieldBoosts

public static Map<String,Float> parseFieldBoosts(String[] fieldLists)

Like parseFieldBoosts(String), but parses all the strings in the provided array (which may be null).

Parameters:

fieldLists - an array of Strings eg. {"fieldOne^2.3", "fieldTwo", fieldThree^-0.4}

Returns:

Map of fieldOne => 2.3, fieldTwo => null, fieldThree => -0.4

parseFieldBoostsAndSlop

public static List<FieldParams> parseFieldBoostsAndSlop(String[] fieldLists,
                                                        int wordGrams,
                                                        int defaultSlop)

/** Like parseFieldBoosts(java.lang.String), but allows for an optional slop value prefixed by "~".

Parameters:

fieldLists - - an array of Strings eg. {"fieldOne^2.3", "fieldTwo", fieldThree~5^-0.4}

wordGrams - - (0=all words, 2,3 = shingle size)

defaultSlop - - the default slop for this param

Returns:

- FieldParams containing the fieldname,boost,slop,and shingle size

setMinShouldMatch

public static void setMinShouldMatch(BooleanQuery.Builder q,
                                     String spec,
                                     boolean mmAutoRelax)

Checks the number of optional clauses in the query, and compares it with the specification string to determine the proper value to use.

If mmAutoRelax=true, we'll perform auto relaxation of mm if tokens are removed from some but not all DisMax clauses, as can happen when stopwords or punctuation tokens are removed in analysis.

Details about the specification format can be found here

A few important notes...

• If the calculations based on the specification determine that no optional clauses are needed, BooleanQuerysetMinMumberShouldMatch will never be called, but the usual rules about BooleanQueries still apply at search time (a BooleanQuery containing no required clauses must still match at least one optional clause)
• No matter what number the calculation arrives at, BooleanQuery.setMinShouldMatch() will never be called with a value greater then the number of optional clauses (or less then 1)

:TODO: should optimize the case where number is same as clauses to just make them all "required"

Parameters:

q - The query as a BooleanQuery.Builder

spec - The mm spec

mmAutoRelax - whether to perform auto relaxation of mm if tokens are removed from some but not all DisMax clauses

setMinShouldMatch

public static void setMinShouldMatch(BooleanQuery.Builder q,
                                     String spec)

setMinShouldMatch

public static BooleanQuery setMinShouldMatch(BooleanQuery q,
                                             String spec)

setMinShouldMatch

public static BooleanQuery setMinShouldMatch(BooleanQuery q,
                                             String spec,
                                             boolean mmAutoRelax)

flattenBooleanQuery

public static void flattenBooleanQuery(BooleanQuery.Builder to,
                                       BooleanQuery from)

Recursively walks the "from" query pulling out sub-queries and adding them to the "to" query.

Boosts are multiplied as needed. Sub-BooleanQueryies which are not optional will not be flattened. From will be mangled durring the walk, so do not attempt to reuse it.

partialEscape

public static CharSequence partialEscape(CharSequence s)

Escapes all special characters except '"', '-', and '+'

stripIllegalOperators

public static CharSequence stripIllegalOperators(CharSequence s)

Strips operators that are used illegally, otherwise returns its input. Some examples of illegal user queries are: "chocolate +- chip", "chocolate - - chip", and "chocolate chip -".

stripUnbalancedQuotes

public static CharSequence stripUnbalancedQuotes(CharSequence s)

Returns its input if there is an even (ie: balanced) number of '"' characters -- otherwise returns a String in which all '"' characters are striped out.

removeNulls

public static <T> NamedList<T> removeNulls(Map.Entry<String,T>[] entries,
                                           NamedList<T> dest)

Adds to dest all the not-null elements of entries that have non-null names

Parameters:

entries - The array of entries to be added to the NamedList dest

dest - The NamedList instance where the not-null elements of entries are added

Returns:

Returns The dest input object

copyNamedListIntoArrayByDocPosInResponse

public static void copyNamedListIntoArrayByDocPosInResponse(NamedList namedList,
                                                            Map<Object,ShardDoc> resultIds,
                                                            Map.Entry<String,Object>[] destArr)

Copies the given namedList assumed to have doc uniqueKey keyed data into destArr at the position of the document in the response. destArr is assumed to be the same size as resultIds is. resultIds comes from ResponseBuilder.resultIds. If the doc key isn't in resultIds then it is ignored. Note: most likely you will call removeNulls(Map.Entry[], NamedList) sometime after calling this.

getSort

public static Sort getSort(SolrQueryRequest req)

Determines the correct Sort based on the request parameter "sort"

Returns:

null if no sort is specified.

parseQueryStrings

public static List<Query> parseQueryStrings(SolrQueryRequest req,
                                            String[] queries)
                                     throws SyntaxError

Turns an array of query strings into a List of Query objects.

Returns:

null if no queries are generated

Throws:

SyntaxError

docListToSolrDocumentList

public static SolrDocumentList docListToSolrDocumentList(DocList docs,
                                                         SolrIndexSearcher searcher,
                                                         Set<String> fields,
                                                         Map<SolrDocument,Integer> ids)
                                                  throws IOException

Convert a DocList to a SolrDocumentList The optional param "ids" is populated with the lucene document id for each SolrDocument.

Parameters:

docs - The DocList to convert

searcher - The SolrIndexSearcher to use to load the docs from the Lucene index

fields - The names of the Fields to load

ids - A map to store the ids of the docs

Returns:

The new SolrDocumentList containing all the loaded docs

Throws:

IOException - if there was a problem loading the docs

Since:

solr 1.4

invokeSetters

public static void invokeSetters(Object bean,
                                 Iterable<Map.Entry<String,Object>> initArgs)

getRequestPurpose

public static String getRequestPurpose(Integer reqPurpose)

Given the integer purpose of a request generates a readable value corresponding the request purposes (there can be more than one on a single request). If there is a purpose parameter present that's not known this method will return "Unknown"

Parameters:

reqPurpose - Numeric request purpose

Returns:

a comma separated list of purposes or "Unknown"