org.apache.solr.search.stats

Class StatsCache

java.lang.Object

org.apache.solr.search.stats.StatsCache

All Implemented Interfaces:

PluginInfoInitialized

Direct Known Subclasses:

ExactStatsCache, LocalStatsCache

public abstract class StatsCache
extends Object
implements PluginInfoInitialized

This class represents a cache of global document frequency information for selected terms. This information is periodically updated from all shards, either through scheduled events of some kind, or on every request when there is no global stats available for terms involved in the query (or if this information is stale due to changes in the shards).

There are instances of this class at the aggregator node (where the partial data from shards is aggregated), and on each core involved in a shard request (where this data is maintained and updated from the aggregator's cache).

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	StatsCache.StatsCacheMetrics

Field Summary

Fields

Modifier and Type	Field and Description
static String	COL_STATS_KEY Value of CollectionStats.
static String	FIELDS_KEY List of fields in the query.
protected PluginInfo	pluginInfo
protected StatsCache.StatsCacheMetrics	statsCacheMetrics
static String	TERM_STATS_KEY Map of terms and TermStats.
static String	TERMS_KEY List of terms in the query.

Constructor Summary

Constructors

Constructor and Description
StatsCache()

Method Summary

Modifier and Type	Method and Description
int	approxCheckMissingStats(ResponseBuilder rb, StatsSource statsSource, Consumer<Term> missingTermStats, Consumer<String> missingFieldStats) Check if the statsSource is missing some term or field statistics info, which then needs to be retrieved.
void	clear() Clear cached statistics.
protected abstract StatsSource	doGet(SolrQueryRequest req)
protected abstract void	doMergeToGlobalStats(SolrQueryRequest req, List<ShardResponse> responses)
protected abstract void	doReceiveGlobalStats(SolrQueryRequest req)
protected abstract ShardRequest	doRetrieveStatsRequest(ResponseBuilder rb)
protected abstract void	doReturnLocalStats(ResponseBuilder rb, SolrIndexSearcher searcher)
protected abstract void	doSendGlobalStats(ResponseBuilder rb, ShardRequest outgoing)
StatsSource	get(SolrQueryRequest req) Prepare a StatsSource that provides stats information to perform local scoring (to be precise, to build a local Weight from the query).
StatsCache.StatsCacheMetrics	getCacheMetrics()
void	init(PluginInfo info)
void	mergeToGlobalStats(SolrQueryRequest req, List<ShardResponse> responses) Process shard responses that contain partial local stats.
void	receiveGlobalStats(SolrQueryRequest req) Receive global stats data from the master and update a local cache of global stats with this global data.
ShardRequest	retrieveStatsRequest(ResponseBuilder rb) Creates a ShardRequest to retrieve per-shard stats related to the current query and the current state of the requester's StatsCache.
void	returnLocalStats(ResponseBuilder rb, SolrIndexSearcher searcher) Prepare a local (from the local shard) response to a "retrieve stats" shard request.
void	sendGlobalStats(ResponseBuilder rb, ShardRequest outgoing) Prepare global stats data to be sent out to shards in this request.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

TERM_STATS_KEY

public static final String TERM_STATS_KEY

Map of terms and TermStats.

See Also:

Constant Field Values

COL_STATS_KEY

public static final String COL_STATS_KEY

Value of CollectionStats.

See Also:

Constant Field Values

TERMS_KEY

public static final String TERMS_KEY

List of terms in the query.

See Also:

Constant Field Values

FIELDS_KEY

public static final String FIELDS_KEY

List of fields in the query.

See Also:

Constant Field Values

statsCacheMetrics

protected StatsCache.StatsCacheMetrics statsCacheMetrics

pluginInfo

protected PluginInfo pluginInfo

Constructor Detail

StatsCache

public StatsCache()

Method Detail

getCacheMetrics

public StatsCache.StatsCacheMetrics getCacheMetrics()

init

public void init(PluginInfo info)

Specified by:

init in interface PluginInfoInitialized

retrieveStatsRequest

public ShardRequest retrieveStatsRequest(ResponseBuilder rb)

Creates a ShardRequest to retrieve per-shard stats related to the current query and the current state of the requester's StatsCache.

This method updates the cache metrics and calls doRetrieveStatsRequest(ResponseBuilder).

Parameters:

rb - contains current request

Returns:

shard request to retrieve stats for terms in the current request, or null if no additional request is needed (e.g. if the information in global cache is already sufficient to satisfy this request).

doRetrieveStatsRequest

protected abstract ShardRequest doRetrieveStatsRequest(ResponseBuilder rb)

returnLocalStats

public void returnLocalStats(ResponseBuilder rb,
                             SolrIndexSearcher searcher)

Prepare a local (from the local shard) response to a "retrieve stats" shard request.

This method updates the cache metrics and calls doReturnLocalStats(ResponseBuilder, SolrIndexSearcher).

Parameters:

rb - response builder

searcher - current local searcher

doReturnLocalStats

protected abstract void doReturnLocalStats(ResponseBuilder rb,
                                           SolrIndexSearcher searcher)

mergeToGlobalStats

public void mergeToGlobalStats(SolrQueryRequest req,
                               List<ShardResponse> responses)

Process shard responses that contain partial local stats. Usually this entails combining per-shard stats for each term.

This method updates the cache metrics and calls doMergeToGlobalStats(SolrQueryRequest, List).

Parameters:

req - query request

responses - responses from shards containing local stats for each shard

doMergeToGlobalStats

protected abstract void doMergeToGlobalStats(SolrQueryRequest req,
                                             List<ShardResponse> responses)

receiveGlobalStats

public void receiveGlobalStats(SolrQueryRequest req)

Receive global stats data from the master and update a local cache of global stats with this global data. This event occurs either as a separate request, or together with the regular query request, in which case this method is called first, before preparing a QueryCommand to be submitted to the local SolrIndexSearcher.

This method updates the cache metrics and calls doReceiveGlobalStats(SolrQueryRequest).

Parameters:

req - query request with global stats data

doReceiveGlobalStats

protected abstract void doReceiveGlobalStats(SolrQueryRequest req)

sendGlobalStats

public void sendGlobalStats(ResponseBuilder rb,
                            ShardRequest outgoing)

Prepare global stats data to be sent out to shards in this request.

This method updates the cache metrics and calls doSendGlobalStats(ResponseBuilder, ShardRequest).

Parameters:

rb - response builder

outgoing - shard request to be sent

doSendGlobalStats

protected abstract void doSendGlobalStats(ResponseBuilder rb,
                                          ShardRequest outgoing)

get

public StatsSource get(SolrQueryRequest req)

Prepare a StatsSource that provides stats information to perform local scoring (to be precise, to build a local Weight from the query).

This method updates the cache metrics and calls doGet(SolrQueryRequest).

Parameters:

req - query request

Returns:

an instance of StatsSource to use in creating a query Weight

doGet

protected abstract StatsSource doGet(SolrQueryRequest req)

clear

public void clear()

Clear cached statistics.

approxCheckMissingStats

public int approxCheckMissingStats(ResponseBuilder rb,
                                   StatsSource statsSource,
                                   Consumer<Term> missingTermStats,
                                   Consumer<String> missingFieldStats)
                            throws IOException

Check if the statsSource is missing some term or field statistics info, which then needs to be retrieved.

NOTE: this uses the local IndexReader for query rewriting, which may expand to less (or different) terms as rewriting the same query on other shards' readers. This in turn may falsely fail to inform the consumers about possibly missing stats, which may lead consumers to skip the fetching of full stats. Consequently this would lead to incorrect global IDF data for the missing terms (because for these terms only local stats would be used).

Parameters:

rb - request to evaluate against the statsSource

statsSource - stats source to check

missingTermStats - consumer of missing term stats

missingFieldStats - consumer of missing field stats

Returns:

approximate number of missing term stats and field stats combined

Throws:

IOException