org.apache.solr.common.cloud

Class ZkStateReader

java.lang.Object

org.apache.solr.common.cloud.ZkStateReader

All Implemented Interfaces:

Closeable, AutoCloseable

public class ZkStateReader
extends Object
implements Closeable

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	ZkStateReader.AliasesManager A class to manage the aliases instance, including watching for changes.
static class	ZkStateReader.ConfigData

Field Summary

Fields

Modifier and Type	Field and Description
static String	ALIASES
ZkStateReader.AliasesManager	aliasesManager Access to the Aliases.
static String	AUTO_ADD_REPLICAS
static String	BASE_URL_PROP
static String	CLUSTER_PROPS
static String	CLUSTER_STATE
protected ClusterState	clusterState A view of the current state of all collections; combines all the different state sources into a single view.
static String	COLLECTION_DEF Deprecated. use CollectionAdminParams.DEFAULTS instead.
static String	COLLECTION_PROP
static String	COLLECTION_PROPS_ZKNODE
static String	COLLECTIONS_ZKNODE
static String	CONFIGNAME_PROP
static String	CONFIGS_ZKNODE
static String	CORE_NAME_PROP SolrCore name.
static String	CORE_NODE_NAME_PROP
static String	ELECTION_NODE
static String	ELECTION_NODE_PROP
static String	FORCE_SET_STATE_PROP
static Set<String>	KNOWN_CLUSTER_PROPS
static String	LEADER_ELECT_ZKNODE
static String	LEADER_PROP
static String	LEGACY_CLOUD
static String	LIVE_NODES_ZKNODE
static String	MAX_AT_ONCE_PROP
static String	MAX_CORES_PER_NODE
static String	MAX_SHARDS_PER_NODE
static String	MAX_WAIT_SECONDS_PROP
static String	NODE_NAME_PROP
static String	NRT_REPLICAS
static String	NUM_SHARDS_PROP
static String	PROPERTY_PROP
static String	PROPERTY_PROP_PREFIX
static String	PROPERTY_VALUE_PROP
static String	PULL_REPLICAS
static String	REJOIN_AT_HEAD_PROP
static String	REPLICA_PROP
static String	REPLICA_TYPE
static String	REPLICATION_FACTOR
static String	ROLES
static String	ROLES_PROP
static String	SHARD_ID_PROP
static String	SHARD_LEADERS_ZKNODE
static String	SHARD_PARENT_PROP
static String	SHARD_RANGE_PROP
static String	SHARD_STATE_PROP
static String	SHARED_STORAGE_PROP
static String	SOLR_AUTOSCALING_CONF_PATH
static String	SOLR_AUTOSCALING_EVENTS_PATH
static String	SOLR_AUTOSCALING_NODE_ADDED_PATH
static String	SOLR_AUTOSCALING_NODE_LOST_PATH
static String	SOLR_AUTOSCALING_TRIGGER_STATE_PATH
static String	SOLR_SECURITY_CONF_PATH
static String	STATE_PROP
static String	STATE_TIMESTAMP_PROP
static int	STATE_UPDATE_DELAY
static String	TLOG_REPLICAS
static String	URL_SCHEME

Constructor Summary

Constructors

Constructor and Description
ZkStateReader(SolrZkClient zkClient)
ZkStateReader(SolrZkClient zkClient, Runnable securityNodeListener)
ZkStateReader(String zkServerAddress, int zkClientTimeout, int zkClientConnectTimeout)

Method Summary

Modifier and Type	Method and Description
void	close()
Integer	compareStateVersions(String coll, int version)
void	createClusterStateWatchersAndUpdate()
void	forceUpdateCollection(String collection) Forcibly refresh a collection's internal state from ZK.
void	forciblyRefreshAllClusterStateSlow() Forcibly refresh cluster state from ZK.
Aliases	getAliases() Get an immutable copy of the present state of the aliases.
AutoScalingConfig	getAutoScalingConfig() Get current AutoScalingConfig.
AutoScalingConfig	getAutoScalingConfig(org.apache.zookeeper.Watcher watcher) Get current AutoScalingConfig.
String	getBaseUrlForNodeName(String nodeName) Returns the baseURL corresponding to a given node's nodeName -- NOTE: does not (currently) imply that the nodeName (or resulting baseURL) exists in the cluster.
Map<String,Object>	getClusterProperties() Get all cluster properties for this cluster N.B.
<T> T	getClusterProperty(List<String> keyPath, T defaultValue) Same as the above but allows a full json path as a list of parts
<T> T	getClusterProperty(String key, T defaultValue) Get a cluster property N.B.
ClusterState	getClusterState()
static DocCollection	getCollectionLive(ZkStateReader zkStateReader, String coll)
static String	getCollectionPath(String coll)
static String	getCollectionPathRoot(String coll)
Map<String,String>	getCollectionProperties(String collection) Get collection properties for a given collection.
ZkConfigManager	getConfigManager()
Replica	getLeader(String collection, String shard)
Replica	getLeaderRetry(String collection, String shard) Get shard leader properties, with retry if none exist.
Replica	getLeaderRetry(String collection, String shard, int timeout) Get shard leader properties, with retry if none exist.
String	getLeaderUrl(String collection, String shard, int timeout)
List<ZkCoreNodeProps>	getReplicaProps(String collection, String shardId, String thisCoreNodeName)
List<ZkCoreNodeProps>	getReplicaProps(String collection, String shardId, String thisCoreNodeName, Replica.State mustMatchStateFilter)
List<ZkCoreNodeProps>	getReplicaProps(String collection, String shardId, String thisCoreNodeName, Replica.State mustMatchStateFilter, Replica.State mustNotMatchStateFilter)
List<ZkCoreNodeProps>	getReplicaProps(String collection, String shardId, String thisCoreNodeName, Replica.State mustMatchStateFilter, Replica.State mustNotMatchStateFilter, EnumSet<Replica.Type> acceptReplicaType)
ZkStateReader.ConfigData	getSecurityProps(boolean getFresh) Returns the content of /security.json from ZooKeeper as a Map If the files doesn't exist, it returns null.
static String	getShardLeadersElectPath(String collection, String shardId) Get path where shard leader elections ephemeral nodes are.
static String	getShardLeadersPath(String collection, String shardId) Get path where shard leader properties live in zookeeper.
Object	getUpdateLock()
SolrZkClient	getZkClient()
String	readConfigName(String collection) Returns config set name for collection.
void	registerCloudCollectionsListener(CloudCollectionsListener cloudCollectionsListener) Register a CloudCollectionsListener to be called when the set of collections within a cloud changes.
void	registerCollectionPropsWatcher(String collection, CollectionPropsWatcher propsWatcher)
void	registerCollectionStateWatcher(String collection, CollectionStateWatcher stateWatcher) Register a CollectionStateWatcher to be called when the state of a collection changes
void	registerCore(String collection) Notify this reader that a local Core is a member of a collection, and so that collection state should be watched.
void	registerLiveNodesListener(LiveNodesListener listener)
void	removeCloudCollectionsListener(CloudCollectionsListener cloudCollectionsListener) Remove a registered CloudCollectionsListener.
void	removeCollectionPropsWatcher(String collection, CollectionPropsWatcher watcher)
void	removeCollectionStateWatcher(String collection, CollectionStateWatcher watcher) Remove a watcher from a collection's watch list.
void	removeLiveNodesListener(LiveNodesListener listener)
void	unregisterCore(String collection) Notify this reader that a local core that is a member of a collection has been closed.
void	updateLiveNodes() Refresh the set of live nodes.
void	waitForState(String collection, long wait, TimeUnit unit, CollectionStatePredicate predicate) Block until a CollectionStatePredicate returns true, or the wait times out Note that the predicate may be called again even after it has returned true, so implementors should avoid changing state within the predicate call itself.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

STATE_UPDATE_DELAY

public static final int STATE_UPDATE_DELAY

BASE_URL_PROP

public static final String BASE_URL_PROP

See Also:

Constant Field Values

NODE_NAME_PROP

public static final String NODE_NAME_PROP

See Also:

Constant Field Values

CORE_NODE_NAME_PROP

public static final String CORE_NODE_NAME_PROP

See Also:

Constant Field Values

ROLES_PROP

public static final String ROLES_PROP

See Also:

Constant Field Values

STATE_PROP

public static final String STATE_PROP

See Also:

Constant Field Values

FORCE_SET_STATE_PROP

public static final String FORCE_SET_STATE_PROP

See Also:

Constant Field Values

CORE_NAME_PROP

public static final String CORE_NAME_PROP

SolrCore name.

See Also:

Constant Field Values

COLLECTION_PROP

public static final String COLLECTION_PROP

See Also:

Constant Field Values

ELECTION_NODE_PROP

public static final String ELECTION_NODE_PROP

See Also:

Constant Field Values

SHARD_ID_PROP

public static final String SHARD_ID_PROP

See Also:

Constant Field Values

REPLICA_PROP

public static final String REPLICA_PROP

See Also:

Constant Field Values

SHARD_RANGE_PROP

public static final String SHARD_RANGE_PROP

See Also:

Constant Field Values

SHARD_STATE_PROP

public static final String SHARD_STATE_PROP

See Also:

Constant Field Values

SHARD_PARENT_PROP

public static final String SHARD_PARENT_PROP

See Also:

Constant Field Values

NUM_SHARDS_PROP

public static final String NUM_SHARDS_PROP

See Also:

Constant Field Values

LEADER_PROP

public static final String LEADER_PROP

See Also:

Constant Field Values

SHARED_STORAGE_PROP

public static final String SHARED_STORAGE_PROP

See Also:

Constant Field Values

PROPERTY_PROP

public static final String PROPERTY_PROP

See Also:

Constant Field Values

PROPERTY_PROP_PREFIX

public static final String PROPERTY_PROP_PREFIX

See Also:

Constant Field Values

PROPERTY_VALUE_PROP

public static final String PROPERTY_VALUE_PROP

See Also:

Constant Field Values

MAX_AT_ONCE_PROP

public static final String MAX_AT_ONCE_PROP

See Also:

Constant Field Values

MAX_WAIT_SECONDS_PROP

public static final String MAX_WAIT_SECONDS_PROP

See Also:

Constant Field Values

STATE_TIMESTAMP_PROP

public static final String STATE_TIMESTAMP_PROP

See Also:

Constant Field Values

COLLECTIONS_ZKNODE

public static final String COLLECTIONS_ZKNODE

See Also:

Constant Field Values

LIVE_NODES_ZKNODE

public static final String LIVE_NODES_ZKNODE

See Also:

Constant Field Values

ALIASES

public static final String ALIASES

See Also:

Constant Field Values

CLUSTER_STATE

public static final String CLUSTER_STATE

See Also:

Constant Field Values

CLUSTER_PROPS

public static final String CLUSTER_PROPS

See Also:

Constant Field Values

COLLECTION_PROPS_ZKNODE

public static final String COLLECTION_PROPS_ZKNODE

See Also:

Constant Field Values

REJOIN_AT_HEAD_PROP

public static final String REJOIN_AT_HEAD_PROP

See Also:

Constant Field Values

SOLR_SECURITY_CONF_PATH

public static final String SOLR_SECURITY_CONF_PATH

See Also:

Constant Field Values

SOLR_AUTOSCALING_CONF_PATH

public static final String SOLR_AUTOSCALING_CONF_PATH

See Also:

Constant Field Values

SOLR_AUTOSCALING_EVENTS_PATH

public static final String SOLR_AUTOSCALING_EVENTS_PATH

See Also:

Constant Field Values

SOLR_AUTOSCALING_TRIGGER_STATE_PATH

public static final String SOLR_AUTOSCALING_TRIGGER_STATE_PATH

See Also:

Constant Field Values

SOLR_AUTOSCALING_NODE_ADDED_PATH

public static final String SOLR_AUTOSCALING_NODE_ADDED_PATH

See Also:

Constant Field Values

SOLR_AUTOSCALING_NODE_LOST_PATH

public static final String SOLR_AUTOSCALING_NODE_LOST_PATH

See Also:

Constant Field Values

REPLICATION_FACTOR

public static final String REPLICATION_FACTOR

See Also:

Constant Field Values

MAX_SHARDS_PER_NODE

public static final String MAX_SHARDS_PER_NODE

See Also:

Constant Field Values

AUTO_ADD_REPLICAS

public static final String AUTO_ADD_REPLICAS

See Also:

Constant Field Values

MAX_CORES_PER_NODE

public static final String MAX_CORES_PER_NODE

See Also:

Constant Field Values

PULL_REPLICAS

public static final String PULL_REPLICAS

See Also:

Constant Field Values

NRT_REPLICAS

public static final String NRT_REPLICAS

See Also:

Constant Field Values

TLOG_REPLICAS

public static final String TLOG_REPLICAS

See Also:

Constant Field Values

ROLES

public static final String ROLES

See Also:

Constant Field Values

CONFIGS_ZKNODE

public static final String CONFIGS_ZKNODE

See Also:

Constant Field Values

CONFIGNAME_PROP

public static final String CONFIGNAME_PROP

See Also:

Constant Field Values

LEGACY_CLOUD

public static final String LEGACY_CLOUD

See Also:

Constant Field Values

COLLECTION_DEF

@Deprecated
public static final String COLLECTION_DEF

Deprecated. use CollectionAdminParams.DEFAULTS instead.

See Also:

Constant Field Values

URL_SCHEME

public static final String URL_SCHEME

See Also:

Constant Field Values

REPLICA_TYPE

public static final String REPLICA_TYPE

See Also:

Constant Field Values

clusterState

protected volatile ClusterState clusterState

A view of the current state of all collections; combines all the different state sources into a single view.

LEADER_ELECT_ZKNODE

public static final String LEADER_ELECT_ZKNODE

See Also:

Constant Field Values

SHARD_LEADERS_ZKNODE

public static final String SHARD_LEADERS_ZKNODE

See Also:

Constant Field Values

ELECTION_NODE

public static final String ELECTION_NODE

See Also:

Constant Field Values

KNOWN_CLUSTER_PROPS

public static final Set<String> KNOWN_CLUSTER_PROPS

aliasesManager

public final ZkStateReader.AliasesManager aliasesManager

Access to the Aliases.

Constructor Detail

ZkStateReader

public ZkStateReader(SolrZkClient zkClient)

ZkStateReader

public ZkStateReader(SolrZkClient zkClient,
                     Runnable securityNodeListener)

ZkStateReader

public ZkStateReader(String zkServerAddress,
                     int zkClientTimeout,
                     int zkClientConnectTimeout)

Method Detail

getAutoScalingConfig

public AutoScalingConfig getAutoScalingConfig()
                                       throws org.apache.zookeeper.KeeperException,
                                              InterruptedException

Get current AutoScalingConfig.

Returns:

current configuration from autoscaling.json. NOTE: this data is retrieved from ZK on each call.

Throws:

org.apache.zookeeper.KeeperException

InterruptedException

getAutoScalingConfig

public AutoScalingConfig getAutoScalingConfig(org.apache.zookeeper.Watcher watcher)
                                       throws org.apache.zookeeper.KeeperException,
                                              InterruptedException

Get current AutoScalingConfig.

Parameters:

watcher - optional Watcher to set on a znode to watch for config changes.

Returns:

current configuration from autoscaling.json. NOTE: this data is retrieved from ZK on each call.

Throws:

org.apache.zookeeper.KeeperException

InterruptedException

readConfigName

public String readConfigName(String collection)

Returns config set name for collection.

Parameters:

collection - to return config set name for

getConfigManager

public ZkConfigManager getConfigManager()

forciblyRefreshAllClusterStateSlow

public void forciblyRefreshAllClusterStateSlow()
                                        throws org.apache.zookeeper.KeeperException,
                                               InterruptedException

Forcibly refresh cluster state from ZK. Do this only to avoid race conditions because it's expensive. It is cheaper to call forceUpdateCollection(String) on a single collection if you must.

Throws:

org.apache.zookeeper.KeeperException

InterruptedException

NOTE: This API is for internal purposes only and might change in incompatible ways in the next release.

forceUpdateCollection

public void forceUpdateCollection(String collection)
                           throws org.apache.zookeeper.KeeperException,
                                  InterruptedException

Forcibly refresh a collection's internal state from ZK. Try to avoid having to resort to this when a better design is possible.

Throws:

org.apache.zookeeper.KeeperException

InterruptedException

updateLiveNodes

public void updateLiveNodes()
                     throws org.apache.zookeeper.KeeperException,
                            InterruptedException

Refresh the set of live nodes.

Throws:

org.apache.zookeeper.KeeperException

InterruptedException

compareStateVersions

public Integer compareStateVersions(String coll,
                                    int version)

createClusterStateWatchersAndUpdate

public void createClusterStateWatchersAndUpdate()
                                         throws org.apache.zookeeper.KeeperException,
                                                InterruptedException

Throws:

org.apache.zookeeper.KeeperException

InterruptedException

registerCloudCollectionsListener

public void registerCloudCollectionsListener(CloudCollectionsListener cloudCollectionsListener)

Register a CloudCollectionsListener to be called when the set of collections within a cloud changes.

removeCloudCollectionsListener

public void removeCloudCollectionsListener(CloudCollectionsListener cloudCollectionsListener)

Remove a registered CloudCollectionsListener.

registerLiveNodesListener

public void registerLiveNodesListener(LiveNodesListener listener)

removeLiveNodesListener

public void removeLiveNodesListener(LiveNodesListener listener)

getClusterState

public ClusterState getClusterState()

Returns:

information about the cluster from ZooKeeper

getUpdateLock

public Object getUpdateLock()

close

public void close()

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

getLeaderUrl

public String getLeaderUrl(String collection,
                           String shard,
                           int timeout)
                    throws InterruptedException

Throws:

InterruptedException

getLeader

public Replica getLeader(String collection,
                         String shard)

getLeaderRetry

public Replica getLeaderRetry(String collection,
                              String shard)
                       throws InterruptedException

Get shard leader properties, with retry if none exist.

Throws:

InterruptedException

getLeaderRetry

public Replica getLeaderRetry(String collection,
                              String shard,
                              int timeout)
                       throws InterruptedException

Get shard leader properties, with retry if none exist.

Throws:

InterruptedException

getShardLeadersPath

public static String getShardLeadersPath(String collection,
                                         String shardId)

Get path where shard leader properties live in zookeeper.

getShardLeadersElectPath

public static String getShardLeadersElectPath(String collection,
                                              String shardId)

Get path where shard leader elections ephemeral nodes are.

getReplicaProps

public List<ZkCoreNodeProps> getReplicaProps(String collection,
                                             String shardId,
                                             String thisCoreNodeName)

getReplicaProps

public List<ZkCoreNodeProps> getReplicaProps(String collection,
                                             String shardId,
                                             String thisCoreNodeName,
                                             Replica.State mustMatchStateFilter)

getReplicaProps

public List<ZkCoreNodeProps> getReplicaProps(String collection,
                                             String shardId,
                                             String thisCoreNodeName,
                                             Replica.State mustMatchStateFilter,
                                             Replica.State mustNotMatchStateFilter)

getReplicaProps

public List<ZkCoreNodeProps> getReplicaProps(String collection,
                                             String shardId,
                                             String thisCoreNodeName,
                                             Replica.State mustMatchStateFilter,
                                             Replica.State mustNotMatchStateFilter,
                                             EnumSet<Replica.Type> acceptReplicaType)

getZkClient

public SolrZkClient getZkClient()

getClusterProperty

public <T> T getClusterProperty(String key,
                                T defaultValue)

Get a cluster property N.B. Cluster properties are updated via ZK watchers, and so may not necessarily be completely up-to-date. If you need to get the latest version, then use a ClusterProperties instance.

Type Parameters:

T - the type of the property

Parameters:

key - the property to read

defaultValue - a default value to use if no such property exists

Returns:

the cluster property, or a default if the property is not set

getClusterProperty

public <T> T getClusterProperty(List<String> keyPath,
                                T defaultValue)

Same as the above but allows a full json path as a list of parts

Parameters:

keyPath - path to the property example ["collectionDefauls", "numShards"]

defaultValue - a default value to use if no such property exists

Returns:

the cluster property, or a default if the property is not set

getClusterProperties

public Map<String,Object> getClusterProperties()

Get all cluster properties for this cluster N.B. Cluster properties are updated via ZK watchers, and so may not necessarily be completely up-to-date. If you need to get the latest version, then use a ClusterProperties instance.

Returns:

a Map of cluster properties

getCollectionProperties

public Map<String,String> getCollectionProperties(String collection)

Get collection properties for a given collection. If the collection is watched, simply return it from the cache, otherwise fetch it directly from zookeeper.

getSecurityProps

public ZkStateReader.ConfigData getSecurityProps(boolean getFresh)

Returns the content of /security.json from ZooKeeper as a Map If the files doesn't exist, it returns null.

getBaseUrlForNodeName

public String getBaseUrlForNodeName(String nodeName)

Returns the baseURL corresponding to a given node's nodeName -- NOTE: does not (currently) imply that the nodeName (or resulting baseURL) exists in the cluster.

WARNING: This API is experimental and might change in incompatible ways in the next release.

getCollectionLive

public static DocCollection getCollectionLive(ZkStateReader zkStateReader,
                                              String coll)

getCollectionPathRoot

public static String getCollectionPathRoot(String coll)

getCollectionPath

public static String getCollectionPath(String coll)

registerCore

public void registerCore(String collection)

Notify this reader that a local Core is a member of a collection, and so that collection state should be watched. Not a public API. This method should only be called from ZkController. The number of cores per-collection is tracked, and adding multiple cores from the same collection does not increase the number of watches.

Parameters:

collection - the collection that the core is a member of

See Also:

unregisterCore(String)

unregisterCore

public void unregisterCore(String collection)

Notify this reader that a local core that is a member of a collection has been closed. Not a public API. This method should only be called from ZkController. If no cores are registered for a collection, and there are no CollectionStateWatchers for that collection either, the collection watch will be removed.

Parameters:

collection - the collection that the core belongs to

registerCollectionStateWatcher

public void registerCollectionStateWatcher(String collection,
                                           CollectionStateWatcher stateWatcher)

Register a CollectionStateWatcher to be called when the state of a collection changes

waitForState

public void waitForState(String collection,
                         long wait,
                         TimeUnit unit,
                         CollectionStatePredicate predicate)
                  throws InterruptedException,
                         TimeoutException

Block until a CollectionStatePredicate returns true, or the wait times out Note that the predicate may be called again even after it has returned true, so implementors should avoid changing state within the predicate call itself.

Parameters:

collection - the collection to watch

wait - how long to wait

unit - the units of the wait parameter

predicate - the predicate to call on state changes

Throws:

InterruptedException - on interrupt

TimeoutException - on timeout

removeCollectionStateWatcher

public void removeCollectionStateWatcher(String collection,
                                         CollectionStateWatcher watcher)

Remove a watcher from a collection's watch list. This allows Zookeeper watches to be removed if there is no interest in the collection.

Parameters:

collection - the collection

watcher - the watcher

registerCollectionPropsWatcher

public void registerCollectionPropsWatcher(String collection,
                                           CollectionPropsWatcher propsWatcher)

removeCollectionPropsWatcher

public void removeCollectionPropsWatcher(String collection,
                                         CollectionPropsWatcher watcher)

getAliases

public Aliases getAliases()

Get an immutable copy of the present state of the aliases. References to this object should not be retained in any context where it will be important to know if aliases have changed.

Returns:

The current aliases, Aliases.EMPTY if not solr cloud, or no aliases have existed yet. Never returns null.