Package org.apache.solr.common.cloud
Class ZkStateReader
- java.lang.Object
-
- org.apache.solr.common.cloud.ZkStateReader
-
- All Implemented Interfaces:
Closeable
,AutoCloseable
,SolrCloseable
public class ZkStateReader extends Object implements SolrCloseable
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description class
ZkStateReader.AliasesManager
A class to manage the aliases instance, including watching for changes.static class
ZkStateReader.ConfigData
-
Field Summary
-
Constructor Summary
Constructors Constructor Description ZkStateReader(String zkServerAddress, int zkClientTimeout, int zkClientConnectTimeout)
ZkStateReader(SolrZkClient zkClient)
ZkStateReader(SolrZkClient zkClient, Runnable securityNodeListener)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description void
close()
Integer
compareStateVersions(String coll, int version)
void
createClusterStateWatchersAndUpdate()
void
forceUpdateCollection(String collection)
Forcibly refresh a collection's internal state from ZK.void
forciblyRefreshAllClusterStateSlow()
Forcibly refresh cluster state from ZK.Aliases
getAliases()
Get an immutable copy of the present state of the aliases.AutoScalingConfig
getAutoScalingConfig()
Get currentAutoScalingConfig
.AutoScalingConfig
getAutoScalingConfig(org.apache.zookeeper.Watcher watcher)
Get currentAutoScalingConfig
.String
getBaseUrlForNodeName(String nodeName)
Returns the baseURL corresponding to a given node's nodeName -- NOTE: does not (currently) imply that the nodeName (or resulting baseURL) exists in the cluster.Map<String,Object>
getClusterProperties()
Get all cluster properties for this cluster N.B.<T> T
getClusterProperty(String key, T defaultValue)
Get a cluster property N.B.<T> T
getClusterProperty(List<String> keyPath, T defaultValue)
Same as the above but allows a full json path as a list of partsClusterState
getClusterState()
static DocCollection
getCollectionLive(ZkStateReader zkStateReader, String coll)
static String
getCollectionPath(String coll)
static String
getCollectionPathRoot(String coll)
Map<String,String>
getCollectionProperties(String collection)
Get collection properties for a given collection.ZkConfigManager
getConfigManager()
Replica
getLeader(String collection, String shard)
Replica
getLeader(Set<String> liveNodes, DocCollection docCollection, String shard)
Replica
getLeaderRetry(String collection, String shard)
Get shard leader properties, with retry if none exist.Replica
getLeaderRetry(String collection, String shard, int timeout)
Get shard leader properties, with retry if none exist.String
getLeaderUrl(String collection, String shard, int timeout)
List<ZkCoreNodeProps>
getReplicaProps(String collection, String shardId, String thisCoreNodeName)
List<ZkCoreNodeProps>
getReplicaProps(String collection, String shardId, String thisCoreNodeName, Replica.State mustMatchStateFilter)
List<ZkCoreNodeProps>
getReplicaProps(String collection, String shardId, String thisCoreNodeName, Replica.State mustMatchStateFilter, Replica.State mustNotMatchStateFilter)
List<ZkCoreNodeProps>
getReplicaProps(String collection, String shardId, String thisCoreNodeName, Replica.State mustMatchStateFilter, Replica.State mustNotMatchStateFilter, EnumSet<Replica.Type> acceptReplicaType)
ZkStateReader.ConfigData
getSecurityProps(boolean getFresh)
Returns the content of /security.json from ZooKeeper as a Map If the files doesn't exist, it returns null.static String
getShardLeadersElectPath(String collection, String shardId)
Get path where shard leader elections ephemeral nodes are.static String
getShardLeadersPath(String collection, String shardId)
Get path where shard leader properties live in zookeeper.Object
getUpdateLock()
SolrZkClient
getZkClient()
boolean
isClosed()
String
readConfigName(String collection)
Returns config set name for collection.void
registerCloudCollectionsListener(CloudCollectionsListener cloudCollectionsListener)
Register a CloudCollectionsListener to be called when the set of collections within a cloud changes.void
registerCollectionPropsWatcher(String collection, CollectionPropsWatcher propsWatcher)
void
registerCollectionStateWatcher(String collection, CollectionStateWatcher stateWatcher)
Register a CollectionStateWatcher to be called when the state of a collection changesvoid
registerCore(String collection)
Notify this reader that a local Core is a member of a collection, and so that collection state should be watched.void
registerLiveNodesListener(LiveNodesListener listener)
void
removeCloudCollectionsListener(CloudCollectionsListener cloudCollectionsListener)
Remove a registered CloudCollectionsListener.void
removeCollectionPropsWatcher(String collection, CollectionPropsWatcher watcher)
void
removeCollectionStateWatcher(String collection, CollectionStateWatcher watcher)
Remove a watcher from a collection's watch list.void
removeLiveNodesListener(LiveNodesListener listener)
void
unregisterCore(String collection)
Notify this reader that a local core that is a member of a collection has been closed.void
updateLiveNodes()
Refresh the set of live nodes.void
waitForLiveNodes(long wait, TimeUnit unit, LiveNodesPredicate predicate)
Block until a LiveNodesStatePredicate returns true, or the wait times out Note that the predicate may be called again even after it has returned true, so implementors should avoid changing state within the predicate call itself.void
waitForState(String collection, long wait, TimeUnit unit, CollectionStatePredicate predicate)
Block until a CollectionStatePredicate returns true, or the wait times out Note that the predicate may be called again even after it has returned true, so implementors should avoid changing state within the predicate call itself.
-
-
-
Field Detail
-
STATE_UPDATE_DELAY
public static final int STATE_UPDATE_DELAY
-
BASE_URL_PROP
public static final String BASE_URL_PROP
- See Also:
- Constant Field Values
-
NODE_NAME_PROP
public static final String NODE_NAME_PROP
- See Also:
- Constant Field Values
-
CORE_NODE_NAME_PROP
public static final String CORE_NODE_NAME_PROP
- See Also:
- Constant Field Values
-
ROLES_PROP
public static final String ROLES_PROP
- See Also:
- Constant Field Values
-
STATE_PROP
public static final String STATE_PROP
- See Also:
- Constant Field Values
-
FORCE_SET_STATE_PROP
public static final String FORCE_SET_STATE_PROP
- See Also:
- Constant Field Values
-
CORE_NAME_PROP
public static final String CORE_NAME_PROP
SolrCore name.- See Also:
- Constant Field Values
-
COLLECTION_PROP
public static final String COLLECTION_PROP
- See Also:
- Constant Field Values
-
ELECTION_NODE_PROP
public static final String ELECTION_NODE_PROP
- See Also:
- Constant Field Values
-
SHARD_ID_PROP
public static final String SHARD_ID_PROP
- See Also:
- Constant Field Values
-
REPLICA_PROP
public static final String REPLICA_PROP
- See Also:
- Constant Field Values
-
SHARD_RANGE_PROP
public static final String SHARD_RANGE_PROP
- See Also:
- Constant Field Values
-
SHARD_STATE_PROP
public static final String SHARD_STATE_PROP
- See Also:
- Constant Field Values
-
SHARD_PARENT_PROP
public static final String SHARD_PARENT_PROP
- See Also:
- Constant Field Values
-
NUM_SHARDS_PROP
public static final String NUM_SHARDS_PROP
- See Also:
- Constant Field Values
-
LEADER_PROP
public static final String LEADER_PROP
- See Also:
- Constant Field Values
-
SHARED_STORAGE_PROP
public static final String SHARED_STORAGE_PROP
- See Also:
- Constant Field Values
-
PROPERTY_PROP
public static final String PROPERTY_PROP
- See Also:
- Constant Field Values
-
PROPERTY_PROP_PREFIX
public static final String PROPERTY_PROP_PREFIX
- See Also:
- Constant Field Values
-
PROPERTY_VALUE_PROP
public static final String PROPERTY_VALUE_PROP
- See Also:
- Constant Field Values
-
MAX_AT_ONCE_PROP
public static final String MAX_AT_ONCE_PROP
- See Also:
- Constant Field Values
-
MAX_WAIT_SECONDS_PROP
public static final String MAX_WAIT_SECONDS_PROP
- See Also:
- Constant Field Values
-
STATE_TIMESTAMP_PROP
public static final String STATE_TIMESTAMP_PROP
- See Also:
- Constant Field Values
-
COLLECTIONS_ZKNODE
public static final String COLLECTIONS_ZKNODE
- See Also:
- Constant Field Values
-
LIVE_NODES_ZKNODE
public static final String LIVE_NODES_ZKNODE
- See Also:
- Constant Field Values
-
ALIASES
public static final String ALIASES
- See Also:
- Constant Field Values
-
CLUSTER_STATE
public static final String CLUSTER_STATE
- See Also:
- Constant Field Values
-
CLUSTER_PROPS
public static final String CLUSTER_PROPS
- See Also:
- Constant Field Values
-
COLLECTION_PROPS_ZKNODE
public static final String COLLECTION_PROPS_ZKNODE
- See Also:
- Constant Field Values
-
REJOIN_AT_HEAD_PROP
public static final String REJOIN_AT_HEAD_PROP
- See Also:
- Constant Field Values
-
SOLR_SECURITY_CONF_PATH
public static final String SOLR_SECURITY_CONF_PATH
- See Also:
- Constant Field Values
-
SOLR_AUTOSCALING_CONF_PATH
public static final String SOLR_AUTOSCALING_CONF_PATH
- See Also:
- Constant Field Values
-
SOLR_AUTOSCALING_EVENTS_PATH
public static final String SOLR_AUTOSCALING_EVENTS_PATH
- See Also:
- Constant Field Values
-
SOLR_AUTOSCALING_TRIGGER_STATE_PATH
public static final String SOLR_AUTOSCALING_TRIGGER_STATE_PATH
- See Also:
- Constant Field Values
-
SOLR_AUTOSCALING_NODE_ADDED_PATH
public static final String SOLR_AUTOSCALING_NODE_ADDED_PATH
- See Also:
- Constant Field Values
-
SOLR_AUTOSCALING_NODE_LOST_PATH
public static final String SOLR_AUTOSCALING_NODE_LOST_PATH
- See Also:
- Constant Field Values
-
REPLICATION_FACTOR
public static final String REPLICATION_FACTOR
- See Also:
- Constant Field Values
-
MAX_SHARDS_PER_NODE
public static final String MAX_SHARDS_PER_NODE
- See Also:
- Constant Field Values
-
AUTO_ADD_REPLICAS
public static final String AUTO_ADD_REPLICAS
- See Also:
- Constant Field Values
-
MAX_CORES_PER_NODE
public static final String MAX_CORES_PER_NODE
- See Also:
- Constant Field Values
-
PULL_REPLICAS
public static final String PULL_REPLICAS
- See Also:
- Constant Field Values
-
NRT_REPLICAS
public static final String NRT_REPLICAS
- See Also:
- Constant Field Values
-
TLOG_REPLICAS
public static final String TLOG_REPLICAS
- See Also:
- Constant Field Values
-
READ_ONLY
public static final String READ_ONLY
- See Also:
- Constant Field Values
-
ROLES
public static final String ROLES
- See Also:
- Constant Field Values
-
CONFIGS_ZKNODE
public static final String CONFIGS_ZKNODE
- See Also:
- Constant Field Values
-
CONFIGNAME_PROP
public static final String CONFIGNAME_PROP
- See Also:
- Constant Field Values
-
LEGACY_CLOUD
public static final String LEGACY_CLOUD
- See Also:
- Constant Field Values
-
COLLECTION_DEF
@Deprecated public static final String COLLECTION_DEF
Deprecated.useCollectionAdminParams.DEFAULTS
instead.- See Also:
- Constant Field Values
-
URL_SCHEME
public static final String URL_SCHEME
- See Also:
- Constant Field Values
-
REPLICA_TYPE
public static final String REPLICA_TYPE
- See Also:
- Constant Field Values
-
clusterState
protected volatile ClusterState clusterState
A view of the current state of all collections; combines all the different state sources into a single view.
-
LEADER_ELECT_ZKNODE
public static final String LEADER_ELECT_ZKNODE
- See Also:
- Constant Field Values
-
SHARD_LEADERS_ZKNODE
public static final String SHARD_LEADERS_ZKNODE
- See Also:
- Constant Field Values
-
ELECTION_NODE
public static final String ELECTION_NODE
- See Also:
- Constant Field Values
-
aliasesManager
public final ZkStateReader.AliasesManager aliasesManager
Access to theAliases
.
-
-
Constructor Detail
-
ZkStateReader
public ZkStateReader(SolrZkClient zkClient)
-
ZkStateReader
public ZkStateReader(SolrZkClient zkClient, Runnable securityNodeListener)
-
ZkStateReader
public ZkStateReader(String zkServerAddress, int zkClientTimeout, int zkClientConnectTimeout)
-
-
Method Detail
-
getAutoScalingConfig
public AutoScalingConfig getAutoScalingConfig() throws org.apache.zookeeper.KeeperException, InterruptedException
Get currentAutoScalingConfig
.- Returns:
- current configuration from
autoscaling.json
. NOTE: this data is retrieved from ZK on each call. - Throws:
org.apache.zookeeper.KeeperException
InterruptedException
-
getAutoScalingConfig
public AutoScalingConfig getAutoScalingConfig(org.apache.zookeeper.Watcher watcher) throws org.apache.zookeeper.KeeperException, InterruptedException
Get currentAutoScalingConfig
.- Parameters:
watcher
- optionalWatcher
to set on a znode to watch for config changes.- Returns:
- current configuration from
autoscaling.json
. NOTE: this data is retrieved from ZK on each call. - Throws:
org.apache.zookeeper.KeeperException
InterruptedException
-
readConfigName
public String readConfigName(String collection)
Returns config set name for collection.- Parameters:
collection
- to return config set name for
-
getConfigManager
public ZkConfigManager getConfigManager()
-
forciblyRefreshAllClusterStateSlow
public void forciblyRefreshAllClusterStateSlow() throws org.apache.zookeeper.KeeperException, InterruptedException
Forcibly refresh cluster state from ZK. Do this only to avoid race conditions because it's expensive. It is cheaper to callforceUpdateCollection(String)
on a single collection if you must.- Throws:
org.apache.zookeeper.KeeperException
InterruptedException
- NOTE: This API is for internal purposes only and might change in incompatible ways in the next release.
-
forceUpdateCollection
public void forceUpdateCollection(String collection) throws org.apache.zookeeper.KeeperException, InterruptedException
Forcibly refresh a collection's internal state from ZK. Try to avoid having to resort to this when a better design is possible.- Throws:
org.apache.zookeeper.KeeperException
InterruptedException
-
updateLiveNodes
public void updateLiveNodes() throws org.apache.zookeeper.KeeperException, InterruptedException
Refresh the set of live nodes.- Throws:
org.apache.zookeeper.KeeperException
InterruptedException
-
createClusterStateWatchersAndUpdate
public void createClusterStateWatchersAndUpdate() throws org.apache.zookeeper.KeeperException, InterruptedException
- Throws:
org.apache.zookeeper.KeeperException
InterruptedException
-
registerCloudCollectionsListener
public void registerCloudCollectionsListener(CloudCollectionsListener cloudCollectionsListener)
Register a CloudCollectionsListener to be called when the set of collections within a cloud changes.
-
removeCloudCollectionsListener
public void removeCloudCollectionsListener(CloudCollectionsListener cloudCollectionsListener)
Remove a registered CloudCollectionsListener.
-
registerLiveNodesListener
public void registerLiveNodesListener(LiveNodesListener listener)
-
removeLiveNodesListener
public void removeLiveNodesListener(LiveNodesListener listener)
-
getClusterState
public ClusterState getClusterState()
- Returns:
- information about the cluster from ZooKeeper
-
getUpdateLock
public Object getUpdateLock()
-
close
public void close()
- Specified by:
close
in interfaceAutoCloseable
- Specified by:
close
in interfaceCloseable
-
isClosed
public boolean isClosed()
- Specified by:
isClosed
in interfaceSolrCloseable
-
getLeaderUrl
public String getLeaderUrl(String collection, String shard, int timeout) throws InterruptedException
- Throws:
InterruptedException
-
getLeader
public Replica getLeader(Set<String> liveNodes, DocCollection docCollection, String shard)
-
getLeaderRetry
public Replica getLeaderRetry(String collection, String shard) throws InterruptedException
Get shard leader properties, with retry if none exist.- Throws:
InterruptedException
-
getLeaderRetry
public Replica getLeaderRetry(String collection, String shard, int timeout) throws InterruptedException
Get shard leader properties, with retry if none exist.- Throws:
InterruptedException
-
getShardLeadersPath
public static String getShardLeadersPath(String collection, String shardId)
Get path where shard leader properties live in zookeeper.
-
getShardLeadersElectPath
public static String getShardLeadersElectPath(String collection, String shardId)
Get path where shard leader elections ephemeral nodes are.
-
getReplicaProps
public List<ZkCoreNodeProps> getReplicaProps(String collection, String shardId, String thisCoreNodeName)
-
getReplicaProps
public List<ZkCoreNodeProps> getReplicaProps(String collection, String shardId, String thisCoreNodeName, Replica.State mustMatchStateFilter)
-
getReplicaProps
public List<ZkCoreNodeProps> getReplicaProps(String collection, String shardId, String thisCoreNodeName, Replica.State mustMatchStateFilter, Replica.State mustNotMatchStateFilter)
-
getReplicaProps
public List<ZkCoreNodeProps> getReplicaProps(String collection, String shardId, String thisCoreNodeName, Replica.State mustMatchStateFilter, Replica.State mustNotMatchStateFilter, EnumSet<Replica.Type> acceptReplicaType)
-
getZkClient
public SolrZkClient getZkClient()
-
getClusterProperty
public <T> T getClusterProperty(String key, T defaultValue)
Get a cluster property N.B. Cluster properties are updated via ZK watchers, and so may not necessarily be completely up-to-date. If you need to get the latest version, then use aClusterProperties
instance.- Type Parameters:
T
- the type of the property- Parameters:
key
- the property to readdefaultValue
- a default value to use if no such property exists- Returns:
- the cluster property, or a default if the property is not set
-
getClusterProperty
public <T> T getClusterProperty(List<String> keyPath, T defaultValue)
Same as the above but allows a full json path as a list of parts- Parameters:
keyPath
- path to the property example ["collectionDefauls", "numShards"]defaultValue
- a default value to use if no such property exists- Returns:
- the cluster property, or a default if the property is not set
-
getClusterProperties
public Map<String,Object> getClusterProperties()
Get all cluster properties for this cluster N.B. Cluster properties are updated via ZK watchers, and so may not necessarily be completely up-to-date. If you need to get the latest version, then use aClusterProperties
instance.- Returns:
- a Map of cluster properties
-
getCollectionProperties
public Map<String,String> getCollectionProperties(String collection)
Get collection properties for a given collection. If the collection is watched, simply return it from the cache, otherwise fetch it directly from zookeeper.
-
getSecurityProps
public ZkStateReader.ConfigData getSecurityProps(boolean getFresh)
Returns the content of /security.json from ZooKeeper as a Map If the files doesn't exist, it returns null.
-
getBaseUrlForNodeName
public String getBaseUrlForNodeName(String nodeName)
Returns the baseURL corresponding to a given node's nodeName -- NOTE: does not (currently) imply that the nodeName (or resulting baseURL) exists in the cluster.- WARNING: This API is experimental and might change in incompatible ways in the next release.
-
getCollectionLive
public static DocCollection getCollectionLive(ZkStateReader zkStateReader, String coll)
-
registerCore
public void registerCore(String collection)
Notify this reader that a local Core is a member of a collection, and so that collection state should be watched. Not a public API. This method should only be called from ZkController. The number of cores per-collection is tracked, and adding multiple cores from the same collection does not increase the number of watches.- Parameters:
collection
- the collection that the core is a member of- See Also:
unregisterCore(String)
-
unregisterCore
public void unregisterCore(String collection)
Notify this reader that a local core that is a member of a collection has been closed. Not a public API. This method should only be called from ZkController. If no cores are registered for a collection, and there are noCollectionStateWatcher
s for that collection either, the collection watch will be removed.- Parameters:
collection
- the collection that the core belongs to
-
registerCollectionStateWatcher
public void registerCollectionStateWatcher(String collection, CollectionStateWatcher stateWatcher)
Register a CollectionStateWatcher to be called when the state of a collection changes
-
waitForState
public void waitForState(String collection, long wait, TimeUnit unit, CollectionStatePredicate predicate) throws InterruptedException, TimeoutException
Block until a CollectionStatePredicate returns true, or the wait times out Note that the predicate may be called again even after it has returned true, so implementors should avoid changing state within the predicate call itself.- Parameters:
collection
- the collection to watchwait
- how long to waitunit
- the units of the wait parameterpredicate
- the predicate to call on state changes- Throws:
InterruptedException
- on interruptTimeoutException
- on timeout
-
waitForLiveNodes
public void waitForLiveNodes(long wait, TimeUnit unit, LiveNodesPredicate predicate) throws InterruptedException, TimeoutException
Block until a LiveNodesStatePredicate returns true, or the wait times out Note that the predicate may be called again even after it has returned true, so implementors should avoid changing state within the predicate call itself.- Parameters:
wait
- how long to waitunit
- the units of the wait parameterpredicate
- the predicate to call on state changes- Throws:
InterruptedException
- on interruptTimeoutException
- on timeout
-
removeCollectionStateWatcher
public void removeCollectionStateWatcher(String collection, CollectionStateWatcher watcher)
Remove a watcher from a collection's watch list. This allows Zookeeper watches to be removed if there is no interest in the collection.- Parameters:
collection
- the collectionwatcher
- the watcher
-
registerCollectionPropsWatcher
public void registerCollectionPropsWatcher(String collection, CollectionPropsWatcher propsWatcher)
-
removeCollectionPropsWatcher
public void removeCollectionPropsWatcher(String collection, CollectionPropsWatcher watcher)
-
getAliases
public Aliases getAliases()
Get an immutable copy of the present state of the aliases. References to this object should not be retained in any context where it will be important to know if aliases have changed.- Returns:
- The current aliases, Aliases.EMPTY if not solr cloud, or no aliases have existed yet. Never returns null.
-
-