Class SimClusterStateProvider
- java.lang.Object
-
- org.apache.solr.cloud.autoscaling.sim.SimClusterStateProvider
-
- All Implemented Interfaces:
Closeable,AutoCloseable,ClusterStateProvider,SolrCloseable
public class SimClusterStateProvider extends Object implements ClusterStateProvider
SimulatedClusterStateProvider.The following behaviors are supported:
- using autoscaling policy for replica placements
- maintaining and up-to-date list of /live_nodes and nodeAdded / nodeLost markers
- running a simulated leader election on collection changes (with throttling), when needed
- maintaining an up-to-date /clusterstate.json (single file format), which also tracks replica states, leader election changes, replica property changes, etc. Note: this file is only written, but never read by the framework!
- maintaining an up-to-date /clusterprops.json. Note: this file is only written, but never read by the framework!
-
-
Field Summary
Fields Modifier and Type Field Description static longDEFAULT_DOC_SIZE_BYTES
-
Constructor Summary
Constructors Constructor Description SimClusterStateProvider(LiveNodesSet liveNodes, SimCloudManager cloudManager)The instance needs to be initialized using thesim*methods in order to ensure proper behavior, otherwise it will behave as a cluster with zero replicas.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description voidclose()voidconnect()voidcopyFrom(ClusterStateProvider other)voidcreateSystemCollection()Map<String,String>getAliasProperties(String alias)Map<String,Object>getClusterProperties()ClusterStategetClusterState()Set<String>getLiveNodes()StringgetPolicyNameByCollection(String coll)ClusterState.CollectionRefgetState(String collection)booleanisClosed()List<String>resolveAlias(String alias)voidsimAddNode(String nodeId)Add a new node to the cluster.voidsimAddReplica(String nodeId, ReplicaInfo replicaInfo, boolean runLeaderElection)Add a replica.voidsimAddReplica(ZkNodeProps message, NamedList results)Add a new replica.voidsimCreateCollection(ZkNodeProps props, NamedList results)Create a new collection.voidsimCreateShard(ZkNodeProps message, NamedList results)Create a new shard.voidsimDeleteAllCollections()Remove all collections.voidsimDeleteCollection(String collection, String async, NamedList results)Delete a collectionvoidsimDeleteShard(ZkNodeProps message, NamedList results)Delete a shard.Map<String,Map<String,Object>>simGetCollectionStats()StringsimGetOverseerLeader()StringsimGetRandomNode()Get random node id.StringsimGetRandomNode(Random random)Get random node id.List<ReplicaInfo>simGetReplicaInfos(String node)Return all replica infos for a node.List<ReplicaInfo>simGetReplicaInfos(String collection, String shard)List<String>simListCollections()List collections.voidsimMoveReplica(ZkNodeProps message, NamedList results)Move replica.QueryResponsesimQuery(QueryRequest req)voidsimRemoveDeadNodes()Remove all replica information related to dead nodes.booleansimRemoveNode(String nodeId)Remove node from a cluster.voidsimRemoveReplica(String nodeId, String coreNodeName)Remove replica.voidsimResetLeaderThrottles()Reset the leader election throttles.booleansimRestoreNode(String nodeId)Restore a previously removed node.voidsimSetClusterProperties(Map<String,Object> properties)Set all cluster properties.voidsimSetClusterProperty(String key, Object value)Set a cluster property.voidsimSetClusterState(ClusterState initialState)Initialize from an existing cluster statevoidsimSetCollectionProperties(String coll, Map<String,Object> properties)Set collection properties.voidsimSetCollectionProperty(String coll, String key, String value)Set collection property.voidsimSetCollectionValue(String collection, String key, Object value)Set per-collection value (eg.voidsimSetCollectionValue(String collection, String key, Object value, boolean delta, boolean divide)Set per-collection value (eg.voidsimSetReplicaValues(String node, Map<String,Map<String,List<ReplicaInfo>>> source, boolean overwrite)voidsimSetShardValue(String collection, String shard, String key, Object value)Set per-collection value (eg.voidsimSetShardValue(String collection, String shard, String key, Object value, boolean delta, boolean divide)Set per-collection value (eg.voidsimSetSliceProperties(String coll, String slice, Map<String,Object> properties)Set slice properties.voidsimSplitShard(ZkNodeProps message, NamedList results)Split a shard.UpdateResponsesimUpdate(UpdateRequest req)Simulate an update by modifying replica metrics.-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.apache.solr.client.solrj.impl.ClusterStateProvider
getClusterProperty, getClusterProperty, getCollection, isRoutedAlias, resolveSimpleAlias
-
-
-
-
Field Detail
-
DEFAULT_DOC_SIZE_BYTES
public static final long DEFAULT_DOC_SIZE_BYTES
- See Also:
- Constant Field Values
-
-
Constructor Detail
-
SimClusterStateProvider
public SimClusterStateProvider(LiveNodesSet liveNodes, SimCloudManager cloudManager) throws Exception
The instance needs to be initialized using thesim*methods in order to ensure proper behavior, otherwise it will behave as a cluster with zero replicas.- Throws:
Exception
-
-
Method Detail
-
copyFrom
public void copyFrom(ClusterStateProvider other) throws Exception
- Throws:
Exception
-
simSetClusterState
public void simSetClusterState(ClusterState initialState) throws Exception
Initialize from an existing cluster state- Parameters:
initialState- initial cluster state- Throws:
Exception
-
simResetLeaderThrottles
public void simResetLeaderThrottles()
Reset the leader election throttles.
-
simGetRandomNode
public String simGetRandomNode()
Get random node id.- Returns:
- one of the live nodes
-
simGetRandomNode
public String simGetRandomNode(Random random)
Get random node id.- Parameters:
random- instance of random.- Returns:
- one of the live nodes
-
simAddNode
public void simAddNode(String nodeId) throws Exception
Add a new node to the cluster.- Parameters:
nodeId- unique node id- Throws:
Exception
-
simRemoveNode
public boolean simRemoveNode(String nodeId) throws Exception
Remove node from a cluster. This is equivalent to a situation when a node is lost. All replicas that were assigned to this node are marked as DOWN.- Parameters:
nodeId- node id- Returns:
- true if a node existed and was removed
- Throws:
Exception
-
simRemoveDeadNodes
public void simRemoveDeadNodes() throws ExceptionRemove all replica information related to dead nodes.- Throws:
Exception
-
simGetOverseerLeader
public String simGetOverseerLeader()
-
simRestoreNode
public boolean simRestoreNode(String nodeId) throws Exception
Restore a previously removed node. This also simulates a short replica recovery state.- Parameters:
nodeId- node id to restore- Returns:
- true when this operation restored any replicas, false otherwise (empty node).
- Throws:
Exception
-
simAddReplica
public void simAddReplica(ZkNodeProps message, NamedList results) throws Exception
Add a new replica. Note that if any details of a replica (node, coreNodeName, SolrCore name, etc) are missing they will be filled in using the policy framework.- Parameters:
message- replica detailsresults- result of the operation- Throws:
Exception
-
simAddReplica
public void simAddReplica(String nodeId, ReplicaInfo replicaInfo, boolean runLeaderElection) throws Exception
Add a replica. Note that all details of the replica must be present here, including node, coreNodeName and SolrCore name.- Parameters:
nodeId- node id where the replica will be addedreplicaInfo- replica inforunLeaderElection- if true then run a leader election after adding the replica.- Throws:
Exception
-
simRemoveReplica
public void simRemoveReplica(String nodeId, String coreNodeName) throws Exception
Remove replica.- Parameters:
nodeId- node idcoreNodeName- coreNodeName- Throws:
Exception
-
simCreateCollection
public void simCreateCollection(ZkNodeProps props, NamedList results) throws Exception
Create a new collection. This operation uses policy framework for node and replica assignments.- Parameters:
props- collection detailsresults- results of the operation.- Throws:
Exception
-
simDeleteCollection
public void simDeleteCollection(String collection, String async, NamedList results) throws Exception
Delete a collection- Parameters:
collection- collection nameasync- async idresults- results of the operation- Throws:
Exception
-
simDeleteAllCollections
public void simDeleteAllCollections() throws ExceptionRemove all collections.- Throws:
Exception
-
simMoveReplica
public void simMoveReplica(ZkNodeProps message, NamedList results) throws Exception
Move replica. This uses a similar algorithm asMoveReplicaCmdmoveNormalReplica(...)method.- Parameters:
message- operation detailsresults- operation results.- Throws:
Exception
-
simCreateShard
public void simCreateShard(ZkNodeProps message, NamedList results) throws Exception
Create a new shard. This uses a similar algorithm asCreateShardCmd.- Parameters:
message- operation detailsresults- operation results- Throws:
Exception
-
simSplitShard
public void simSplitShard(ZkNodeProps message, NamedList results) throws Exception
Split a shard. This uses a similar algorithm asSplitShardCmd, including simulating its quirks, and leaving the original parent slice in place.- Parameters:
message- operation detailsresults- operation results.- Throws:
Exception
-
simDeleteShard
public void simDeleteShard(ZkNodeProps message, NamedList results) throws Exception
Delete a shard. This uses a similar algorithm asDeleteShardCmd- Parameters:
message- operation detailsresults- operation results- Throws:
Exception
-
createSystemCollection
public void createSystemCollection() throws IOException- Throws:
IOException
-
simUpdate
public UpdateResponse simUpdate(UpdateRequest req) throws SolrException, InterruptedException, IOException
Simulate an update by modifying replica metrics. The following core metrics are updated:SEARCHER.searcher.numDocs- increased by added docs, decreased by deleteById and deleteByQuerySEARCHER.searcher.deletedDocs- decreased by deleteById and deleteByQuery by up tonumDocsSEARCHER.searcher.maxDoc- always increased by the number of added docs.
IMPORTANT limitations:
- document replacements are always counted as new docs
- delete by ID always succeeds (unless numDocs == 0)
- deleteByQuery is not supported unless the query is
*:*
- Parameters:
req- update request. This request MUST have thecollectionparam set.- Returns:
UpdateResponse- Throws:
SolrException- on errors, such as nonexistent collection or unsupported deleteByQueryInterruptedExceptionIOException
-
simQuery
public QueryResponse simQuery(QueryRequest req) throws SolrException, InterruptedException, IOException
-
simSetClusterProperties
public void simSetClusterProperties(Map<String,Object> properties) throws Exception
Set all cluster properties. This also updates the clusterprops.json data inDistribStateManager- Parameters:
properties- properties to set- Throws:
Exception
-
simSetClusterProperty
public void simSetClusterProperty(String key, Object value) throws Exception
Set a cluster property. This also updates the clusterprops.json data inDistribStateManager- Parameters:
key- property namevalue- property value- Throws:
Exception
-
simSetCollectionProperties
public void simSetCollectionProperties(String coll, Map<String,Object> properties) throws Exception
Set collection properties.- Parameters:
coll- collection nameproperties- properties- Throws:
Exception
-
simSetCollectionProperty
public void simSetCollectionProperty(String coll, String key, String value) throws Exception
Set collection property.- Parameters:
coll- collection namekey- property namevalue- property value- Throws:
Exception
-
simSetSliceProperties
public void simSetSliceProperties(String coll, String slice, Map<String,Object> properties) throws Exception
Set slice properties.- Parameters:
coll- collection nameslice- slice nameproperties- slice properties- Throws:
Exception
-
simSetCollectionValue
public void simSetCollectionValue(String collection, String key, Object value) throws Exception
Set per-collection value (eg. a metric). This value will be applied to each replica.- Parameters:
collection- collection namekey- property namevalue- property value- Throws:
Exception
-
simSetCollectionValue
public void simSetCollectionValue(String collection, String key, Object value, boolean delta, boolean divide) throws Exception
Set per-collection value (eg. a metric). This value will be applied to each replica.
-
simSetShardValue
public void simSetShardValue(String collection, String shard, String key, Object value) throws Exception
Set per-collection value (eg. a metric). This value will be applied to each replica in a selected shard.- Parameters:
collection- collection nameshard- shard name. If null then all shards will be affected.key- property namevalue- property value- Throws:
Exception
-
simSetShardValue
public void simSetShardValue(String collection, String shard, String key, Object value, boolean delta, boolean divide) throws Exception
Set per-collection value (eg. a metric). This value will be applied to each replica in a selected shard.- Parameters:
collection- collection nameshard- shard name. If null then all shards will be affected.key- property namevalue- property valuedelta- if true then treat the numeric value as delta to add to the existing value (or set the value to delta if missing)divide- if the value is aNumberand this is true, then the value will be evenly divided by the number of replicas.- Throws:
Exception
-
simSetReplicaValues
public void simSetReplicaValues(String node, Map<String,Map<String,List<ReplicaInfo>>> source, boolean overwrite)
-
simGetReplicaInfos
public List<ReplicaInfo> simGetReplicaInfos(String node)
Return all replica infos for a node.- Parameters:
node- node id- Returns:
- copy of the list of replicas on that node, or empty list if none
-
simGetReplicaInfos
public List<ReplicaInfo> simGetReplicaInfos(String collection, String shard)
-
simListCollections
public List<String> simListCollections() throws InterruptedException
List collections.- Returns:
- list of existing collections.
- Throws:
InterruptedException
-
simGetCollectionStats
public Map<String,Map<String,Object>> simGetCollectionStats() throws IOException, InterruptedException
- Throws:
IOExceptionInterruptedException
-
getState
public ClusterState.CollectionRef getState(String collection)
- Specified by:
getStatein interfaceClusterStateProvider
-
getLiveNodes
public Set<String> getLiveNodes()
- Specified by:
getLiveNodesin interfaceClusterStateProvider
-
resolveAlias
public List<String> resolveAlias(String alias)
- Specified by:
resolveAliasin interfaceClusterStateProvider
-
getAliasProperties
public Map<String,String> getAliasProperties(String alias)
- Specified by:
getAliasPropertiesin interfaceClusterStateProvider
-
getClusterState
public ClusterState getClusterState() throws IOException
- Specified by:
getClusterStatein interfaceClusterStateProvider- Throws:
IOException
-
getClusterProperties
public Map<String,Object> getClusterProperties()
- Specified by:
getClusterPropertiesin interfaceClusterStateProvider
-
getPolicyNameByCollection
public String getPolicyNameByCollection(String coll)
- Specified by:
getPolicyNameByCollectionin interfaceClusterStateProvider
-
connect
public void connect()
- Specified by:
connectin interfaceClusterStateProvider
-
close
public void close() throws IOException- Specified by:
closein interfaceAutoCloseable- Specified by:
closein interfaceCloseable- Throws:
IOException
-
isClosed
public boolean isClosed()
- Specified by:
isClosedin interfaceSolrCloseable
-
-