Class SimClusterStateProvider
- java.lang.Object
-
- org.apache.solr.cloud.autoscaling.sim.SimClusterStateProvider
-
- All Implemented Interfaces:
Closeable
,AutoCloseable
,ClusterStateProvider
,SolrCloseable
public class SimClusterStateProvider extends Object implements ClusterStateProvider
SimulatedClusterStateProvider
.The following behaviors are supported:
- using autoscaling policy for replica placements
- maintaining and up-to-date list of /live_nodes and nodeAdded / nodeLost markers
- running a simulated leader election on collection changes (with throttling), when needed
- maintaining an up-to-date /clusterstate.json (single file format), which also tracks replica states, leader election changes, replica property changes, etc. Note: this file is only written, but never read by the framework!
- maintaining an up-to-date /clusterprops.json. Note: this file is only written, but never read by the framework!
-
-
Field Summary
Fields Modifier and Type Field Description static long
DEFAULT_DOC_SIZE_BYTES
-
Constructor Summary
Constructors Constructor Description SimClusterStateProvider(LiveNodesSet liveNodes, SimCloudManager cloudManager)
The instance needs to be initialized using thesim*
methods in order to ensure proper behavior, otherwise it will behave as a cluster with zero replicas.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
close()
void
connect()
void
copyFrom(ClusterStateProvider other)
void
createSystemCollection()
Map<String,String>
getAliasProperties(String alias)
Map<String,Object>
getClusterProperties()
ClusterState
getClusterState()
Set<String>
getLiveNodes()
String
getPolicyNameByCollection(String coll)
ClusterState.CollectionRef
getState(String collection)
boolean
isClosed()
List<String>
resolveAlias(String alias)
void
simAddNode(String nodeId)
Add a new node to the cluster.void
simAddReplica(String nodeId, ReplicaInfo replicaInfo, boolean runLeaderElection)
Add a replica.void
simAddReplica(ZkNodeProps message, NamedList results)
Add a new replica.void
simCreateCollection(ZkNodeProps props, NamedList results)
Create a new collection.void
simCreateShard(ZkNodeProps message, NamedList results)
Create a new shard.void
simDeleteAllCollections()
Remove all collections.void
simDeleteCollection(String collection, String async, NamedList results)
Delete a collectionvoid
simDeleteShard(ZkNodeProps message, NamedList results)
Delete a shard.Map<String,Map<String,Object>>
simGetCollectionStats()
String
simGetOverseerLeader()
String
simGetRandomNode()
Get random node id.String
simGetRandomNode(Random random)
Get random node id.ReplicaInfo
simGetReplicaInfo(String collection, String coreNode)
List<ReplicaInfo>
simGetReplicaInfos(String node)
Return all replica infos for a node.List<ReplicaInfo>
simGetReplicaInfos(String collection, String shard)
List<String>
simListCollections()
List collections.void
simMoveReplica(ZkNodeProps message, NamedList results)
Move replica.QueryResponse
simQuery(QueryRequest req)
void
simRemoveDeadNodes()
Remove all replica information related to dead nodes.boolean
simRemoveNode(String nodeId)
Remove node from a cluster.void
simRemoveReplica(String nodeId, String collection, String coreNodeName)
Remove replica.void
simResetLeaderThrottles()
Reset the leader election throttles.boolean
simRestoreNode(String nodeId)
Restore a previously removed node.void
simSetClusterProperties(Map<String,Object> properties)
Set all cluster properties.void
simSetClusterProperty(String key, Object value)
Set a cluster property.void
simSetClusterState(ClusterState initialState)
Initialize from an existing cluster statevoid
simSetCollectionProperties(String coll, Map<String,Object> properties)
Set collection properties.void
simSetCollectionProperty(String coll, String key, String value)
Set collection property.void
simSetCollectionValue(String collection, String key, Object value)
Set per-collection value (eg.void
simSetCollectionValue(String collection, String key, Object value, boolean delta, boolean divide)
Set per-collection value (eg.void
simSetOpDelays(String collection, Map<String,Long> delays)
void
simSetReplicaValues(String node, Map<String,Map<String,List<ReplicaInfo>>> source, boolean overwrite)
void
simSetShardValue(String collection, String shard, String key, Object value)
Set per-collection value (eg.void
simSetShardValue(String collection, String shard, String key, Object value, boolean delta, boolean divide)
Set per-collection value (eg.void
simSetSliceProperties(String coll, String slice, Map<String,Object> properties)
Set slice properties.void
simSplitShard(ZkNodeProps message, NamedList results)
Split a shard.UpdateResponse
simUpdate(UpdateRequest req)
Simulate an update by modifying replica metrics.-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.apache.solr.client.solrj.impl.ClusterStateProvider
getClusterProperty, getClusterProperty, getCollection, isRoutedAlias, resolveSimpleAlias
-
-
-
-
Field Detail
-
DEFAULT_DOC_SIZE_BYTES
public static final long DEFAULT_DOC_SIZE_BYTES
- See Also:
- Constant Field Values
-
-
Constructor Detail
-
SimClusterStateProvider
public SimClusterStateProvider(LiveNodesSet liveNodes, SimCloudManager cloudManager) throws Exception
The instance needs to be initialized using thesim*
methods in order to ensure proper behavior, otherwise it will behave as a cluster with zero replicas.- Throws:
Exception
-
-
Method Detail
-
copyFrom
public void copyFrom(ClusterStateProvider other) throws Exception
- Throws:
Exception
-
simSetClusterState
public void simSetClusterState(ClusterState initialState) throws Exception
Initialize from an existing cluster state- Parameters:
initialState
- initial cluster state- Throws:
Exception
-
simResetLeaderThrottles
public void simResetLeaderThrottles()
Reset the leader election throttles.
-
simGetRandomNode
public String simGetRandomNode()
Get random node id.- Returns:
- one of the live nodes
-
simGetRandomNode
public String simGetRandomNode(Random random)
Get random node id.- Parameters:
random
- instance of random.- Returns:
- one of the live nodes
-
simAddNode
public void simAddNode(String nodeId) throws Exception
Add a new node to the cluster.- Parameters:
nodeId
- unique node id- Throws:
Exception
-
simRemoveNode
public boolean simRemoveNode(String nodeId) throws Exception
Remove node from a cluster. This is equivalent to a situation when a node is lost. All replicas that were assigned to this node are marked as DOWN.- Parameters:
nodeId
- node id- Returns:
- true if a node existed and was removed
- Throws:
Exception
-
simRemoveDeadNodes
public void simRemoveDeadNodes() throws Exception
Remove all replica information related to dead nodes.- Throws:
Exception
-
simGetOverseerLeader
public String simGetOverseerLeader()
-
simRestoreNode
public boolean simRestoreNode(String nodeId) throws Exception
Restore a previously removed node. This also simulates a short replica recovery state.- Parameters:
nodeId
- node id to restore- Returns:
- true when this operation restored any replicas, false otherwise (empty node).
- Throws:
Exception
-
simAddReplica
public void simAddReplica(ZkNodeProps message, NamedList results) throws Exception
Add a new replica. Note that if any details of a replica (node, coreNodeName, SolrCore name, etc) are missing they will be filled in using the policy framework.- Parameters:
message
- replica detailsresults
- result of the operation- Throws:
Exception
-
simAddReplica
public void simAddReplica(String nodeId, ReplicaInfo replicaInfo, boolean runLeaderElection) throws Exception
Add a replica. Note that all details of the replica must be present here, including node, coreNodeName and SolrCore name.- Parameters:
nodeId
- node id where the replica will be addedreplicaInfo
- replica inforunLeaderElection
- if true then run a leader election after adding the replica.- Throws:
Exception
-
simRemoveReplica
public void simRemoveReplica(String nodeId, String collection, String coreNodeName) throws Exception
Remove replica.- Parameters:
nodeId
- node idcoreNodeName
- coreNodeName- Throws:
Exception
-
simCreateCollection
public void simCreateCollection(ZkNodeProps props, NamedList results) throws Exception
Create a new collection. This operation uses policy framework for node and replica assignments.- Parameters:
props
- collection detailsresults
- results of the operation.- Throws:
Exception
-
simDeleteCollection
public void simDeleteCollection(String collection, String async, NamedList results) throws Exception
Delete a collection- Parameters:
collection
- collection nameasync
- async idresults
- results of the operation- Throws:
Exception
-
simDeleteAllCollections
public void simDeleteAllCollections() throws Exception
Remove all collections.- Throws:
Exception
-
simMoveReplica
public void simMoveReplica(ZkNodeProps message, NamedList results) throws Exception
Move replica. This uses a similar algorithm asMoveReplicaCmd
moveNormalReplica(...)
method.- Parameters:
message
- operation detailsresults
- operation results.- Throws:
Exception
-
simCreateShard
public void simCreateShard(ZkNodeProps message, NamedList results) throws Exception
Create a new shard. This uses a similar algorithm asCreateShardCmd
.- Parameters:
message
- operation detailsresults
- operation results- Throws:
Exception
-
simSplitShard
public void simSplitShard(ZkNodeProps message, NamedList results) throws Exception
Split a shard. This uses a similar algorithm asSplitShardCmd
, including simulating its quirks, and leaving the original parent slice in place.- Parameters:
message
- operation detailsresults
- operation results.- Throws:
Exception
-
simDeleteShard
public void simDeleteShard(ZkNodeProps message, NamedList results) throws Exception
Delete a shard. This uses a similar algorithm asDeleteShardCmd
- Parameters:
message
- operation detailsresults
- operation results- Throws:
Exception
-
createSystemCollection
public void createSystemCollection() throws IOException
- Throws:
IOException
-
simUpdate
public UpdateResponse simUpdate(UpdateRequest req) throws SolrException, InterruptedException, IOException
Simulate an update by modifying replica metrics. The following core metrics are updated:SEARCHER.searcher.numDocs
- increased by added docs, decreased by deleteById and deleteByQuerySEARCHER.searcher.deletedDocs
- decreased by deleteById and deleteByQuery by up tonumDocs
SEARCHER.searcher.maxDoc
- always increased by the number of added docs.
IMPORTANT limitations:
- document replacements are always counted as new docs
- delete by ID always succeeds (unless numDocs == 0)
- deleteByQuery is not supported unless the query is
*:*
- Parameters:
req
- update request. This request MUST have thecollection
param set.- Returns:
UpdateResponse
- Throws:
SolrException
- on errors, such as nonexistent collection or unsupported deleteByQueryInterruptedException
IOException
-
simQuery
public QueryResponse simQuery(QueryRequest req) throws SolrException, InterruptedException, IOException
-
simSetClusterProperties
public void simSetClusterProperties(Map<String,Object> properties) throws Exception
Set all cluster properties. This also updates the clusterprops.json data inDistribStateManager
- Parameters:
properties
- properties to set- Throws:
Exception
-
simSetClusterProperty
public void simSetClusterProperty(String key, Object value) throws Exception
Set a cluster property. This also updates the clusterprops.json data inDistribStateManager
- Parameters:
key
- property namevalue
- property value- Throws:
Exception
-
simSetCollectionProperties
public void simSetCollectionProperties(String coll, Map<String,Object> properties) throws Exception
Set collection properties.- Parameters:
coll
- collection nameproperties
- properties- Throws:
Exception
-
simSetCollectionProperty
public void simSetCollectionProperty(String coll, String key, String value) throws Exception
Set collection property.- Parameters:
coll
- collection namekey
- property namevalue
- property value- Throws:
Exception
-
simSetSliceProperties
public void simSetSliceProperties(String coll, String slice, Map<String,Object> properties) throws Exception
Set slice properties.- Parameters:
coll
- collection nameslice
- slice nameproperties
- slice properties- Throws:
Exception
-
simSetCollectionValue
public void simSetCollectionValue(String collection, String key, Object value) throws Exception
Set per-collection value (eg. a metric). This value will be applied to each replica.- Parameters:
collection
- collection namekey
- property namevalue
- property value- Throws:
Exception
-
simSetCollectionValue
public void simSetCollectionValue(String collection, String key, Object value, boolean delta, boolean divide) throws Exception
Set per-collection value (eg. a metric). This value will be applied to each replica.
-
simSetShardValue
public void simSetShardValue(String collection, String shard, String key, Object value) throws Exception
Set per-collection value (eg. a metric). This value will be applied to each replica in a selected shard.- Parameters:
collection
- collection nameshard
- shard name. If null then all shards will be affected.key
- property namevalue
- property value- Throws:
Exception
-
simSetShardValue
public void simSetShardValue(String collection, String shard, String key, Object value, boolean delta, boolean divide) throws Exception
Set per-collection value (eg. a metric). This value will be applied to each replica in a selected shard.- Parameters:
collection
- collection nameshard
- shard name. If null then all shards will be affected.key
- property namevalue
- property valuedelta
- if true then treat the numeric value as delta to add to the existing value (or set the value to delta if missing)divide
- if the value is aNumber
and this is true, then the value will be evenly divided by the number of replicas.- Throws:
Exception
-
simSetReplicaValues
public void simSetReplicaValues(String node, Map<String,Map<String,List<ReplicaInfo>>> source, boolean overwrite)
-
simGetReplicaInfos
public List<ReplicaInfo> simGetReplicaInfos(String node)
Return all replica infos for a node.- Parameters:
node
- node id- Returns:
- copy of the list of replicas on that node, or empty list if none
-
simGetReplicaInfos
public List<ReplicaInfo> simGetReplicaInfos(String collection, String shard)
-
simGetReplicaInfo
public ReplicaInfo simGetReplicaInfo(String collection, String coreNode)
-
simListCollections
public List<String> simListCollections() throws InterruptedException
List collections.- Returns:
- list of existing collections.
- Throws:
InterruptedException
-
simGetCollectionStats
public Map<String,Map<String,Object>> simGetCollectionStats() throws IOException, InterruptedException
- Throws:
IOException
InterruptedException
-
getState
public ClusterState.CollectionRef getState(String collection)
- Specified by:
getState
in interfaceClusterStateProvider
-
getLiveNodes
public Set<String> getLiveNodes()
- Specified by:
getLiveNodes
in interfaceClusterStateProvider
-
resolveAlias
public List<String> resolveAlias(String alias)
- Specified by:
resolveAlias
in interfaceClusterStateProvider
-
getAliasProperties
public Map<String,String> getAliasProperties(String alias)
- Specified by:
getAliasProperties
in interfaceClusterStateProvider
-
getClusterState
public ClusterState getClusterState() throws IOException
- Specified by:
getClusterState
in interfaceClusterStateProvider
- Throws:
IOException
-
getClusterProperties
public Map<String,Object> getClusterProperties()
- Specified by:
getClusterProperties
in interfaceClusterStateProvider
-
getPolicyNameByCollection
public String getPolicyNameByCollection(String coll)
- Specified by:
getPolicyNameByCollection
in interfaceClusterStateProvider
-
connect
public void connect()
- Specified by:
connect
in interfaceClusterStateProvider
-
close
public void close() throws IOException
- Specified by:
close
in interfaceAutoCloseable
- Specified by:
close
in interfaceCloseable
- Throws:
IOException
-
isClosed
public boolean isClosed()
- Specified by:
isClosed
in interfaceSolrCloseable
-
-