Package org.apache.solr.cloud

Class DistributedApiAsyncTracker

java.lang.Object

org.apache.solr.cloud.DistributedApiAsyncTracker

public class DistributedApiAsyncTracker
extends Object

Class holding the implementation required for tracking asynchronous Collection API (or other) tasks when the Collection API is distributed.

This replaces the features provided by the distributed maps on ZK paths /overseer/collection-map-completed, /overseer/collection-map-failure and /overseer/async_ids when the Collection API commands are handled by the Overseer.

It works by using two Zookeeper directories, one for persistent nodes for each new async id and one for ephemeral nodes for each async id currently being processed (in flight).
A persistent async node has either no data, or has a serialized OverseerSolrResponse as content. An ephemeral async node has two possible states (content): 'S' or 'R'.

The actual state of an async task is built from a combination of the two nodes:

 +===================+=========================================+=================================================+====================+
 |                   | persistent=success OverseerSolrResponse | persistent=null or failed OverseerSolrResponse  | No persistent node |
 +===================+=========================================+=================================================+====================+
 | ephemeral="S"     | Task completed successfully             | Task submitted                                  | Unknown task       |
 +-------------------+-----------------------------------------+-------------------------------------------------+--------------------+
 | ephemeral="R"     | Task completed successfully             | Task running                                    | Unknown task       |
 +-------------------+-----------------------------------------+-------------------------------------------------+--------------------+
 | No ephemeral node | Task completed successfully             | Task failed (see response or null=node failure) | Unknown task       |
 +-------------------+-----------------------------------------+-------------------------------------------------+--------------------+
 

Field Summary

Fields

Modifier and Type	Field	Description
static int	MAX_TRACKED_ASYNC_TASKS	Corresponds to Overseer.NUM_RESPONSES_TO_STORE.

Constructor Summary

Constructors

Constructor	Description
DistributedApiAsyncTracker(SolrZkClient zkClient, String rootPath)

Method Summary

Modifier and Type	Method	Description
void	cancelAsyncId(String asyncId)	Cancels the tracking of an asyncId, if the corresponding command could not be executed.
boolean	createNewAsyncJobTracker(String asyncId)	After a successful call to this method, caller MUST eventually call setTaskCompleted(java.lang.String, org.apache.solr.cloud.OverseerSolrResponse) or cancelAsyncId(java.lang.String) otherwise the task will forever be considered as in progress.
void	deleteAllAsyncIds()	Deletes all async id's for completed or failed async jobs.
boolean	deleteSingleAsyncId(String asyncId)	Deletes a single async tracking ID if the corresponding job has completed or failed.
Pair<RequestStatusState,OverseerSolrResponse>	getAsyncTaskRequestStatus(String asyncId)	This method implements the logic described in the class Javadoc table (DistributedApiAsyncTracker), using the two sets of tracking info to build the actual state of an async task.
void	setTaskCompleted(String asyncId, OverseerSolrResponse solrResponse)	Mark the completion (success or error) of an async task.
void	setTaskRunning(String asyncId)	Initially an async task is submitted.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MAX_TRACKED_ASYNC_TASKS

public static int MAX_TRACKED_ASYNC_TASKS

Corresponds to Overseer.NUM_RESPONSES_TO_STORE. The size of the persistent store of async ID's put in Zookeeper. This is the max total tracked async request ID's over all nodes running in the distributed Collection API.

Constructor Detail

DistributedApiAsyncTracker

public DistributedApiAsyncTracker(SolrZkClient zkClient,
                                  String rootPath)

Method Detail

createNewAsyncJobTracker

public boolean createNewAsyncJobTracker(String asyncId)

After a successful call to this method, caller MUST eventually call setTaskCompleted(java.lang.String, org.apache.solr.cloud.OverseerSolrResponse) or cancelAsyncId(java.lang.String) otherwise the task will forever be considered as in progress.

Parameters:

asyncId - if null this method will do nothing.

Returns:

true if the asyncId was not already in use (or is null) and false if it is already in use and can't be allocated again.

setTaskRunning

public void setTaskRunning(String asyncId)

Initially an async task is submitted. Just before it actually starts execution it is set to running.

setTaskCompleted

public void setTaskCompleted(String asyncId,
                             OverseerSolrResponse solrResponse)

Mark the completion (success or error) of an async task. The success or error is judged by the contents of the OverseerSolrResponse.

cancelAsyncId

public void cancelAsyncId(String asyncId)

Cancels the tracking of an asyncId, if the corresponding command could not be executed.

getAsyncTaskRequestStatus

public Pair<RequestStatusState,OverseerSolrResponse> getAsyncTaskRequestStatus(String asyncId)
                                                                              throws Exception

This method implements the logic described in the class Javadoc table (DistributedApiAsyncTracker), using the two sets of tracking info to build the actual state of an async task.

Returns the status of an async task, and when relevant the corresponding response from the command execution. The returned OverseerSolrResponse will not be null when the returned RequestStatusState is RequestStatusState.COMPLETED or RequestStatusState.FAILED (and will be null in all other cases).

Throws:

Exception

deleteSingleAsyncId

public boolean deleteSingleAsyncId(String asyncId)
                            throws Exception

Deletes a single async tracking ID if the corresponding job has completed or failed.

Returns:

true if the asyncId was found to be of a completed or failed job and was successfully removed, false if the id was not found or was found for a submitted or running job (these are not removed).

Throws:

Exception

deleteAllAsyncIds

public void deleteAllAsyncIds()
                       throws Exception

Deletes all async id's for completed or failed async jobs. Does not touch id's for submitted or running jobs.

Throws:

Exception