Package org.apache.solr.cloud

Class DistributedApiAsyncTracker

java.lang.Object
org.apache.solr.cloud.DistributedApiAsyncTracker
public class DistributedApiAsyncTracker extends Object
Class holding the implementation required for tracking asynchronous Collection API (or other) tasks when the Collection API is distributed.

This replaces the features provided by the distributed maps on ZK paths /overseer/collection-map-completed, /overseer/collection-map-failure and /overseer/async_ids when the Collection API commands are handled by the Overseer.

It works by using two Zookeeper directories, one for persistent nodes for each new async id and one for ephemeral nodes for each async id currently being processed (in flight).
A persistent async node has either no data, or has a serialized OverseerSolrResponse as content. An ephemeral async node has two possible states (content): 'S' or 'R'.

The actual state of an async task is built from a combination of the two nodes: 

 +===================+=========================================+=================================================+====================+
 |                   | persistent=success OverseerSolrResponse | persistent=null or failed OverseerSolrResponse  | No persistent node |
 +===================+=========================================+=================================================+====================+
 | ephemeral="S"     | Task completed successfully             | Task submitted                                  | Unknown task       |
 +-------------------+-----------------------------------------+-------------------------------------------------+--------------------+
 | ephemeral="R"     | Task completed successfully             | Task running                                    | Unknown task       |
 +-------------------+-----------------------------------------+-------------------------------------------------+--------------------+
 | No ephemeral node | Task completed successfully             | Task failed (see response or null=node failure) | Unknown task       |
 +-------------------+-----------------------------------------+-------------------------------------------------+--------------------+

  • Field Details

    • MAX_TRACKED_ASYNC_TASKS

      public static int MAX_TRACKED_ASYNC_TASKS
      Corresponds to Overseer.NUM_RESPONSES_TO_STORE. The size of the persistent store of async ID's put in Zookeeper. This is the max total tracked async request ID's over all nodes running in the distributed Collection API.

  • Constructor Details

    • DistributedApiAsyncTracker

      public DistributedApiAsyncTracker(org.apache.solr.common.cloud.SolrZkClient zkClient, String rootPath)

  • Method Details

    • createNewAsyncJobTracker

      public boolean createNewAsyncJobTracker(String asyncId)
      After a successful call to this method, caller MUST eventually call setTaskCompleted(java.lang.String, org.apache.solr.cloud.OverseerSolrResponse) or cancelAsyncId(java.lang.String) otherwise the task will forever be considered as in progress.
      Parameters:
      asyncId - if null this method will do nothing.
      Returns:
      true if the asyncId was not already in use (or is null) and false if it is already in use and can't be allocated again.

    • setTaskRunning

      public void setTaskRunning(String asyncId)
      Initially an async task is submitted. Just before it actually starts execution it is set to running.

    • setTaskCompleted

      public void setTaskCompleted(String asyncId, OverseerSolrResponse solrResponse)
      Mark the completion (success or error) of an async task. The success or error is judged by the contents of the OverseerSolrResponse.

    • cancelAsyncId

      public void cancelAsyncId(String asyncId)
      Cancels the tracking of an asyncId, if the corresponding command could not be executed.

    • getAsyncTaskRequestStatus

      public org.apache.solr.common.util.Pair<org.apache.solr.client.solrj.response.RequestStatusState,OverseerSolrResponse> getAsyncTaskRequestStatus(String asyncId) throws Exception
      This method implements the logic described in the class Javadoc table (DistributedApiAsyncTracker), using the two sets of tracking info to build the actual state of an async task.

      Returns the status of an async task, and when relevant the corresponding response from the command execution. The returned OverseerSolrResponse will not be null when the returned RequestStatusState is RequestStatusState.COMPLETED or RequestStatusState.FAILED (and will be null in all other cases).

      Throws:
      Exception

    • deleteSingleAsyncId

      public boolean deleteSingleAsyncId(String asyncId) throws Exception
      Deletes a single async tracking ID if the corresponding job has completed or failed.
      Returns:
      true if the asyncId was found to be of a completed or failed job and was successfully removed, false if the id was not found or was found for a submitted or running job (these are not removed).
      Throws:
      Exception

    • deleteAllAsyncIds

      public void deleteAllAsyncIds() throws Exception
      Deletes all async id's for completed or failed async jobs. Does not touch id's for submitted or running jobs.
      Throws:
      Exception