Package org.apache.solr.cloud

Class DistributedApiAsyncTracker

  • public class DistributedApiAsyncTracker
extends Object
    Class holding the implementation required for tracking asynchronous Collection API (or other) tasks when the Collection API is distributed.

    This replaces the features provided by the distributed maps on ZK paths /overseer/collection-map-completed, /overseer/collection-map-failure and /overseer/async_ids when the Collection API commands are handled by the Overseer.

    It works by using two Zookeeper directories, one for persistent nodes for each new async id and one for ephemeral nodes for each async id currently being processed (in flight).
    A persistent async node has either no data, or has a serialized OverseerSolrResponse as content. An ephemeral async node has two possible states (content): 'S' or 'R'.

    The actual state of an async task is built from a combination of the two nodes: 

     +===================+=========================================+=================================================+====================+
 |                   | persistent=success OverseerSolrResponse | persistent=null or failed OverseerSolrResponse  | No persistent node |
 +===================+=========================================+=================================================+====================+
 | ephemeral="S"     | Task completed successfully             | Task submitted                                  | Unknown task       |
 +-------------------+-----------------------------------------+-------------------------------------------------+--------------------+
 | ephemeral="R"     | Task completed successfully             | Task running                                    | Unknown task       |
 +-------------------+-----------------------------------------+-------------------------------------------------+--------------------+
 | No ephemeral node | Task completed successfully             | Task failed (see response or null=node failure) | Unknown task       |
 +-------------------+-----------------------------------------+-------------------------------------------------+--------------------+

    • Field Detail

      • MAX_TRACKED_ASYNC_TASKS

        public static int MAX_TRACKED_ASYNC_TASKS
        Corresponds to Overseer.NUM_RESPONSES_TO_STORE. The size of the persistent store of async ID's put in Zookeeper. This is the max total tracked async request ID's over all nodes running in the distributed Collection API.

    • Constructor Detail

      • DistributedApiAsyncTracker

        public DistributedApiAsyncTracker​(org.apache.solr.common.cloud.SolrZkClient zkClient,
                                  String rootPath)

    • Method Detail

      • setTaskRunning

        public void setTaskRunning​(String asyncId)
        Initially an async task is submitted. Just before it actually starts execution it is set to running.

      • setTaskCompleted

        public void setTaskCompleted​(String asyncId,
                             OverseerSolrResponse solrResponse)
        Mark the completion (success or error) of an async task. The success or error is judged by the contents of the OverseerSolrResponse.

      • cancelAsyncId

        public void cancelAsyncId​(String asyncId)
        Cancels the tracking of an asyncId, if the corresponding command could not be executed.

      • getAsyncTaskRequestStatus

        public org.apache.solr.common.util.Pair<org.apache.solr.client.solrj.response.RequestStatusState,​OverseerSolrResponse> getAsyncTaskRequestStatus​(String asyncId)
                                                                                                                                                throws Exception
        This method implements the logic described in the class Javadoc table (DistributedApiAsyncTracker), using the two sets of tracking info to build the actual state of an async task.

        Returns the status of an async task, and when relevant the corresponding response from the command execution. The returned OverseerSolrResponse will not be null when the returned RequestStatusState is RequestStatusState.COMPLETED or RequestStatusState.FAILED (and will be null in all other cases).

        Throws:
        Exception

      • deleteSingleAsyncId

        public boolean deleteSingleAsyncId​(String asyncId)
                            throws Exception
        Deletes a single async tracking ID if the corresponding job has completed or failed.
        Returns:
        true if the asyncId was found to be of a completed or failed job and was successfully removed, false if the id was not found or was found for a submitted or running job (these are not removed).
        Throws:
        Exception

      • deleteAllAsyncIds

        public void deleteAllAsyncIds()
                       throws Exception
        Deletes all async id's for completed or failed async jobs. Does not touch id's for submitted or running jobs.
        Throws:
        Exception