Class ZkDistributedQueue

  • All Implemented Interfaces:
    org.apache.solr.client.solrj.cloud.DistributedQueue
    Direct Known Subclasses:
    OverseerTaskQueue

    public class ZkDistributedQueue
    extends Object
    implements org.apache.solr.client.solrj.cloud.DistributedQueue
    A ZK-based distributed queue. Optimized for single-consumer, multiple-producer: if there are multiple consumers on the same ZK queue, the results should be correct but inefficient.

    This implementation (with help from subclass OverseerTaskQueue) is used for the /overseer/collection-queue-work queue used for Collection and Config Set API calls to the Overseer.

    Implementation note: In order to enqueue a message into this queue, a CreateMode.EPHEMERAL_SEQUENTIAL response node is created and watched at /overseer/collection-queue-work/qnr-monotonically_increasng_id, then a corresponding CreateMode.PERSISTENT request node reusing the same id is created at /overseer/collection-queue-work/qn-response_id.

    • Constructor Summary

      Constructors 
      Constructor Description
      ZkDistributedQueue​(org.apache.solr.common.cloud.SolrZkClient zookeeper, String dir)  
      ZkDistributedQueue​(org.apache.solr.common.cloud.SolrZkClient zookeeper, String dir, Stats stats)  
      ZkDistributedQueue​(org.apache.solr.common.cloud.SolrZkClient zookeeper, String dir, Stats stats, int maxQueueSize)  
      ZkDistributedQueue​(org.apache.solr.common.cloud.SolrZkClient zookeeper, String dir, Stats stats, int maxQueueSize, org.apache.solr.common.cloud.ConnectionManager.IsClosed higherLevelIsClosed)  
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      Map<String,​Object> getStats()  
      Stats getZkStats()  
      void offer​(byte[] data)
      Inserts data into queue.
      byte[] peek()
      Returns the data at the first element of the queue, or null if the queue is empty.
      byte[] peek​(boolean block)
      Returns the data at the first element of the queue, or null if the queue is empty and block is false.
      byte[] peek​(long wait)
      Returns the data at the first element of the queue, or null if the queue is empty after wait ms.
      Collection<org.apache.solr.common.util.Pair<String,​byte[]>> peekElements​(int max, long waitMillis, Predicate<String> acceptFilter)
      Return the currently-known set of elements, using child names from memory.
      byte[] poll()
      Attempts to remove the head of the queue and return it.
      byte[] remove()
      Attempts to remove the head of the queue and return it.
      void remove​(Collection<String> paths)  
      byte[] take()
      Removes the head of the queue and returns it, blocks until it succeeds.
      • Methods inherited from interface org.apache.solr.client.solrj.cloud.DistributedQueue

        offer
    • Constructor Detail

      • ZkDistributedQueue

        public ZkDistributedQueue​(org.apache.solr.common.cloud.SolrZkClient zookeeper,
                                  String dir)
      • ZkDistributedQueue

        public ZkDistributedQueue​(org.apache.solr.common.cloud.SolrZkClient zookeeper,
                                  String dir,
                                  Stats stats)
      • ZkDistributedQueue

        public ZkDistributedQueue​(org.apache.solr.common.cloud.SolrZkClient zookeeper,
                                  String dir,
                                  Stats stats,
                                  int maxQueueSize)
      • ZkDistributedQueue

        public ZkDistributedQueue​(org.apache.solr.common.cloud.SolrZkClient zookeeper,
                                  String dir,
                                  Stats stats,
                                  int maxQueueSize,
                                  org.apache.solr.common.cloud.ConnectionManager.IsClosed higherLevelIsClosed)
    • Method Detail

      • peek

        public byte[] peek()
                    throws org.apache.zookeeper.KeeperException,
                           InterruptedException
        Returns the data at the first element of the queue, or null if the queue is empty.
        Specified by:
        peek in interface org.apache.solr.client.solrj.cloud.DistributedQueue
        Returns:
        data at the first element of the queue, or null.
        Throws:
        org.apache.zookeeper.KeeperException
        InterruptedException
      • peek

        public byte[] peek​(boolean block)
                    throws org.apache.zookeeper.KeeperException,
                           InterruptedException
        Returns the data at the first element of the queue, or null if the queue is empty and block is false.
        Specified by:
        peek in interface org.apache.solr.client.solrj.cloud.DistributedQueue
        Parameters:
        block - if true, blocks until an element enters the queue
        Returns:
        data at the first element of the queue, or null.
        Throws:
        org.apache.zookeeper.KeeperException
        InterruptedException
      • peek

        public byte[] peek​(long wait)
                    throws org.apache.zookeeper.KeeperException,
                           InterruptedException
        Returns the data at the first element of the queue, or null if the queue is empty after wait ms.
        Specified by:
        peek in interface org.apache.solr.client.solrj.cloud.DistributedQueue
        Parameters:
        wait - max wait time in ms.
        Returns:
        data at the first element of the queue, or null.
        Throws:
        org.apache.zookeeper.KeeperException
        InterruptedException
      • poll

        public byte[] poll()
                    throws org.apache.zookeeper.KeeperException,
                           InterruptedException
        Attempts to remove the head of the queue and return it. Returns null if the queue is empty.
        Specified by:
        poll in interface org.apache.solr.client.solrj.cloud.DistributedQueue
        Returns:
        Head of the queue or null.
        Throws:
        org.apache.zookeeper.KeeperException
        InterruptedException
      • take

        public byte[] take()
                    throws org.apache.zookeeper.KeeperException,
                           InterruptedException
        Removes the head of the queue and returns it, blocks until it succeeds.
        Specified by:
        take in interface org.apache.solr.client.solrj.cloud.DistributedQueue
        Returns:
        The former head of the queue
        Throws:
        org.apache.zookeeper.KeeperException
        InterruptedException
      • offer

        public void offer​(byte[] data)
                   throws org.apache.zookeeper.KeeperException,
                          InterruptedException
        Inserts data into queue. If there are no other queue consumers, the offered element will be immediately visible when this method returns.
        Specified by:
        offer in interface org.apache.solr.client.solrj.cloud.DistributedQueue
        Throws:
        org.apache.zookeeper.KeeperException
        InterruptedException
      • getZkStats

        public Stats getZkStats()
      • getStats

        public Map<String,​Object> getStats()
        Specified by:
        getStats in interface org.apache.solr.client.solrj.cloud.DistributedQueue
      • peekElements

        public Collection<org.apache.solr.common.util.Pair<String,​byte[]>> peekElements​(int max,
                                                                                              long waitMillis,
                                                                                              Predicate<String> acceptFilter)
                                                                                       throws org.apache.zookeeper.KeeperException,
                                                                                              InterruptedException
        Return the currently-known set of elements, using child names from memory. If no children are found, or no children pass acceptFilter, waits up to waitMillis for at least one child to become available.

        Package-private to support OverseerTaskQueue specifically.

        Specified by:
        peekElements in interface org.apache.solr.client.solrj.cloud.DistributedQueue
        Throws:
        org.apache.zookeeper.KeeperException
        InterruptedException