Package org.apache.solr.cloud

Class ZkDistributedQueue

java.lang.Object
org.apache.solr.cloud.ZkDistributedQueue
All Implemented Interfaces:
org.apache.solr.client.solrj.cloud.DistributedQueue
Direct Known Subclasses:
OverseerTaskQueue
public class ZkDistributedQueue extends Object implements org.apache.solr.client.solrj.cloud.DistributedQueue
A ZK-based distributed queue. Optimized for single-consumer, multiple-producer: if there are multiple consumers on the same ZK queue, the results should be correct but inefficient.

This implementation (with help from subclass OverseerTaskQueue) is used for the /overseer/collection-queue-work queue used for Collection and Config Set API calls to the Overseer.

Implementation note: In order to enqueue a message into this queue, a CreateMode.EPHEMERAL_SEQUENTIAL response node is created and watched at /overseer/collection-queue-work/qnr-monotonically_increasng_id, then a corresponding CreateMode.PERSISTENT request node reusing the same id is created at /overseer/collection-queue-work/qn-response_id.

  • Constructor Summary

    Constructors
    Constructor
    Description
    ZkDistributedQueue(org.apache.solr.common.cloud.SolrZkClient zookeeper, String dir)
     
    ZkDistributedQueue(org.apache.solr.common.cloud.SolrZkClient zookeeper, String dir, Stats stats)
     
    ZkDistributedQueue(org.apache.solr.common.cloud.SolrZkClient zookeeper, String dir, Stats stats, int maxQueueSize)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    Map<String,Object>
    getStats()
     
    Stats
    getZkStats()
     
    void
    offer(byte[] data)
    Inserts data into queue.
    byte[]
    peek()
    Returns the data at the first element of the queue, or null if the queue is empty.
    byte[]
    peek(boolean block)
    Returns the data at the first element of the queue, or null if the queue is empty and block is false.
    byte[]
    peek(long wait)
    Returns the data at the first element of the queue, or null if the queue is empty after wait ms.
    Collection<org.apache.solr.common.util.Pair<String,byte[]>>
    peekElements(int max, long waitMillis, Predicate<String> acceptFilter)
    Return the currently-known set of elements, using child names from memory.
    byte[]
    poll()
    Attempts to remove the head of the queue and return it.
    byte[]
    remove()
    Attempts to remove the head of the queue and return it.
    void
    remove(Collection<String> paths)
     
    byte[]
    take()
    Removes the head of the queue and returns it, blocks until it succeeds.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.apache.solr.client.solrj.cloud.DistributedQueue

    offer

  • Constructor Details

    • ZkDistributedQueue

      public ZkDistributedQueue(org.apache.solr.common.cloud.SolrZkClient zookeeper, String dir)

    • ZkDistributedQueue

      public ZkDistributedQueue(org.apache.solr.common.cloud.SolrZkClient zookeeper, String dir, Stats stats)

    • ZkDistributedQueue

      public ZkDistributedQueue(org.apache.solr.common.cloud.SolrZkClient zookeeper, String dir, Stats stats, int maxQueueSize)

  • Method Details

    • peek

      public byte[] peek() throws org.apache.zookeeper.KeeperException, InterruptedException
      Returns the data at the first element of the queue, or null if the queue is empty.
      Specified by:
      peek in interface org.apache.solr.client.solrj.cloud.DistributedQueue
      Returns:
      data at the first element of the queue, or null.
      Throws:
      org.apache.zookeeper.KeeperException
      InterruptedException

    • peek

      public byte[] peek(boolean block) throws org.apache.zookeeper.KeeperException, InterruptedException
      Returns the data at the first element of the queue, or null if the queue is empty and block is false.
      Specified by:
      peek in interface org.apache.solr.client.solrj.cloud.DistributedQueue
      Parameters:
      block - if true, blocks until an element enters the queue
      Returns:
      data at the first element of the queue, or null.
      Throws:
      org.apache.zookeeper.KeeperException
      InterruptedException

    • peek

      public byte[] peek(long wait) throws org.apache.zookeeper.KeeperException, InterruptedException
      Returns the data at the first element of the queue, or null if the queue is empty after wait ms.
      Specified by:
      peek in interface org.apache.solr.client.solrj.cloud.DistributedQueue
      Parameters:
      wait - max wait time in ms.
      Returns:
      data at the first element of the queue, or null.
      Throws:
      org.apache.zookeeper.KeeperException
      InterruptedException

    • poll

      public byte[] poll() throws org.apache.zookeeper.KeeperException, InterruptedException
      Attempts to remove the head of the queue and return it. Returns null if the queue is empty.
      Specified by:
      poll in interface org.apache.solr.client.solrj.cloud.DistributedQueue
      Returns:
      Head of the queue or null.
      Throws:
      org.apache.zookeeper.KeeperException
      InterruptedException

    • remove

      public byte[] remove() throws NoSuchElementException, org.apache.zookeeper.KeeperException, InterruptedException
      Attempts to remove the head of the queue and return it.
      Specified by:
      remove in interface org.apache.solr.client.solrj.cloud.DistributedQueue
      Returns:
      The former head of the queue
      Throws:
      NoSuchElementException
      org.apache.zookeeper.KeeperException
      InterruptedException

    • remove

      public void remove(Collection<String> paths) throws org.apache.zookeeper.KeeperException, InterruptedException
      Throws:
      org.apache.zookeeper.KeeperException
      InterruptedException

    • take

      public byte[] take() throws org.apache.zookeeper.KeeperException, InterruptedException
      Removes the head of the queue and returns it, blocks until it succeeds.
      Specified by:
      take in interface org.apache.solr.client.solrj.cloud.DistributedQueue
      Returns:
      The former head of the queue
      Throws:
      org.apache.zookeeper.KeeperException
      InterruptedException

    • offer

      public void offer(byte[] data) throws org.apache.zookeeper.KeeperException, InterruptedException
      Inserts data into queue. If there are no other queue consumers, the offered element will be immediately visible when this method returns.
      Specified by:
      offer in interface org.apache.solr.client.solrj.cloud.DistributedQueue
      Throws:
      org.apache.zookeeper.KeeperException
      InterruptedException

    • getZkStats

      public Stats getZkStats()

    • getStats

      public Map<String,Object> getStats()
      Specified by:
      getStats in interface org.apache.solr.client.solrj.cloud.DistributedQueue

    • peekElements

      public Collection<org.apache.solr.common.util.Pair<String,byte[]>> peekElements(int max, long waitMillis, Predicate<String> acceptFilter) throws org.apache.zookeeper.KeeperException, InterruptedException
      Return the currently-known set of elements, using child names from memory. If no children are found, or no children pass acceptFilter, waits up to waitMillis for at least one child to become available.

      Package-private to support OverseerTaskQueue specifically.

      Specified by:
      peekElements in interface org.apache.solr.client.solrj.cloud.DistributedQueue
      Throws:
      org.apache.zookeeper.KeeperException
      InterruptedException