Class ExportWriter

  • All Implemented Interfaces:
    Closeable, AutoCloseable, SolrCore.RawWriter

    public class ExportWriter
    extends Object
    implements SolrCore.RawWriter, Closeable
    Prepares and writes the documents requested by /export requests ExportWriter gathers and sorts the documents for a core using "stream sorting".

    Stream sorting works by repeatedly processing and modifying a bitmap of matching documents. Each pass over the bitmap identifies the smallest DOCUMENT_BATCH_SIZE docs that haven't been sent yet and stores them in a Priority Queue. They are then exported (written across the wire) and marked as sent (unset in the bitmap). This process repeats until all matching documents have been sent.

    This streaming approach is light on memory (only DOCUMENT_BATCH_SIZE documents are ever stored in memory at once), and it allows ExportWriter to scale well with regard to numDocs.