org.apache.solr.handler.export

Class ExportWriter

java.lang.Object

org.apache.solr.handler.export.ExportWriter

All Implemented Interfaces:

Closeable, AutoCloseable, SolrCore.RawWriter

public class ExportWriter
extends Object
implements SolrCore.RawWriter, Closeable

Prepares and writes the documents requested by /export requests ExportWriter gathers and sorts the documents for a core using "stream sorting".

Stream sorting works by repeatedly processing and modifying a bitmap of matching documents. Each pass over the bitmap identifies the smallest DOCUMENT_BATCH_SIZE docs that haven't been sent yet and stores them in a Priority Queue. They are then exported (written across the wire) and marked as sent (unset in the bitmap). This process repeats until all matching documents have been sent.

This streaming approach is light on memory (only DOCUMENT_BATCH_SIZE documents are ever stored in memory at once), and it allows ExportWriter to scale well with regard to numDocs.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ExportWriter.IgnoreException

Constructor Summary

Constructors

Constructor and Description
ExportWriter(SolrQueryRequest req, SolrQueryResponse res, String wt)

Method Summary

Modifier and Type	Method and Description
protected void	addDocsToItemWriter(List<LeafReaderContext> leaves, IteratorWriter.ItemWriter writer, org.apache.solr.handler.export.SortDoc[] docsToExport, int outDocsIndex)
void	close()
String	getContentType()
protected org.apache.solr.handler.export.FieldWriter[]	getFieldWriters(String[] fields, SolrIndexSearcher searcher)
protected void	identifyLowestSortingUnexportedDocs(List<LeafReaderContext> leaves, org.apache.solr.handler.export.SortDoc sortDoc, org.apache.solr.handler.export.SortQueue queue)
protected int	transferBatchToArrayForOutput(org.apache.solr.handler.export.SortQueue queue, org.apache.solr.handler.export.SortDoc[] destinationArr)
void	write(OutputStream os)
protected void	writeDoc(org.apache.solr.handler.export.SortDoc sortDoc, List<LeafReaderContext> leaves, MapWriter.EntryWriter ew)
protected void	writeDocs(SolrQueryRequest req, IteratorWriter.ItemWriter writer, Sort sort)
protected void	writeException(Exception e, PushWriter w, boolean logException)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ExportWriter

public ExportWriter(SolrQueryRequest req,
                    SolrQueryResponse res,
                    String wt)

Method Detail

getContentType

public String getContentType()

Specified by:

getContentType in interface SolrCore.RawWriter

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException

writeException

protected void writeException(Exception e,
                              PushWriter w,
                              boolean logException)
                       throws IOException

Throws:

IOException

write

public void write(OutputStream os)
           throws IOException

Specified by:

write in interface SolrCore.RawWriter

Throws:

IOException

identifyLowestSortingUnexportedDocs

protected void identifyLowestSortingUnexportedDocs(List<LeafReaderContext> leaves,
                                                   org.apache.solr.handler.export.SortDoc sortDoc,
                                                   org.apache.solr.handler.export.SortQueue queue)
                                            throws IOException

Throws:

IOException

transferBatchToArrayForOutput

protected int transferBatchToArrayForOutput(org.apache.solr.handler.export.SortQueue queue,
                                            org.apache.solr.handler.export.SortDoc[] destinationArr)

addDocsToItemWriter

protected void addDocsToItemWriter(List<LeafReaderContext> leaves,
                                   IteratorWriter.ItemWriter writer,
                                   org.apache.solr.handler.export.SortDoc[] docsToExport,
                                   int outDocsIndex)
                            throws IOException

Throws:

IOException

writeDocs

protected void writeDocs(SolrQueryRequest req,
                         IteratorWriter.ItemWriter writer,
                         Sort sort)
                  throws IOException

Throws:

IOException

writeDoc

protected void writeDoc(org.apache.solr.handler.export.SortDoc sortDoc,
                        List<LeafReaderContext> leaves,
                        MapWriter.EntryWriter ew)
                 throws IOException

Throws:

IOException

getFieldWriters

protected org.apache.solr.handler.export.FieldWriter[] getFieldWriters(String[] fields,
                                                                       SolrIndexSearcher searcher)
                                                                throws IOException

Throws:

IOException