org.apache.solr.handler.export

Class ExportWriter

java.lang.Object

org.apache.solr.handler.export.ExportWriter

All Implemented Interfaces:

Closeable, AutoCloseable, SolrCore.RawWriter

public class ExportWriter
extends Object
implements SolrCore.RawWriter, Closeable

Prepares and writes the documents requested by /export requests ExportWriter gathers and sorts the documents for a core using "stream sorting".

Stream sorting works by repeatedly processing and modifying a bitmap of matching documents. Each pass over the bitmap identifies the smallest docs (default is DEFAULT_BATCH_SIZE) that haven't been sent yet and stores them in a Priority Queue. They are then exported (written across the wire) and marked as sent (unset in the bitmap). This process repeats until all matching documents have been sent.

This streaming approach is light on memory (only up to 2x batch size documents are ever stored in memory at once), and it allows ExportWriter to scale well with regard to numDocs.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ExportWriter.IgnoreException

Field Summary

Fields

Modifier and Type	Field and Description
static String	BATCH_SIZE_PARAM
static int	DEFAULT_BATCH_SIZE

Constructor Summary

Constructors

Constructor and Description
ExportWriter(SolrQueryRequest req, SolrQueryResponse res, String wt, StreamContext initialStreamContext, SolrMetricsContext solrMetricsContext, String metricsPath, ExportHandler exportHandler)

Method Summary

Modifier and Type	Method and Description
void	close()
String	getContentType()
org.apache.solr.handler.export.FieldWriter[]	getFieldWriters(String[] fields, SolrIndexSearcher searcher)
void	write(OutputStream os)
protected void	writeDocs(SolrQueryRequest req, OutputStream os, IteratorWriter.ItemWriter writer, Sort sort)
protected void	writeException(Exception e, PushWriter w, boolean logException)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

BATCH_SIZE_PARAM

public static final String BATCH_SIZE_PARAM

See Also:

Constant Field Values

DEFAULT_BATCH_SIZE

public static final int DEFAULT_BATCH_SIZE

See Also:

Constant Field Values

Constructor Detail

ExportWriter

public ExportWriter(SolrQueryRequest req,
                    SolrQueryResponse res,
                    String wt,
                    StreamContext initialStreamContext,
                    SolrMetricsContext solrMetricsContext,
                    String metricsPath,
                    ExportHandler exportHandler)

Method Detail

getContentType

public String getContentType()

Specified by:

getContentType in interface SolrCore.RawWriter

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException

writeException

protected void writeException(Exception e,
                              PushWriter w,
                              boolean logException)
                       throws IOException

Throws:

IOException

write

public void write(OutputStream os)
           throws IOException

Specified by:

write in interface SolrCore.RawWriter

Throws:

IOException

writeDocs

protected void writeDocs(SolrQueryRequest req,
                         OutputStream os,
                         IteratorWriter.ItemWriter writer,
                         Sort sort)
                  throws IOException

Throws:

IOException

getFieldWriters

public org.apache.solr.handler.export.FieldWriter[] getFieldWriters(String[] fields,
                                                                    SolrIndexSearcher searcher)
                                                             throws IOException

Throws:

IOException