Package org.apache.solr.handler.export
Class ExportWriter
- java.lang.Object
-
- org.apache.solr.handler.export.ExportWriter
-
- All Implemented Interfaces:
Closeable
,AutoCloseable
,SolrCore.RawWriter
public class ExportWriter extends Object implements SolrCore.RawWriter, Closeable
Prepares and writes the documents requested by /export requestsExportWriter
gathers and sorts the documents for a core using "stream sorting".Stream sorting works by repeatedly processing and modifying a bitmap of matching documents. Each pass over the bitmap identifies the smallest
DOCUMENT_BATCH_SIZE
docs that haven't been sent yet and stores them in a Priority Queue. They are then exported (written across the wire) and marked as sent (unset in the bitmap). This process repeats until all matching documents have been sent.This streaming approach is light on memory (only
DOCUMENT_BATCH_SIZE
documents are ever stored in memory at once), and it allowsExportWriter
to scale well with regard to numDocs.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
ExportWriter.ExportWriterStream
static class
ExportWriter.IgnoreException
-
Constructor Summary
Constructors Constructor Description ExportWriter(SolrQueryRequest req, SolrQueryResponse res, String wt, StreamContext initialStreamContext)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description protected void
addDocsToItemWriter(List<org.apache.lucene.index.LeafReaderContext> leaves, IteratorWriter.ItemWriter writer, org.apache.solr.handler.export.SortDoc[] docsToExport, int outDocsIndex)
void
close()
String
getContentType()
protected org.apache.solr.handler.export.FieldWriter[]
getFieldWriters(String[] fields, SolrIndexSearcher searcher)
protected void
identifyLowestSortingUnexportedDocs(List<org.apache.lucene.index.LeafReaderContext> leaves, org.apache.solr.handler.export.SortDoc sortDoc, org.apache.solr.handler.export.SortQueue queue)
protected int
transferBatchToArrayForOutput(org.apache.solr.handler.export.SortQueue queue, org.apache.solr.handler.export.SortDoc[] destinationArr)
void
write(OutputStream os)
protected void
writeDocs(SolrQueryRequest req, IteratorWriter.ItemWriter writer, org.apache.lucene.search.Sort sort)
protected void
writeException(Exception e, PushWriter w, boolean logException)
-
-
-
Constructor Detail
-
ExportWriter
public ExportWriter(SolrQueryRequest req, SolrQueryResponse res, String wt, StreamContext initialStreamContext)
-
-
Method Detail
-
getContentType
public String getContentType()
- Specified by:
getContentType
in interfaceSolrCore.RawWriter
-
close
public void close() throws IOException
- Specified by:
close
in interfaceAutoCloseable
- Specified by:
close
in interfaceCloseable
- Throws:
IOException
-
writeException
protected void writeException(Exception e, PushWriter w, boolean logException) throws IOException
- Throws:
IOException
-
write
public void write(OutputStream os) throws IOException
- Specified by:
write
in interfaceSolrCore.RawWriter
- Throws:
IOException
-
identifyLowestSortingUnexportedDocs
protected void identifyLowestSortingUnexportedDocs(List<org.apache.lucene.index.LeafReaderContext> leaves, org.apache.solr.handler.export.SortDoc sortDoc, org.apache.solr.handler.export.SortQueue queue) throws IOException
- Throws:
IOException
-
transferBatchToArrayForOutput
protected int transferBatchToArrayForOutput(org.apache.solr.handler.export.SortQueue queue, org.apache.solr.handler.export.SortDoc[] destinationArr)
-
addDocsToItemWriter
protected void addDocsToItemWriter(List<org.apache.lucene.index.LeafReaderContext> leaves, IteratorWriter.ItemWriter writer, org.apache.solr.handler.export.SortDoc[] docsToExport, int outDocsIndex) throws IOException
- Throws:
IOException
-
writeDocs
protected void writeDocs(SolrQueryRequest req, IteratorWriter.ItemWriter writer, org.apache.lucene.search.Sort sort) throws IOException
- Throws:
IOException
-
getFieldWriters
protected org.apache.solr.handler.export.FieldWriter[] getFieldWriters(String[] fields, SolrIndexSearcher searcher) throws IOException
- Throws:
IOException
-
-