Package org.apache.solr.client.solrj.io.stream

Class CloudSolrStream

java.lang.Object

org.apache.solr.client.solrj.io.stream.TupleStream

org.apache.solr.client.solrj.io.stream.CloudSolrStream

All Implemented Interfaces:

Closeable, Serializable, AutoCloseable, Expressible, org.apache.solr.common.MapWriter, org.apache.solr.common.NavigableObject, org.noggit.JSONWriter.Writable

Direct Known Subclasses:

DrillStream, ParallelStream, ShuffleStream, TopicStream

public class CloudSolrStream
extends TupleStream
implements Expressible

Connects to Zookeeper to pick replicas from a specific collection to send the query to. Under the covers the SolrStream instances send the query to the replicas. SolrStreams are opened using a thread pool, but a single thread is used to iterate and merge Tuples from each SolrStream.

Since:

5.1.0

See Also:

• Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected class	CloudSolrStream.StreamOpener
protected class	CloudSolrStream.TupleWrapper

Nested classes/interfaces inherited from class org.apache.solr.client.solrj.io.stream.TupleStream

TupleStream.IgnoreException

Nested classes/interfaces inherited from interface org.apache.solr.common.MapWriter

org.apache.solr.common.MapWriter.EntryWriter

Field Summary

Fields

Modifier and Type	Field	Description
protected String	collection
protected StreamComparator	comp
protected Map<String,Tuple>	eofTuples
protected Map<String,String>	fieldMappings
protected org.apache.solr.common.params.SolrParams	params
protected String	path
protected org.apache.solr.client.solrj.impl.CloudSolrClient.CloudSolrClientConnection	solrConnection
protected List<TupleStream>	solrStreams
protected StreamContext	streamContext
protected TreeSet<CloudSolrStream.TupleWrapper>	tuples

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	CloudSolrStream()
	CloudSolrStream(org.apache.solr.client.solrj.impl.CloudSolrClient.CloudSolrClientConnection solrConnection, String collectionName, String path, org.apache.solr.common.params.SolrParams params)
	CloudSolrStream(org.apache.solr.client.solrj.impl.CloudSolrClient.CloudSolrClientConnection solrConnection, String collectionName, org.apache.solr.common.params.SolrParams params)
	CloudSolrStream(StreamExpression expression, StreamFactory factory)

Method Summary

Modifier and Type	Method	Description
protected Tuple	_read()
protected org.apache.solr.common.params.ModifiableSolrParams	adjustParams(org.apache.solr.common.params.ModifiableSolrParams params)
List<TupleStream>	children()
void	close()	Closes the CloudSolrStream
protected void	constructStreams()
Map<String,Tuple>	getEofTuples()
static List<org.apache.solr.common.cloud.Slice>	getSlices(String collectionName, org.apache.solr.client.solrj.impl.CloudSolrClient cloudSolrClient, boolean checkAlias)
StreamComparator	getStreamSort()	Return the stream sort - ie, the order in which records are returned
void	open()	Opens the CloudSolrStream
Tuple	read()
void	setFieldMappings(Map<String,String> fieldMappings)
void	setStreamContext(StreamContext context)
void	setTrace(boolean trace)
Explanation	toExplanation(StreamFactory factory)	Returns an explanation about the stream object
StreamExpression	toExpression(StreamFactory factory)

Methods inherited from class org.apache.solr.client.solrj.io.stream.TupleStream

buildSolrParamsExcept, getCost, getShards, getShards, getStreamNodeId, writeMap

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.solr.common.MapWriter

jsonStr, write

Methods inherited from interface org.apache.solr.common.NavigableObject

_forEachEntry, _forEachEntry, _get, _get, _getStr, _getStr, _size

Field Details

solrConnection

protected org.apache.solr.client.solrj.impl.CloudSolrClient.CloudSolrClientConnection solrConnection

collection

protected String collection

path

protected String path

params

protected org.apache.solr.common.params.SolrParams params

fieldMappings

protected Map<String,String> fieldMappings

comp

protected StreamComparator comp

eofTuples

protected transient Map<String,Tuple> eofTuples

solrStreams

protected transient List<TupleStream> solrStreams

tuples

protected transient TreeSet<CloudSolrStream.TupleWrapper> tuples

streamContext

protected transient StreamContext streamContext

Constructor Details

CloudSolrStream

protected CloudSolrStream()

CloudSolrStream

public CloudSolrStream(org.apache.solr.client.solrj.impl.CloudSolrClient.CloudSolrClientConnection solrConnection,
 String collectionName,
 org.apache.solr.common.params.SolrParams params)
                throws IOException

Parameters:

solrConnection - Zookeeper or HTTPS(s) ensemble connection string

collectionName - Name of the collection to operate on

params - Map<String, String[]> of parameter/value pairs

Throws:

IOException - Something went wrong

CloudSolrStream

public CloudSolrStream(org.apache.solr.client.solrj.impl.CloudSolrClient.CloudSolrClientConnection solrConnection,
 String collectionName,
 String path,
 org.apache.solr.common.params.SolrParams params)
                throws IOException

Parameters:

solrConnection - Zookeeper or HTTPS(s) ensemble connection string

collectionName - Name of the collection to operate on

path - the request handler path to query (e.g. "/export"). If not provided (i.e. null), the handler is instead resolved from a "qt" param embedded in params, or defaults to "/select" if no such param is present.

params - Map<String, String[]> of parameter/value pairs

Throws:

IOException - Something went wrong

CloudSolrStream

public CloudSolrStream(StreamExpression expression,
 StreamFactory factory)
                throws IOException

Throws:

IOException

Method Details

toExpression

public StreamExpression toExpression(StreamFactory factory)
                              throws IOException

Specified by:

toExpression in interface Expressible

Throws:

IOException

toExplanation

public Explanation toExplanation(StreamFactory factory)
                          throws IOException

Description copied from interface: Expressible

Returns an explanation about the stream object

Specified by:

toExplanation in interface Expressible

Specified by:

toExplanation in class TupleStream

Parameters:

factory - Stream factory for this, contains information about the function name

Returns:

Explanation about this stream object containing explanations of any child stream objects

Throws:

IOException - throw on any error

setFieldMappings

public void setFieldMappings(Map<String,String> fieldMappings)

setTrace

public void setTrace(boolean trace)

setStreamContext

public void setStreamContext(StreamContext context)

Specified by:

setStreamContext in class TupleStream

open

public void open()
          throws IOException

Opens the CloudSolrStream

Specified by:

open in class TupleStream

Throws:

IOException

getEofTuples

public Map<String,Tuple> getEofTuples()

children

public List<TupleStream> children()

Specified by:

children in class TupleStream

getSlices

public static List<org.apache.solr.common.cloud.Slice> getSlices(String collectionName,
 org.apache.solr.client.solrj.impl.CloudSolrClient cloudSolrClient,
 boolean checkAlias)
                                                          throws IOException

Throws:

IOException

constructStreams

protected void constructStreams()
                         throws IOException

Throws:

IOException

close

public void close()
           throws IOException

Closes the CloudSolrStream

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Specified by:

close in class TupleStream

Throws:

IOException

getStreamSort

public StreamComparator getStreamSort()

Return the stream sort - ie, the order in which records are returned

Specified by:

getStreamSort in class TupleStream

read

public Tuple read()
           throws IOException

Specified by:

read in class TupleStream

Throws:

IOException

_read

protected Tuple _read()
               throws IOException

Throws:

IOException

adjustParams

protected org.apache.solr.common.params.ModifiableSolrParams adjustParams(org.apache.solr.common.params.ModifiableSolrParams params)