Package org.apache.solr.common.util

Class JavaBinCodec

java.lang.Object

org.apache.solr.common.util.JavaBinCodec

All Implemented Interfaces:

Closeable, AutoCloseable, PushWriter

public class JavaBinCodec
extends Object
implements PushWriter

Defines a space-efficient serialization/deserialization format for transferring data.

JavaBinCodec has built in support many commonly used types. This includes primitive types (boolean, byte, short, double, int, long, float), common Java containers/utilities (Date, Map, Collection, Iterator, String, Object[], byte[]), and frequently used Solr types (NamedList, SolrDocument, SolrDocumentList). Each of the above types has a pair of associated methods which read and write that type to a stream.

Classes that aren't supported natively can still be serialized/deserialized by providing an JavaBinCodec.ObjectResolver object that knows how to work with the unsupported class. This allows JavaBinCodec to be used to marshall/unmarshall arbitrary content.

NOTE -- JavaBinCodec instances cannot be reused for more than one marshall or unmarshall operation.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
class	JavaBinCodec.BinEntryWriter
static interface	JavaBinCodec.ObjectResolver	Allows extension of JavaBinCodec to support serialization of arbitrary data types.
static class	JavaBinCodec.StringCache
static interface	JavaBinCodec.WritableDocFields

Field Summary

Fields

Modifier and Type	Field	Description
static final byte	ARR
static final byte	BOOL_FALSE
static final byte	BOOL_TRUE
static final byte	BYTE
static final byte	BYTEARR
protected FastOutputStream	daos
static final byte	DATE
static final byte	DOUBLE
static final byte	END
protected static final Object	END_OBJ
static final byte	ENUM_FIELD_VALUE
final JavaBinCodec.BinEntryWriter	ew
static final byte	EXTERN_STRING
static final byte	FLOAT
static final byte	INT
final IteratorWriter.ItemWriter	itemWriter
static final byte	ITERATOR
static final byte	LONG
static final byte	MAP
static final byte	MAP_ENTRY
static final byte	MAP_ENTRY_ITER
static final byte	NAMED_LST
static final byte	NULL
static final byte	ORDERED_MAP
static final byte	PRIMITIVE_ARR
protected boolean	readStringAsCharSeq
static final byte	SHORT
static final byte	SINT
static final byte	SLONG
static final byte	SOLRDOC
static final byte	SOLRDOCLST
static final byte	SOLRINPUTDOC
static final byte	STR
static final byte	TAG_AND_LEN
protected byte	tagByte
static final byte	UUID

Constructor Summary

Constructors

Constructor	Description
JavaBinCodec()
JavaBinCodec(OutputStream os, JavaBinCodec.ObjectResolver resolver)	Use this to use this as a PushWriter.
JavaBinCodec(JavaBinCodec.ObjectResolver resolver)
JavaBinCodec(JavaBinCodec.ObjectResolver resolver, JavaBinCodec.StringCache stringCache)

Method Summary

Modifier and Type	Method	Description
protected FastInputStream	_init(FastInputStream dis)
void	close()
protected SolrInputDocument	createSolrInputDocument(int sz)
JavaBinCodec.ObjectResolver	getResolver()
long	getTotalBytesWritten()
void	init(FastOutputStream os)	expert: sets a new output stream
protected FastInputStream	initRead(byte[] buf)
protected FastInputStream	initRead(InputStream is)
protected void	initWrite(OutputStream os)
void	marshal(Object nl, OutputStream os)
protected Map<Object,Object>	newMap(int size)	create a new Map object
List<Object>	readArray(DataInputInputStream dis)
protected List	readArray(DataInputInputStream dis, int sz)
byte[]	readByteArray(DataInputInputStream dis)
EnumFieldValue	readEnumFieldValue(DataInputInputStream dis)	read EnumFieldValue (int+string) from input stream
CharSequence	readExternString(DataInputInputStream fis)
List<Object>	readIterator(DataInputInputStream fis)
Map<?,Object>	readMap(DataInputInputStream dis)
protected Map<Object,Object>	readMap(DataInputInputStream dis, int sz)
void	readMapAsNamedList(boolean readMapAsNamedList)	If set, Maps will be deserialized as SimpleOrderedMap instead of LinkedHashMap
protected Map<?,Object>	readMapAsSimpleOrderedMapForStringKeys(DataInputInputStream dis, int sz)
Map.Entry<Object,Object>	readMapEntry(DataInputInputStream dis)
Map<Object,Object>	readMapIter(DataInputInputStream dis)
NamedList<Object>	readNamedList(DataInputInputStream dis)
protected Object	readObject(DataInputInputStream dis)
SimpleOrderedMap<Object>	readOrderedMap(DataInputInputStream dis)
Object	readPrimitiveArray(DataInputInputStream dis)
int	readSize(DataInputInputStream in)
int	readSmallInt(DataInputInputStream dis)
long	readSmallLong(DataInputInputStream dis)
SolrDocument	readSolrDocument(DataInputInputStream dis)
SolrDocumentList	readSolrDocumentList(DataInputInputStream dis)
SolrInputDocument	readSolrInputDocument(DataInputInputStream dis)
CharSequence	readStr(DataInputInputStream dis)
CharSequence	readStr(DataInputInputStream dis, JavaBinCodec.StringCache stringCache, boolean readStringAsCharSeq)
protected CharSequence	readUtf8(DataInputInputStream dis)
protected CharSequence	readUtf8(DataInputInputStream dis, int sz)
Object	readVal(DataInputInputStream dis)
static int	readVInt(DataInputInputStream in)	The counterpart for writeVInt(int, FastOutputStream)
static long	readVLong(DataInputInputStream in)
JavaBinCodec	setReadStringAsCharSeq(boolean flag)
JavaBinCodec	setWritableDocFields(JavaBinCodec.WritableDocFields writableDocFields)
protected boolean	toWrite(String key)
Object	unmarshal(byte[] buf)
Object	unmarshal(InputStream is)
void	writeArray(Object[] arr)
void	writeArray(Collection<?> coll)
void	writeArray(List<?> l)
void	writeBoolArr(boolean[] vals)
protected void	writeBoolean(boolean val)
void	writeByteArray(byte[] arr, int offset, int len)
protected void	writeDouble(double val)
void	writeDoubleArr(double[] vals)
void	writeEnumFieldValue(EnumFieldValue enumFieldValue)	write EnumFieldValue as tag+int value+string value
void	writeExternString(CharSequence s)
void	writeFloat(float val)
void	writeFloatArr(float[] vals)
void	writeInt(int val)
void	writeIntArr(int[] vals)
void	writeIterator(Iterator<?> iter)
void	writeIterator(IteratorWriter val)	Write an array.
boolean	writeKnownType(Object val)
void	writeLong(long val)
void	writeLongArr(long[] vals)
void	writeMap(Map<?,?> val)
void	writeMap(MapWriter val)	Write a Map.
void	writeMapEntry(Map.Entry<?,?> val)
void	writeNamedList(NamedList<?> nl)
boolean	writePrimitive(Object val)
void	writePrimitiveArrHeader(byte tag, int len)
void	writeShortArr(short[] vals)
void	writeSolrDocument(SolrDocument doc)
void	writeSolrDocumentList(SolrDocumentList docs)
void	writeSolrInputDocument(SolrInputDocument sdoc)
void	writeStr(CharSequence s)	write the string as tag+length, with length being the number of UTF-8 bytes
void	writeTag(byte tag)
void	writeTag(byte tag, int size)
void	writeUTF8Str(Utf8CharSequence utf8)
void	writeVal(Object val)
static void	writeVInt(int i, FastOutputStream out)	Special method for variable length int (copied from lucene).
static void	writeVLong(long i, FastOutputStream out)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

NULL

public static final byte NULL

See Also:

• Constant Field Values

BOOL_TRUE

public static final byte BOOL_TRUE

See Also:

• Constant Field Values

BOOL_FALSE

public static final byte BOOL_FALSE

See Also:

• Constant Field Values

BYTE

public static final byte BYTE

See Also:

• Constant Field Values

SHORT

public static final byte SHORT

See Also:

• Constant Field Values

DOUBLE

public static final byte DOUBLE

See Also:

• Constant Field Values

INT

public static final byte INT

See Also:

• Constant Field Values

LONG

public static final byte LONG

See Also:

• Constant Field Values

FLOAT

public static final byte FLOAT

See Also:

• Constant Field Values

DATE

public static final byte DATE

See Also:

• Constant Field Values

MAP

public static final byte MAP

See Also:

• Constant Field Values

SOLRDOC

public static final byte SOLRDOC

See Also:

• Constant Field Values

SOLRDOCLST

public static final byte SOLRDOCLST

See Also:

• Constant Field Values

BYTEARR

public static final byte BYTEARR

See Also:

• Constant Field Values

ITERATOR

public static final byte ITERATOR

See Also:

• Constant Field Values

END

public static final byte END

See Also:

• Constant Field Values

SOLRINPUTDOC

public static final byte SOLRINPUTDOC

See Also:

• Constant Field Values

MAP_ENTRY_ITER

public static final byte MAP_ENTRY_ITER

See Also:

• Constant Field Values

ENUM_FIELD_VALUE

public static final byte ENUM_FIELD_VALUE

See Also:

• Constant Field Values

MAP_ENTRY

public static final byte MAP_ENTRY

See Also:

• Constant Field Values

UUID

public static final byte UUID

See Also:

• Constant Field Values

PRIMITIVE_ARR

public static final byte PRIMITIVE_ARR

See Also:

• Constant Field Values

TAG_AND_LEN

public static final byte TAG_AND_LEN

See Also:

• Constant Field Values

STR

public static final byte STR

See Also:

• Constant Field Values

SINT

public static final byte SINT

See Also:

• Constant Field Values

SLONG

public static final byte SLONG

See Also:

• Constant Field Values

ARR

public static final byte ARR

See Also:

• Constant Field Values

ORDERED_MAP

public static final byte ORDERED_MAP

See Also:

• Constant Field Values

NAMED_LST

public static final byte NAMED_LST

See Also:

• Constant Field Values

EXTERN_STRING

public static final byte EXTERN_STRING

See Also:

• Constant Field Values

daos

protected FastOutputStream daos

readStringAsCharSeq

protected boolean readStringAsCharSeq

END_OBJ

protected static final Object END_OBJ

tagByte

protected byte tagByte

ew

public final JavaBinCodec.BinEntryWriter ew

itemWriter

public final IteratorWriter.ItemWriter itemWriter

Constructor Details

JavaBinCodec

public JavaBinCodec()

JavaBinCodec

public JavaBinCodec(OutputStream os,
 JavaBinCodec.ObjectResolver resolver)
             throws IOException

Use this to use this as a PushWriter. ensure that close() is called explicitly after use

Parameters:

os - The output stream

Throws:

IOException

JavaBinCodec

public JavaBinCodec(JavaBinCodec.ObjectResolver resolver)

JavaBinCodec

public JavaBinCodec(JavaBinCodec.ObjectResolver resolver,
 JavaBinCodec.StringCache stringCache)

Method Details

setReadStringAsCharSeq

public JavaBinCodec setReadStringAsCharSeq(boolean flag)

setWritableDocFields

public JavaBinCodec setWritableDocFields(JavaBinCodec.WritableDocFields writableDocFields)

getResolver

public JavaBinCodec.ObjectResolver getResolver()

marshal

public void marshal(Object nl,
 OutputStream os)
             throws IOException

Throws:

IOException

initWrite

protected void initWrite(OutputStream os)
                  throws IOException

Throws:

IOException

init

public void init(FastOutputStream os)

expert: sets a new output stream

unmarshal

public Object unmarshal(byte[] buf)
                 throws IOException

Throws:

IOException

unmarshal

public Object unmarshal(InputStream is)
                 throws IOException

Throws:

IOException

initRead

protected FastInputStream initRead(InputStream is)
                            throws IOException

Throws:

IOException

initRead

protected FastInputStream initRead(byte[] buf)
                            throws IOException

Throws:

IOException

_init

protected FastInputStream _init(FastInputStream dis)
                         throws IOException

Throws:

IOException

readOrderedMap

public SimpleOrderedMap<Object> readOrderedMap(DataInputInputStream dis)
                                        throws IOException

Throws:

IOException

readNamedList

public NamedList<Object> readNamedList(DataInputInputStream dis)
                                throws IOException

Throws:

IOException

writeNamedList

public void writeNamedList(NamedList<?> nl)
                    throws IOException

Throws:

IOException

writeVal

public void writeVal(Object val)
              throws IOException

Throws:

IOException

readVal

public Object readVal(DataInputInputStream dis)
               throws IOException

Throws:

IOException

readObject

protected Object readObject(DataInputInputStream dis)
                     throws IOException

Throws:

IOException

writeKnownType

public boolean writeKnownType(Object val)
                       throws IOException

Throws:

IOException

readPrimitiveArray

public Object readPrimitiveArray(DataInputInputStream dis)
                          throws IOException

Throws:

IOException

writePrimitiveArrHeader

public void writePrimitiveArrHeader(byte tag,
 int len)
                             throws IOException

Throws:

IOException

writeFloatArr

public void writeFloatArr(float[] vals)
                   throws IOException

Throws:

IOException

writeIntArr

public void writeIntArr(int[] vals)
                 throws IOException

Throws:

IOException

writeDoubleArr

public void writeDoubleArr(double[] vals)
                    throws IOException

Throws:

IOException

writeLongArr

public void writeLongArr(long[] vals)
                  throws IOException

Throws:

IOException

writeBoolArr

public void writeBoolArr(boolean[] vals)
                  throws IOException

Throws:

IOException

writeShortArr

public void writeShortArr(short[] vals)
                   throws IOException

Throws:

IOException

writeMap

public void writeMap(MapWriter val)
              throws IOException

Description copied from interface: PushWriter

Write a Map. The map is opened in the beginning of the method and closed at the end. All map entries MUST be written before this method returns

Specified by:

writeMap in interface PushWriter

Throws:

IOException

writeTag

public void writeTag(byte tag)
              throws IOException

Throws:

IOException

writeTag

public void writeTag(byte tag,
 int size)
              throws IOException

Throws:

IOException

writeByteArray

public void writeByteArray(byte[] arr,
 int offset,
 int len)
                    throws IOException

Throws:

IOException

readByteArray

public byte[] readByteArray(DataInputInputStream dis)
                     throws IOException

Throws:

IOException

writeSolrDocument

public void writeSolrDocument(SolrDocument doc)
                       throws IOException

Throws:

IOException

toWrite

protected boolean toWrite(String key)

readSolrDocument

public SolrDocument readSolrDocument(DataInputInputStream dis)
                              throws IOException

Throws:

IOException

readSolrDocumentList

public SolrDocumentList readSolrDocumentList(DataInputInputStream dis)
                                      throws IOException

Throws:

IOException

writeSolrDocumentList

public void writeSolrDocumentList(SolrDocumentList docs)
                           throws IOException

Throws:

IOException

readSolrInputDocument

public SolrInputDocument readSolrInputDocument(DataInputInputStream dis)
                                        throws IOException

Throws:

IOException

createSolrInputDocument

protected SolrInputDocument createSolrInputDocument(int sz)

writeSolrInputDocument

public void writeSolrInputDocument(SolrInputDocument sdoc)
                            throws IOException

Throws:

IOException

readMapIter

public Map<Object,Object> readMapIter(DataInputInputStream dis)
                               throws IOException

Throws:

IOException

newMap

protected Map<Object,Object> newMap(int size)

create a new Map object

Parameters:

size - expected size, -1 means unknown size

readMap

public Map<?,Object> readMap(DataInputInputStream dis)
                      throws IOException

Throws:

IOException

readMap

protected Map<Object,Object> readMap(DataInputInputStream dis,
 int sz)
                              throws IOException

Throws:

IOException

readMapAsSimpleOrderedMapForStringKeys

protected Map<?,Object> readMapAsSimpleOrderedMapForStringKeys(DataInputInputStream dis,
 int sz)
                                                        throws IOException

Throws:

IOException

writeIterator

public void writeIterator(IteratorWriter val)
                   throws IOException

Description copied from interface: PushWriter

Write an array. The array is opened at the beginning of this method and closed at the end. All array entries must be written before this method returns

Specified by:

writeIterator in interface PushWriter

Throws:

IOException

writeIterator

public void writeIterator(Iterator<?> iter)
                   throws IOException

Throws:

IOException

readIterator

public List<Object> readIterator(DataInputInputStream fis)
                          throws IOException

Throws:

IOException

writeArray

public void writeArray(List<?> l)
                throws IOException

Throws:

IOException

writeArray

public void writeArray(Collection<?> coll)
                throws IOException

Throws:

IOException

writeArray

public void writeArray(Object[] arr)
                throws IOException

Throws:

IOException

readArray

public List<Object> readArray(DataInputInputStream dis)
                       throws IOException

Throws:

IOException

readArray

protected List readArray(DataInputInputStream dis,
 int sz)
                  throws IOException

Throws:

IOException

writeEnumFieldValue

public void writeEnumFieldValue(EnumFieldValue enumFieldValue)
                         throws IOException

write EnumFieldValue as tag+int value+string value

Parameters:

enumFieldValue - to write

Throws:

IOException

writeMapEntry

public void writeMapEntry(Map.Entry<?,?> val)
                   throws IOException

Throws:

IOException

readEnumFieldValue

public EnumFieldValue readEnumFieldValue(DataInputInputStream dis)
                                  throws IOException

read EnumFieldValue (int+string) from input stream

Parameters:

dis - data input stream

Returns:

EnumFieldValue

Throws:

IOException

readMapEntry

public Map.Entry<Object,Object> readMapEntry(DataInputInputStream dis)
                                      throws IOException

Throws:

IOException

writeStr

public void writeStr(CharSequence s)
              throws IOException

write the string as tag+length, with length being the number of UTF-8 bytes

Throws:

IOException

readStr

public CharSequence readStr(DataInputInputStream dis)
                     throws IOException

Throws:

IOException

readStr

public CharSequence readStr(DataInputInputStream dis,
 JavaBinCodec.StringCache stringCache,
 boolean readStringAsCharSeq)
                     throws IOException

Throws:

IOException

readUtf8

protected CharSequence readUtf8(DataInputInputStream dis)
                         throws IOException

Throws:

IOException

readUtf8

protected CharSequence readUtf8(DataInputInputStream dis,
 int sz)
                         throws IOException

Throws:

IOException

writeInt

public void writeInt(int val)
              throws IOException

Throws:

IOException

readSmallInt

public int readSmallInt(DataInputInputStream dis)
                 throws IOException

Throws:

IOException

writeLong

public void writeLong(long val)
               throws IOException

Throws:

IOException

readSmallLong

public long readSmallLong(DataInputInputStream dis)
                   throws IOException

Throws:

IOException

writeFloat

public void writeFloat(float val)
                throws IOException

Throws:

IOException

writePrimitive

public boolean writePrimitive(Object val)
                       throws IOException

Throws:

IOException

writeBoolean

protected void writeBoolean(boolean val)
                     throws IOException

Throws:

IOException

writeDouble

protected void writeDouble(double val)
                    throws IOException

Throws:

IOException

writeMap

public void writeMap(Map<?,?> val)
              throws IOException

Throws:

IOException

readSize

public int readSize(DataInputInputStream in)
             throws IOException

Throws:

IOException

writeVInt

public static void writeVInt(int i,
 FastOutputStream out)
                      throws IOException

Special method for variable length int (copied from lucene). Usually used for writing the length of a collection/array/map In most of the cases the length can be represented in one byte (length < 127) so it saves 3 bytes/object

Throws:

IOException - If there is a low-level I/O error.

readVInt

public static int readVInt(DataInputInputStream in)
                    throws IOException

The counterpart for writeVInt(int, FastOutputStream)

Throws:

IOException - If there is a low-level I/O error.

writeVLong

public static void writeVLong(long i,
 FastOutputStream out)
                       throws IOException

Throws:

IOException

readVLong

public static long readVLong(DataInputInputStream in)
                      throws IOException

Throws:

IOException

writeExternString

public void writeExternString(CharSequence s)
                       throws IOException

Throws:

IOException

readExternString

public CharSequence readExternString(DataInputInputStream fis)
                              throws IOException

Throws:

IOException

writeUTF8Str

public void writeUTF8Str(Utf8CharSequence utf8)
                  throws IOException

Throws:

IOException

getTotalBytesWritten

public long getTotalBytesWritten()

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Throws:

IOException

readMapAsNamedList

public void readMapAsNamedList(boolean readMapAsNamedList)

If set, Maps will be deserialized as SimpleOrderedMap instead of LinkedHashMap