Package org.apache.solr.hdfs.update

Class HdfsTransactionLog

java.lang.Object

org.apache.solr.update.TransactionLog

org.apache.solr.hdfs.update.HdfsTransactionLog

All Implemented Interfaces:

Closeable, AutoCloseable

public class HdfsTransactionLog
extends TransactionLog

Log Format: List{Operation, Version, ...} ADD, VERSION, DOC DELETE, VERSION, ID_BYTES DELETE_BY_QUERY, VERSION, String

TODO: keep two files, one for [operation, version, id] and the other for the actual document data. That way we could throw away document log files more readily while retaining the smaller operation log files longer (and we can retrieve the stored fields from the latest documents from the index).

This would require keeping all source fields stored of course.

This would also allow to not log document data for requests with commit=true in them (since we know that if the request succeeds, all docs will be committed)

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
class	HdfsTransactionLog.HDFSLogReader
class	HdfsTransactionLog.HDFSReverseReader
class	HdfsTransactionLog.HDFSSortedLogReader

Nested classes/interfaces inherited from class org.apache.solr.update.TransactionLog

TransactionLog.FSReverseReader, TransactionLog.LogCodec, TransactionLog.LogReader, TransactionLog.ReverseReader, TransactionLog.SortedLogReader

Field Summary

Fields inherited from class org.apache.solr.update.TransactionLog

deleteOnClose, END_MESSAGE, fos, globalStringList, globalStringMap, isBuffer, refcount, resolver

Method Summary

Modifier and Type	Method	Description
void	close()
void	closeOutput()
boolean	endsWithCommit()
void	finish(UpdateLog.SyncLevel syncLevel)
long	getLogSize()
TransactionLog.LogReader	getReader(long startingPos)	Returns a reader that can be used while a log is still in use.
TransactionLog.ReverseReader	getReverseReader()	Returns a single threaded reverse reader
TransactionLog.LogReader	getSortedReader(long startingPos)
Object	lookup(long pos)
String	toString()
long	writeCommit(CommitUpdateCommand cmd)

Methods inherited from class org.apache.solr.update.TransactionLog

addGlobalStrings, checkWriteHeader, decref, endRecord, forceClose, getLogSizeFromStream, incref, numRecords, position, try_incref, write, write, writeData, writeDelete, writeDeleteByQuery, writeLogHeader

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

endsWithCommit

public boolean endsWithCommit()
                       throws IOException

Overrides:

endsWithCommit in class TransactionLog

Throws:

IOException

writeCommit

public long writeCommit(CommitUpdateCommand cmd)

Overrides:

writeCommit in class TransactionLog

lookup

public Object lookup(long pos)

Overrides:

lookup in class TransactionLog

closeOutput

public void closeOutput()

Overrides:

closeOutput in class TransactionLog

getLogSize

public long getLogSize()

Overrides:

getLogSize in class TransactionLog

finish

public void finish(UpdateLog.SyncLevel syncLevel)

Overrides:

finish in class TransactionLog

close

public void close()

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Overrides:

close in class TransactionLog

toString

public String toString()

Overrides:

toString in class TransactionLog

getReader

public TransactionLog.LogReader getReader(long startingPos)

Returns a reader that can be used while a log is still in use. Currently only *one* LogReader may be outstanding, and that log may only be used from a single thread.

Overrides:

getReader in class TransactionLog

getSortedReader

public TransactionLog.LogReader getSortedReader(long startingPos)

Overrides:

getSortedReader in class TransactionLog

getReverseReader

public TransactionLog.ReverseReader getReverseReader()
                                              throws IOException

Returns a single threaded reverse reader

Overrides:

getReverseReader in class TransactionLog

Throws:

IOException