Package org.apache.solr.update
Class TransactionLog
- java.lang.Object
-
- org.apache.solr.update.TransactionLog
-
- All Implemented Interfaces:
Closeable,AutoCloseable
public class TransactionLog extends Object implements Closeable
Log Format: List{Operation, Version, ...} ADD, VERSION, DOC DELETE, VERSION, ID_BYTES DELETE_BY_QUERY, VERSION, StringTODO: keep two files, one for [operation, version, id] and the other for the actual document data. That way we could throw away document log files more readily while retaining the smaller operation log files longer (and we can retrieve the stored fields from the latest documents from the index).
This would require keeping all source fields stored of course.
This would also allow to not log document data for requests with commit=true in them (since we know that if the request succeeds, all docs will be committed)
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description classTransactionLog.FSReverseReaderclassTransactionLog.LogCodecclassTransactionLog.LogReaderstatic classTransactionLog.ReverseReaderclassTransactionLog.SortedLogReader
-
Field Summary
Fields Modifier and Type Field Description protected booleandeleteOnClosestatic StringEND_MESSAGEprotected org.apache.solr.common.util.FastOutputStreamfosprotected List<String>globalStringListprotected Map<String,Integer>globalStringMapbooleanisBufferprotected AtomicIntegerrefcountprotected static org.apache.solr.common.util.JavaBinCodec.ObjectResolverresolver
-
Constructor Summary
Constructors Modifier Constructor Description protectedTransactionLog()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description protected voidaddGlobalStrings(Collection<String> strings)protected voidcheckWriteHeader(TransactionLog.LogCodec codec, org.apache.solr.common.SolrInputDocument optional)voidclose()voidcloseOutput()Move to a read-only state, closing and releasing resources while keeping the log available for readsvoiddecref()protected voidendRecord(long startRecordPosition)booleanendsWithCommit()voidfinish(UpdateLog.SyncLevel syncLevel)voidforceClose()longgetLogSize()longgetLogSizeFromStream()TransactionLog.LogReadergetReader(long startingPos)Returns a reader that can be used while a log is still in use.TransactionLog.ReverseReadergetReverseReader()Returns a single threaded reverse readerTransactionLog.LogReadergetSortedReader(long startingPos)voidincref()Objectlookup(long pos)intnumRecords()Returns the number of records in the log (currently includes the header and an optional commit).longposition()returns the current position in the log fileStringtoString()booleantry_incref()longwrite(AddUpdateCommand cmd)Writes an add update command to the transaction log.longwrite(AddUpdateCommand cmd, long prevPointer)Writes an add update command to the transaction log.longwriteCommit(CommitUpdateCommand cmd)longwriteData(Object o)longwriteDelete(DeleteUpdateCommand cmd)longwriteDeleteByQuery(DeleteUpdateCommand cmd)protected voidwriteLogHeader(TransactionLog.LogCodec codec)
-
-
-
Field Detail
-
END_MESSAGE
public static final String END_MESSAGE
- See Also:
- Constant Field Values
-
fos
protected org.apache.solr.common.util.FastOutputStream fos
-
isBuffer
public boolean isBuffer
-
deleteOnClose
protected volatile boolean deleteOnClose
-
refcount
protected AtomicInteger refcount
-
resolver
protected static final org.apache.solr.common.util.JavaBinCodec.ObjectResolver resolver
-
-
Method Detail
-
numRecords
public int numRecords()
Returns the number of records in the log (currently includes the header and an optional commit). Note: currently returns 0 for reopened existing log files.
-
endsWithCommit
public boolean endsWithCommit() throws IOException- Throws:
IOException
-
writeData
public long writeData(Object o)
-
addGlobalStrings
protected void addGlobalStrings(Collection<String> strings)
-
writeLogHeader
protected void writeLogHeader(TransactionLog.LogCodec codec) throws IOException
- Throws:
IOException
-
endRecord
protected void endRecord(long startRecordPosition) throws IOException- Throws:
IOException
-
checkWriteHeader
protected void checkWriteHeader(TransactionLog.LogCodec codec, org.apache.solr.common.SolrInputDocument optional) throws IOException
- Throws:
IOException
-
write
public long write(AddUpdateCommand cmd)
Writes an add update command to the transaction log. This is not applicable for in-place updates; usewrite(AddUpdateCommand, long). (The previous pointer (applicable for in-place updates) is set to -1 while writing the command to the transaction log.)- Parameters:
cmd- The add update command to be written- Returns:
- Returns the position pointer of the written update command
- See Also:
write(AddUpdateCommand, long)
-
write
public long write(AddUpdateCommand cmd, long prevPointer)
Writes an add update command to the transaction log. This should be called only for writing in-place updates, or else pass -1 as the prevPointer.- Parameters:
cmd- The add update command to be writtenprevPointer- The pointer in the transaction log which this update depends on (applicable for in-place updates)- Returns:
- Returns the position pointer of the written update command
-
writeDelete
public long writeDelete(DeleteUpdateCommand cmd)
-
writeDeleteByQuery
public long writeDeleteByQuery(DeleteUpdateCommand cmd)
-
writeCommit
public long writeCommit(CommitUpdateCommand cmd)
-
lookup
public Object lookup(long pos)
-
incref
public void incref()
-
try_incref
public boolean try_incref()
-
decref
public void decref()
-
position
public long position()
returns the current position in the log file
-
closeOutput
public void closeOutput()
Move to a read-only state, closing and releasing resources while keeping the log available for reads
-
finish
public void finish(UpdateLog.SyncLevel syncLevel)
-
close
public void close()
- Specified by:
closein interfaceAutoCloseable- Specified by:
closein interfaceCloseable
-
forceClose
public void forceClose()
-
getLogSize
public long getLogSize()
-
getLogSizeFromStream
public long getLogSizeFromStream()
- Returns:
- the FastOutputStream size
-
getReader
public TransactionLog.LogReader getReader(long startingPos)
Returns a reader that can be used while a log is still in use. Currently only *one* LogReader may be outstanding, and that log may only be used from a single thread.
-
getSortedReader
public TransactionLog.LogReader getSortedReader(long startingPos)
-
getReverseReader
public TransactionLog.ReverseReader getReverseReader() throws IOException
Returns a single threaded reverse reader- Throws:
IOException
-
-