Package org.apache.solr.update
Class TransactionLog
- java.lang.Object
-
- org.apache.solr.update.TransactionLog
-
- All Implemented Interfaces:
Closeable
,AutoCloseable
public class TransactionLog extends Object implements Closeable
Log Format: List{Operation, Version, ...} ADD, VERSION, DOC DELETE, VERSION, ID_BYTES DELETE_BY_QUERY, VERSION, StringTODO: keep two files, one for [operation, version, id] and the other for the actual document data. That way we could throw away document log files more readily while retaining the smaller operation log files longer (and we can retrieve the stored fields from the latest documents from the index).
This would require keeping all source fields stored of course.
This would also allow to not log document data for requests with commit=true in them (since we know that if the request succeeds, all docs will be committed)
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description class
TransactionLog.FSReverseReader
class
TransactionLog.LogCodec
class
TransactionLog.LogReader
class
TransactionLog.ReverseReader
class
TransactionLog.SortedLogReader
-
Field Summary
Fields Modifier and Type Field Description protected boolean
deleteOnClose
static String
END_MESSAGE
protected FastOutputStream
fos
protected List<String>
globalStringList
protected Map<String,Integer>
globalStringMap
boolean
isBuffer
protected AtomicInteger
refcount
protected static JavaBinCodec.ObjectResolver
resolver
-
Constructor Summary
Constructors Modifier Constructor Description protected
TransactionLog()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description protected void
addGlobalStrings(Collection<String> strings)
protected void
checkWriteHeader(TransactionLog.LogCodec codec, SolrInputDocument optional)
void
close()
void
closeOutput()
Move to a read-only state, closing and releasing resources while keeping the log available for readsvoid
decref()
protected void
endRecord(long startRecordPosition)
boolean
endsWithCommit()
void
finish(UpdateLog.SyncLevel syncLevel)
void
forceClose()
long
getLogSize()
long
getLogSizeFromStream()
TransactionLog.LogReader
getReader(long startingPos)
Returns a reader that can be used while a log is still in use.TransactionLog.ReverseReader
getReverseReader()
Returns a single threaded reverse readerTransactionLog.LogReader
getSortedReader(long startingPos)
void
incref()
Object
lookup(long pos)
int
numRecords()
Returns the number of records in the log (currently includes the header and an optional commit).long
position()
returns the current position in the log fileString
toString()
boolean
try_incref()
long
write(AddUpdateCommand cmd)
Writes an add update command to the transaction log.long
write(AddUpdateCommand cmd, long prevPointer)
Writes an add update command to the transaction log.long
writeCommit(CommitUpdateCommand cmd)
long
writeData(Object o)
long
writeDelete(DeleteUpdateCommand cmd)
long
writeDeleteByQuery(DeleteUpdateCommand cmd)
protected void
writeLogHeader(TransactionLog.LogCodec codec)
-
-
-
Field Detail
-
END_MESSAGE
public static final String END_MESSAGE
- See Also:
- Constant Field Values
-
fos
protected FastOutputStream fos
-
isBuffer
public boolean isBuffer
-
deleteOnClose
protected volatile boolean deleteOnClose
-
refcount
protected AtomicInteger refcount
-
resolver
protected static final JavaBinCodec.ObjectResolver resolver
-
-
Method Detail
-
numRecords
public int numRecords()
Returns the number of records in the log (currently includes the header and an optional commit). Note: currently returns 0 for reopened existing log files.
-
endsWithCommit
public boolean endsWithCommit() throws IOException
- Throws:
IOException
-
writeData
public long writeData(Object o)
-
addGlobalStrings
protected void addGlobalStrings(Collection<String> strings)
-
writeLogHeader
protected void writeLogHeader(TransactionLog.LogCodec codec) throws IOException
- Throws:
IOException
-
endRecord
protected void endRecord(long startRecordPosition) throws IOException
- Throws:
IOException
-
checkWriteHeader
protected void checkWriteHeader(TransactionLog.LogCodec codec, SolrInputDocument optional) throws IOException
- Throws:
IOException
-
write
public long write(AddUpdateCommand cmd)
Writes an add update command to the transaction log. This is not applicable for in-place updates; usewrite(AddUpdateCommand, long)
. (The previous pointer (applicable for in-place updates) is set to -1 while writing the command to the transaction log.)- Parameters:
cmd
- The add update command to be written- Returns:
- Returns the position pointer of the written update command
- See Also:
write(AddUpdateCommand, long)
-
write
public long write(AddUpdateCommand cmd, long prevPointer)
Writes an add update command to the transaction log. This should be called only for writing in-place updates, or else pass -1 as the prevPointer.- Parameters:
cmd
- The add update command to be writtenprevPointer
- The pointer in the transaction log which this update depends on (applicable for in-place updates)- Returns:
- Returns the position pointer of the written update command
-
writeDelete
public long writeDelete(DeleteUpdateCommand cmd)
-
writeDeleteByQuery
public long writeDeleteByQuery(DeleteUpdateCommand cmd)
-
writeCommit
public long writeCommit(CommitUpdateCommand cmd)
-
lookup
public Object lookup(long pos)
-
incref
public void incref()
-
try_incref
public boolean try_incref()
-
decref
public void decref()
-
position
public long position()
returns the current position in the log file
-
closeOutput
public void closeOutput()
Move to a read-only state, closing and releasing resources while keeping the log available for reads
-
finish
public void finish(UpdateLog.SyncLevel syncLevel)
-
close
public void close()
- Specified by:
close
in interfaceAutoCloseable
- Specified by:
close
in interfaceCloseable
-
forceClose
public void forceClose()
-
getLogSize
public long getLogSize()
-
getLogSizeFromStream
public long getLogSizeFromStream()
- Returns:
- the FastOutputStream size
-
getReader
public TransactionLog.LogReader getReader(long startingPos)
Returns a reader that can be used while a log is still in use. Currently only *one* LogReader may be outstanding, and that log may only be used from a single thread.
-
getSortedReader
public TransactionLog.LogReader getSortedReader(long startingPos)
-
getReverseReader
public TransactionLog.ReverseReader getReverseReader() throws IOException
Returns a single threaded reverse reader- Throws:
IOException
-
-