java.lang.Object
- org.apache.solr.update.UpdateLog

All Implemented Interfaces:

AutoCloseable, SolrMetricProducer, PluginInfoInitialized
```
public class UpdateLog
extends Object
implements PluginInfoInitialized, SolrMetricProducer
```
This holds references to the transaction logs. It also keeps a map of unique key to location in log (along with the update's version). This map is only cleared on soft or hard commit

WARNING: This API is experimental and might change in incompatible ways in the next release.

Nested Class Summary

Nested Classes
Modifier and Type	Class	Description
`static class`	`UpdateLog.DBQ`	Holds the query and the version for a DeleteByQuery command
`protected static class`	`UpdateLog.DeleteUpdate`
`static class`	`UpdateLog.LogPtr`
`class`	`UpdateLog.RecentUpdates`
`static class`	`UpdateLog.RecoveryInfo`
`static class`	`UpdateLog.State`
`static class`	`UpdateLog.SyncLevel`
`protected static class`	`UpdateLog.Update`

Field Summary

Fields
Modifier and Type	Field	Description
`static int`	`ADD`
`protected com.codahale.metrics.Meter`	`applyingBufferedOpsMeter`
`static String`	`BUFFER_TLOG_NAME`
`protected com.codahale.metrics.Gauge<Integer>`	`bufferedOpsGauge`
`protected TransactionLog`	`bufferTlog`
`protected boolean`	`cancelApplyBufferUpdate`
`static int`	`COMMIT`
`protected com.codahale.metrics.Meter`	`copyOverOldUpdatesMeter`
`protected String`	`dataDir`
`protected UpdateLog.SyncLevel`	`defaultSyncLevel`
`static int`	`DELETE`
`static int`	`DELETE_BY_QUERY`
`protected LinkedList<UpdateLog.DBQ>`	`deleteByQueries`
`protected boolean`	`existOldBufferLog`
`static int`	`FLAGS_IDX`	The index of the flags value in an entry from the transaction log.
`protected Collection<String>`	`globalStrings`
`protected long`	`id`
`protected String`	`lastDataDir`
`static String`	`LOG_FILENAME_PATTERN`
`protected Deque<TransactionLog>`	`logs`
`protected Map<org.apache.lucene.util.BytesRef,UpdateLog.LogPtr>`	`map`
`protected int`	`maxNumLogsToKeep`
`protected Deque<TransactionLog>`	`newestLogsOnStartup`
`protected int`	`numDeletesByQueryToKeep`
`protected int`	`numDeletesToKeep`
`protected int`	`numOldRecords`
`protected int`	`numRecordsToKeep`
`protected int`	`numVersionBuckets`
`protected LinkedHashMap<org.apache.lucene.util.BytesRef,UpdateLog.LogPtr>`	`oldDeletes`
`static int`	`OPERATION_MASK`
`static int`	`PREV_POINTER_IDX`	The index of the previous pointer in an entry from the transaction log.
`static int`	`PREV_VERSION_IDX`	The index of the previous version in an entry from the transaction log.
`protected Map<org.apache.lucene.util.BytesRef,UpdateLog.LogPtr>`	`prevMap`
`protected Map<org.apache.lucene.util.BytesRef,UpdateLog.LogPtr>`	`prevMap2`
`protected TransactionLog`	`prevMapLog`
`protected TransactionLog`	`prevMapLog2`
`protected TransactionLog`	`prevTlog`
`protected TransactionLog`	`prevTlogOnPrecommit`
`protected UpdateLog.RecoveryInfo`	`recoveryInfo`
`protected com.codahale.metrics.Meter`	`replayOpsMeter`
`protected SolrMetricsContext`	`solrMetricsContext`
`protected List<Long>`	`startingVersions`
`protected UpdateLog.State`	`state`
`static Runnable`	`testing_logReplayFinishHook`
`static Runnable`	`testing_logReplayHook`
`protected TransactionLog`	`tlog`
`static String`	`TLOG_NAME`
`protected Path`	`tlogDir`
`protected String[]`	`tlogFiles`
`protected UpdateHandler`	`uhandler`
`static int`	`UPDATE_INPLACE`
`static int`	`VERSION_IDX`	The index of the _version_ value in an entry from the transaction log.
`protected VersionInfo`	`versionInfo`

Constructor Summary

Constructors
Constructor Description

UpdateLog()

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method	Description
`void`	`add(AddUpdateCommand cmd)`
`void`	`add(AddUpdateCommand cmd, boolean clearCaches)`
`protected void`	`addOldLog(TransactionLog oldLog, boolean removeOld)`
`Future<UpdateLog.RecoveryInfo>`	`applyBufferedUpdates()`	Returns the Future to wait on, or null if no replay was needed
`long`	`applyPartialUpdates(org.apache.lucene.util.BytesRef id, long prevPointer, long prevVersion, Set<String> onlyTheseFields, org.apache.solr.common.SolrDocumentBase<?,?> latestPartialDoc)`	Goes over backwards, following the prevPointer, to merge all partial updates into the passed doc.
`void`	`bufferUpdates()`
`void`	`clearLog(SolrCore core, PluginInfo ulogPluginInfo)`	Clears the logs on the file system.
`void`	`close(boolean committed)`
`void`	`close(boolean committed, boolean deleteOnClose)`
`void`	`commitAndSwitchToNewTlog(CommitUpdateCommand cuc)`	Block updates, append a commit at current tlog, then copy over updates to a new tlog.
`static AddUpdateCommand`	`convertTlogEntryToAddUpdateCommand(SolrQueryRequest req, List<?> entry, int operation, long version)`	Given a entry from the transaction log containing a document, return a new AddUpdateCommand that can be applied to ADD the document or do an UPDATE_INPLACE.
`void`	`copyOverBufferingUpdates(CommitUpdateCommand cuc)`	Block updates, append a commit at current tlog, then copy over buffer updates to new tlog and bring back ulog to active state.
`void`	`copyOverOldUpdates(long commitVersion)`
`void`	`copyOverOldUpdates(long commitVersion, TransactionLog oldTlog)`	Copy over updates from prevTlog or last tlog (in tlog folder) to a new tlog
`void`	`delete(DeleteUpdateCommand cmd)`
`void`	`deleteAll()`	currently for testing only
`protected void`	`deleteBufferLogs()`
`void`	`deleteByQuery(DeleteUpdateCommand cmd)`
`static void`	`deleteFile(Path file)`
`boolean`	`dropBufferedUpdates()`	Returns true if we were able to drop buffered updates and return to the ACTIVE state
`protected void`	`ensureBufferTlog()`
`protected void`	`ensureLog()`
`boolean`	`existOldBufferLog()`
`void`	`finish(UpdateLog.SyncLevel syncLevel)`
`long`	`getCurrentLogSizeFromStream()`
`List<UpdateLog.DBQ>`	`getDBQNewer(long version)`
`long`	`getLastLogId()`
`String`	`getLogDir()`
`String[]`	`getLogList(File directory)`
`int`	`getMaxNumLogsToKeep()`
`int`	`getNumRecordsToKeep()`
`int`	`getNumVersionBuckets()`
`UpdateLog.RecentUpdates`	`getRecentUpdates()`	The RecentUpdates object returned must be closed after use
`SolrMetricsContext`	`getSolrMetricsContext()`	Implementations should return the context used in `SolrMetricProducer.initializeMetrics(SolrMetricsContext, String)` to ensure proper cleanup of metrics at the end of the life-cycle of this component.
`List<Long>`	`getStartingVersions()`
`UpdateLog.State`	`getState()`
`protected String`	`getTlogDir(SolrCore core, PluginInfo info)`
`long`	`getTotalLogsNumber()`
`long`	`getTotalLogsSize()`
`VersionInfo`	`getVersionInfo()`
`boolean`	`hasUncommittedChanges()`
`void`	`init(PluginInfo info)`
`void`	`init(UpdateHandler uhandler, SolrCore core)`
`void`	`initializeMetrics(SolrMetricsContext parentContext, String scope)`	Initialize metrics specific to this producer.
`Object`	`lookup(org.apache.lucene.util.BytesRef indexedId)`
`Long`	`lookupVersion(org.apache.lucene.util.BytesRef indexedId)`
`protected void`	`newMap()`
`TransactionLog`	`newTransactionLog(Path tlogFile, Collection<String> globalStrings, boolean openExisting)`	Returns a new `TransactionLog`.
`protected static int`	`objToInt(Object obj, int def)`
`void`	`openRealtimeSearcher()`	Opens a new realtime searcher and clears the id caches.
`void`	`postCommit(CommitUpdateCommand cmd)`
`void`	`postSoftCommit(CommitUpdateCommand cmd)`
`void`	`preCommit(CommitUpdateCommand cmd)`
`void`	`preSoftCommit(CommitUpdateCommand cmd)`
`Future<UpdateLog.RecoveryInfo>`	`recoverFromCurrentLog()`	Replay current tlog, so all updates will be written to index.
`Future<UpdateLog.RecoveryInfo>`	`recoverFromLog()`
`String`	`toString()`
`protected void`	`trackDeleteByQuery(String q, long version)`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.solr.metrics.SolrMetricProducer
close

Field Detail

LOG_FILENAME_PATTERN

public static String LOG_FILENAME_PATTERN

TLOG_NAME
```
public static String TLOG_NAME
```

BUFFER_TLOG_NAME
```
public static String BUFFER_TLOG_NAME
```

ADD
```
public static final int ADD
```
See Also:

Constant Field Values

DELETE
```
public static final int DELETE
```
See Also:

Constant Field Values

DELETE_BY_QUERY
```
public static final int DELETE_BY_QUERY
```
See Also:

Constant Field Values

COMMIT
```
public static final int COMMIT
```
See Also:

Constant Field Values

UPDATE_INPLACE
```
public static final int UPDATE_INPLACE
```
See Also:

Constant Field Values

OPERATION_MASK
```
public static final int OPERATION_MASK
```
See Also:

Constant Field Values

FLAGS_IDX
```
public static final int FLAGS_IDX
```
The index of the flags value in an entry from the transaction log.

See Also:

Constant Field Values

VERSION_IDX
```
public static final int VERSION_IDX
```
The index of the _version_ value in an entry from the transaction log.

See Also:

Constant Field Values

PREV_POINTER_IDX
```
public static final int PREV_POINTER_IDX
```
The index of the previous pointer in an entry from the transaction log. This is only relevant if flags (indexed at FLAGS_IDX) includes UPDATE_INPLACE.

See Also:

Constant Field Values

PREV_VERSION_IDX
```
public static final int PREV_VERSION_IDX
```
The index of the previous version in an entry from the transaction log. This is only relevant if flags (indexed at FLAGS_IDX) includes UPDATE_INPLACE.

See Also:

Constant Field Values

id
```
protected long id
```

state

protected volatile UpdateLog.State state

bufferTlog
```
protected TransactionLog bufferTlog
```

tlog
```
protected TransactionLog tlog
```

prevTlog
```
protected TransactionLog prevTlog
```

prevTlogOnPrecommit

protected TransactionLog prevTlogOnPrecommit

logs

protected final Deque<TransactionLog> logs

newestLogsOnStartup

protected Deque<TransactionLog> newestLogsOnStartup

numOldRecords
```
protected int numOldRecords
```

map

protected Map<org.apache.lucene.util.BytesRef,UpdateLog.LogPtr> map

prevMap

protected Map<org.apache.lucene.util.BytesRef,UpdateLog.LogPtr> prevMap

prevMap2

protected Map<org.apache.lucene.util.BytesRef,UpdateLog.LogPtr> prevMap2

prevMapLog
```
protected TransactionLog prevMapLog
```

prevMapLog2
```
protected TransactionLog prevMapLog2
```

numDeletesToKeep
```
protected final int numDeletesToKeep
```
See Also:

Constant Field Values

numDeletesByQueryToKeep
```
protected final int numDeletesByQueryToKeep
```
See Also:

Constant Field Values

numRecordsToKeep
```
protected int numRecordsToKeep
```

maxNumLogsToKeep
```
protected int maxNumLogsToKeep
```

numVersionBuckets
```
protected int numVersionBuckets
```

existOldBufferLog
```
protected boolean existOldBufferLog
```

oldDeletes

protected LinkedHashMap<org.apache.lucene.util.BytesRef,UpdateLog.LogPtr> oldDeletes

deleteByQueries

protected LinkedList<UpdateLog.DBQ> deleteByQueries

tlogFiles
```
protected String[] tlogFiles
```

tlogDir
```
protected Path tlogDir
```

globalStrings

protected Collection<String> globalStrings

dataDir
```
protected String dataDir
```

lastDataDir
```
protected String lastDataDir
```

versionInfo
```
protected VersionInfo versionInfo
```

defaultSyncLevel

protected UpdateLog.SyncLevel defaultSyncLevel

uhandler

protected volatile UpdateHandler uhandler

cancelApplyBufferUpdate

protected volatile boolean cancelApplyBufferUpdate

startingVersions
```
protected List<Long> startingVersions
```

bufferedOpsGauge

protected com.codahale.metrics.Gauge<Integer> bufferedOpsGauge

applyingBufferedOpsMeter

protected com.codahale.metrics.Meter applyingBufferedOpsMeter

replayOpsMeter

protected com.codahale.metrics.Meter replayOpsMeter

copyOverOldUpdatesMeter

protected com.codahale.metrics.Meter copyOverOldUpdatesMeter

solrMetricsContext

protected SolrMetricsContext solrMetricsContext

testing_logReplayHook

public static Runnable testing_logReplayHook

testing_logReplayFinishHook

public static Runnable testing_logReplayFinishHook

recoveryInfo

protected UpdateLog.RecoveryInfo recoveryInfo

Constructor Detail
- UpdateLog
```
public UpdateLog()
```

Method Detail
- getTotalLogsSize
```
public long getTotalLogsSize()
```
- getCurrentLogSizeFromStream
```
public long getCurrentLogSizeFromStream()
```
  Returns:
  
  the current transaction log's size (based on its output stream)
- getTotalLogsNumber
```
public long getTotalLogsNumber()
```
- getVersionInfo
```
public VersionInfo getVersionInfo()
```
- getNumRecordsToKeep
```
public int getNumRecordsToKeep()
```
- getMaxNumLogsToKeep
```
public int getMaxNumLogsToKeep()
```
- getNumVersionBuckets
```
public int getNumVersionBuckets()
```
- objToInt
```
protected static int objToInt(Object obj,
                              int def)
```
- init
```
public void init(PluginInfo info)
```
  Specified by:
  
  init in interface PluginInfoInitialized
- init
```
public void init(UpdateHandler uhandler,
                 SolrCore core)
```
- initializeMetrics
```
public void initializeMetrics(SolrMetricsContext parentContext,
                              String scope)
```
  Description copied from interface: SolrMetricProducer
  
  Initialize metrics specific to this producer.
  
  Specified by:
  
  initializeMetrics in interface SolrMetricProducer
  
  Parameters:
  
  parentContext - parent metrics context. If this component has the same life-cycle as the parent it can simply use the parent context, otherwise it should obtain a child context using SolrMetricsContext.getChildContext(Object) passing this as the child object.
  
  scope - component scope
- getSolrMetricsContext
```
public SolrMetricsContext getSolrMetricsContext()
```
  Description copied from interface: SolrMetricProducer
  
  Implementations should return the context used in SolrMetricProducer.initializeMetrics(SolrMetricsContext, String) to ensure proper cleanup of metrics at the end of the life-cycle of this component. This should be the child context if one was created, or null if the parent context was used.
  
  Specified by:
  
  getSolrMetricsContext in interface SolrMetricProducer
- newTransactionLog
```
public TransactionLog newTransactionLog(Path tlogFile,
                                        Collection<String> globalStrings,
                                        boolean openExisting)
```
  Returns a new TransactionLog. Sub-classes can override this method to change the implementation of the transaction log.
- getLogDir
```
public String getLogDir()
```
- getStartingVersions
```
public List<Long> getStartingVersions()
```
- existOldBufferLog
```
public boolean existOldBufferLog()
```
- addOldLog
```
protected void addOldLog(TransactionLog oldLog,
                         boolean removeOld)
```
- getLogList
```
public String[] getLogList(File directory)
```
- getLastLogId
```
public long getLastLogId()
```
- add
```
public void add(AddUpdateCommand cmd)
```
- add
```
public void add(AddUpdateCommand cmd,
                boolean clearCaches)
```
- delete
```
public void delete(DeleteUpdateCommand cmd)
```
- deleteByQuery
```
public void deleteByQuery(DeleteUpdateCommand cmd)
```
- openRealtimeSearcher
```
public void openRealtimeSearcher()
```
  Opens a new realtime searcher and clears the id caches. This may also be called when we updates are being buffered (from PeerSync/IndexFingerprint)
- deleteAll
```
public void deleteAll()
```
  currently for testing only
- trackDeleteByQuery
```
protected void trackDeleteByQuery(String q,
                                  long version)
```
- getDBQNewer
```
public List<UpdateLog.DBQ> getDBQNewer(long version)
```
- newMap
```
protected void newMap()
```
- hasUncommittedChanges
```
public boolean hasUncommittedChanges()
```
- preCommit
```
public void preCommit(CommitUpdateCommand cmd)
```
- postCommit
```
public void postCommit(CommitUpdateCommand cmd)
```
- preSoftCommit
```
public void preSoftCommit(CommitUpdateCommand cmd)
```
- postSoftCommit
```
public void postSoftCommit(CommitUpdateCommand cmd)
```
- applyPartialUpdates
```
public long applyPartialUpdates(org.apache.lucene.util.BytesRef id,
                                long prevPointer,
                                long prevVersion,
                                Set<String> onlyTheseFields,
                                org.apache.solr.common.SolrDocumentBase<?,?> latestPartialDoc)
```
  Goes over backwards, following the prevPointer, to merge all partial updates into the passed doc. Stops at either a full document, or if there are no previous entries to follow in the update log.
  
  Parameters:
  
  id - Binary representation of the unique key field
  
  prevPointer - Pointer to the previous entry in the ulog, based on which the current in-place update was made.
  
  prevVersion - Version of the previous entry in the ulog, based on which the current in-place update was made.
  
  onlyTheseFields - When a non-null set of field names is passed in, the resolve process only attempts to populate the given fields in this set. When this set is null, it resolves all fields.
  
  latestPartialDoc - Partial document that is to be populated
  
  Returns:
  
  Returns 0 if a full document was found in the log, -1 if no full document was found. If full document was supposed to be found in the tlogs, but couldn't be found (because the logs were rotated) then the prevPointer is returned.
- lookup
```
public Object lookup(org.apache.lucene.util.BytesRef indexedId)
```
- lookupVersion
```
public Long lookupVersion(org.apache.lucene.util.BytesRef indexedId)
```
- finish
```
public void finish(UpdateLog.SyncLevel syncLevel)
```
- recoverFromLog
```
public Future<UpdateLog.RecoveryInfo> recoverFromLog()
```
- recoverFromCurrentLog
```
public Future<UpdateLog.RecoveryInfo> recoverFromCurrentLog()
```
  Replay current tlog, so all updates will be written to index. This is must do task for a tlog replica become a new leader.
  
  Returns:
  
  future of this task
- copyOverBufferingUpdates
```
public void copyOverBufferingUpdates(CommitUpdateCommand cuc)
```
  Block updates, append a commit at current tlog, then copy over buffer updates to new tlog and bring back ulog to active state. So any updates which hasn't made it to the index is preserved in the current tlog, this also make RTG work
  
  Parameters:
  
  cuc - any updates that have version larger than the version of cuc will be copied over
- commitAndSwitchToNewTlog
```
public void commitAndSwitchToNewTlog(CommitUpdateCommand cuc)
```
  Block updates, append a commit at current tlog, then copy over updates to a new tlog. So any updates which hasn't made it to the index is preserved in the current tlog
  
  Parameters:
  
  cuc - any updates that have version larger than the version of cuc will be copied over
- copyOverOldUpdates
```
public void copyOverOldUpdates(long commitVersion)
```
- copyOverOldUpdates
```
public void copyOverOldUpdates(long commitVersion,
                               TransactionLog oldTlog)
```
  Copy over updates from prevTlog or last tlog (in tlog folder) to a new tlog
  
  Parameters:
  
  commitVersion - any updates that have version larger than the commitVersion will be copied over
- ensureBufferTlog
```
protected void ensureBufferTlog()
```
- deleteBufferLogs
```
protected void deleteBufferLogs()
```
- ensureLog
```
protected void ensureLog()
```
- close
```
public void close(boolean committed)
```
- close
```
public void close(boolean committed,
                  boolean deleteOnClose)
```
- getRecentUpdates
```
public UpdateLog.RecentUpdates getRecentUpdates()
```
  The RecentUpdates object returned must be closed after use
- bufferUpdates
```
public void bufferUpdates()
```
- dropBufferedUpdates
```
public boolean dropBufferedUpdates()
```
  Returns true if we were able to drop buffered updates and return to the ACTIVE state
- applyBufferedUpdates
```
public Future<UpdateLog.RecoveryInfo> applyBufferedUpdates()
```
  Returns the Future to wait on, or null if no replay was needed
- getState
```
public UpdateLog.State getState()
```
- toString
```
public String toString()
```
  Overrides:
  
  toString in class Object
- convertTlogEntryToAddUpdateCommand
```
public static AddUpdateCommand convertTlogEntryToAddUpdateCommand(SolrQueryRequest req,
                                                                  List<?> entry,
                                                                  int operation,
                                                                  long version)
```
  Given a entry from the transaction log containing a document, return a new AddUpdateCommand that can be applied to ADD the document or do an UPDATE_INPLACE.
  
  Parameters:
  
  req - The request to use as the owner of the new AddUpdateCommand
  
  entry - Entry from the transaction log that contains the document to be added
  
  operation - The value of the operation flag; this must be either ADD or UPDATE_INPLACE -- if it is UPDATE_INPLACE then the previous version will also be read from the entry
  
  version - Version already obtained from the entry.
- deleteFile
```
public static void deleteFile(Path file)
```
- getTlogDir
```
protected String getTlogDir(SolrCore core,
                            PluginInfo info)
```
- clearLog
```
public void clearLog(SolrCore core,
                     PluginInfo ulogPluginInfo)
```
  Clears the logs on the file system. Only call before init.
  
  Parameters:
  
  core - the SolrCore
  
  ulogPluginInfo - the init info for the UpdateHandler

Class UpdateLog

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Methods inherited from interface org.apache.solr.metrics.SolrMetricProducer

Field Detail

LOG_FILENAME_PATTERN

TLOG_NAME

BUFFER_TLOG_NAME

ADD

DELETE

DELETE_BY_QUERY

COMMIT

UPDATE_INPLACE

OPERATION_MASK

FLAGS_IDX

VERSION_IDX

PREV_POINTER_IDX

PREV_VERSION_IDX

id

state

bufferTlog

tlog

prevTlog

prevTlogOnPrecommit

logs

newestLogsOnStartup

numOldRecords

map

prevMap

prevMap2

prevMapLog

prevMapLog2

numDeletesToKeep

numDeletesByQueryToKeep

numRecordsToKeep

maxNumLogsToKeep

numVersionBuckets

existOldBufferLog

oldDeletes

deleteByQueries

tlogFiles

tlogDir

globalStrings

dataDir

lastDataDir

versionInfo

defaultSyncLevel

uhandler

cancelApplyBufferUpdate

startingVersions

bufferedOpsGauge

applyingBufferedOpsMeter

replayOpsMeter

copyOverOldUpdatesMeter

solrMetricsContext

testing_logReplayHook

testing_logReplayFinishHook

recoveryInfo

Constructor Detail

UpdateLog

Method Detail

getTotalLogsSize

getCurrentLogSizeFromStream

getTotalLogsNumber

getVersionInfo

getNumRecordsToKeep

getMaxNumLogsToKeep

getNumVersionBuckets

objToInt

init

init

initializeMetrics

getSolrMetricsContext

newTransactionLog

getLogDir

getStartingVersions

existOldBufferLog