org.apache.solr.update

Class CdcrUpdateLog

java.lang.Object

org.apache.solr.update.UpdateLog

org.apache.solr.update.CdcrUpdateLog

All Implemented Interfaces:

SolrMetricProducer, PluginInfoInitialized

public class CdcrUpdateLog
extends UpdateLog

An extension of the UpdateLog for the CDCR scenario.
Compared to the original update log implementation, transaction logs are removed based on pointers instead of a fixed size limit. Pointers are created by the CDC replicators and correspond to replication checkpoints. If all pointers are ahead of a transaction log, this transaction log is removed.
Given that the number of transaction logs can become considerable if some pointers are lagging behind, the CdcrUpdateLog.CdcrLogReader provides a CdcrUpdateLog.CdcrLogReader.seek(long) method to efficiently lookup a particular transaction log file given a version number.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	CdcrUpdateLog.BufferedUpdates
class	CdcrUpdateLog.CdcrLogReader

Nested classes/interfaces inherited from class org.apache.solr.update.UpdateLog

UpdateLog.DBQ, UpdateLog.LogPtr, UpdateLog.RecentUpdates, UpdateLog.RecoveryInfo, UpdateLog.State, UpdateLog.SyncLevel

Field Summary

Fields

Modifier and Type	Field and Description
static String	LOG_FILENAME_PATTERN
protected Map<CdcrUpdateLog.CdcrLogReader,org.apache.solr.update.CdcrUpdateLog.CdcrLogPointer>	logPointers

Fields inherited from class org.apache.solr.update.UpdateLog

ADD, applyingBufferedOpsMeter, BUFFER_TLOG_NAME, bufferedOpsGauge, bufferTlog, cancelApplyBufferUpdate, COMMIT, copyOverOldUpdatesMeter, dataDir, defaultSyncLevel, DELETE, DELETE_BY_QUERY, deleteByQueries, existOldBufferLog, FLAGS_IDX, globalStrings, lastDataDir, logs, map, maxNumLogsToKeep, maxVersionFromIndex, metricManager, newestLogsOnStartup, numDeletesByQueryToKeep, numDeletesToKeep, numOldRecords, numRecordsToKeep, numVersionBuckets, oldDeletes, OPERATION_MASK, PREV_POINTER_IDX, PREV_VERSION_IDX, prevMap, prevMap2, prevMapLog, prevMapLog2, prevTlog, prevTlogOnPrecommit, recoveryInfo, registryName, replayOpsMeter, state, testing_logReplayFinishHook, testing_logReplayHook, tlog, TLOG_NAME, tlogDir, tlogFiles, UPDATE_INPLACE, VERSION_IDX, versionInfo

Constructor Summary

Constructors

Constructor and Description
CdcrUpdateLog()

Method Summary

Modifier and Type	Method and Description
void	add(AddUpdateCommand cmd, boolean clearCaches)
protected void	addOldLog(TransactionLog oldLog, boolean removeOld)
void	close(boolean committed, boolean deleteOnClose)
void	delete(DeleteUpdateCommand cmd)
void	deleteByQuery(DeleteUpdateCommand cmd)
void	disableBuffer() Disable the buffering of the tlogs.
void	enableBuffer() Enable the buffering of the tlogs.
protected void	ensureLog(long startVersion)
CdcrUpdateLog.CdcrLogReader	getBufferToggle()
long	getLastLogId()
void	init(UpdateHandler uhandler, SolrCore core)
void	initForRecovery(File bufferedTlog, long offset) expert: Initialise the update log with a tlog file containing buffered updates.
boolean	isBuffering() Is the update log buffering the tlogs ?
CdcrUpdateLog.CdcrLogReader	newLogReader() Creates a new CdcrUpdateLog.CdcrLogReader initialised with the current list of tlogs.
TransactionLog	newTransactionLog(File tlogFile, Collection<String> globalStrings, boolean openExisting) Returns a new TransactionLog.
CdcrUpdateLog.BufferedUpdates	resetForRecovery() expert: Reset the update log before initialisation.

Methods inherited from class org.apache.solr.update.UpdateLog

add, applyBufferedUpdates, applyPartialUpdates, bufferUpdates, clearLog, close, commitAndSwitchToNewTlog, convertTlogEntryToAddUpdateCommand, copyOverBufferingUpdates, copyOverOldUpdates, copyOverOldUpdates, deleteAll, deleteBufferLogs, deleteFile, dropBufferedUpdates, ensureBufferTlog, ensureLog, existOldBufferLog, finish, getBufferLogList, getCurrentLogSizeFromStream, getCurrentMaxVersion, getDBQNewer, getFs, getLogDir, getLogList, getMaxNumLogsToKeep, getNumRecordsToKeep, getNumVersionBuckets, getRecentUpdates, getStartingVersions, getState, getTlogDir, getTotalLogsNumber, getTotalLogsSize, getVersionInfo, hasUncommittedChanges, init, initializeMetrics, lookup, lookupVersion, newMap, objToInt, openRealtimeSearcher, postCommit, postSoftCommit, preCommit, preSoftCommit, recoverFromCurrentLog, recoverFromLog, seedBucketsWithHighestVersion, seedBucketsWithHighestVersion, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

logPointers

protected final Map<CdcrUpdateLog.CdcrLogReader,org.apache.solr.update.CdcrUpdateLog.CdcrLogPointer> logPointers

LOG_FILENAME_PATTERN

public static String LOG_FILENAME_PATTERN

Constructor Detail

CdcrUpdateLog

public CdcrUpdateLog()

Method Detail

init

public void init(UpdateHandler uhandler,
                 SolrCore core)

Overrides:

init in class UpdateLog

newTransactionLog

public TransactionLog newTransactionLog(File tlogFile,
                                        Collection<String> globalStrings,
                                        boolean openExisting)

Description copied from class: UpdateLog

Returns a new TransactionLog. Sub-classes can override this method to change the implementation of the transaction log.

Overrides:

newTransactionLog in class UpdateLog

addOldLog

protected void addOldLog(TransactionLog oldLog,
                         boolean removeOld)

Overrides:

addOldLog in class UpdateLog

getLastLogId

public long getLastLogId()

Overrides:

getLastLogId in class UpdateLog

add

public void add(AddUpdateCommand cmd,
                boolean clearCaches)

Overrides:

add in class UpdateLog

delete

public void delete(DeleteUpdateCommand cmd)

Overrides:

delete in class UpdateLog

deleteByQuery

public void deleteByQuery(DeleteUpdateCommand cmd)

Overrides:

deleteByQuery in class UpdateLog

newLogReader

public CdcrUpdateLog.CdcrLogReader newLogReader()

Creates a new CdcrUpdateLog.CdcrLogReader initialised with the current list of tlogs.

enableBuffer

public void enableBuffer()

Enable the buffering of the tlogs. When buffering is activated, the update logs will not remove any old transaction log files.

disableBuffer

public void disableBuffer()

Disable the buffering of the tlogs.

getBufferToggle

public CdcrUpdateLog.CdcrLogReader getBufferToggle()

isBuffering

public boolean isBuffering()

Is the update log buffering the tlogs ?

ensureLog

protected void ensureLog(long startVersion)

resetForRecovery

public CdcrUpdateLog.BufferedUpdates resetForRecovery()

expert: Reset the update log before initialisation. This is called by IndexFetcher.moveTlogFiles(File) during a a Recovery operation in order to re-initialise the UpdateLog with a new set of tlog files.

See Also:

initForRecovery(File, long)

initForRecovery

public void initForRecovery(File bufferedTlog,
                            long offset)

expert: Initialise the update log with a tlog file containing buffered updates. This is called by IndexFetcher.moveTlogFiles(File) during a Recovery operation. This is mainly a copy of the original UpdateLog.init(UpdateHandler, SolrCore) method, but modified to:

• preserve the same VersionInfo instance in order to not "unblock" updates, since the IndexFetcher.moveTlogFiles(File) acquired a write lock from this instance.
• copy the buffered updates.

See Also:

resetForRecovery()

close

public void close(boolean committed,
                  boolean deleteOnClose)

Overrides:

close in class UpdateLog