Package org.apache.solr.update
Class CdcrUpdateLog
- java.lang.Object
-
- org.apache.solr.update.UpdateLog
-
- org.apache.solr.update.CdcrUpdateLog
-
- All Implemented Interfaces:
AutoCloseable
,SolrMetricProducer
,PluginInfoInitialized
public class CdcrUpdateLog extends UpdateLog
An extension of theUpdateLog
for the CDCR scenario.
Compared to the original update log implementation, transaction logs are removed based on pointers instead of a fixed size limit. Pointers are created by the CDC replicators and correspond to replication checkpoints. If all pointers are ahead of a transaction log, this transaction log is removed.
Given that the number of transaction logs can become considerable if some pointers are lagging behind, theCdcrUpdateLog.CdcrLogReader
provides aCdcrUpdateLog.CdcrLogReader.seek(long)
method to efficiently lookup a particular transaction log file given a version number.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
CdcrUpdateLog.BufferedUpdates
class
CdcrUpdateLog.CdcrLogReader
-
Nested classes/interfaces inherited from class org.apache.solr.update.UpdateLog
UpdateLog.DBQ, UpdateLog.LogPtr, UpdateLog.RecentUpdates, UpdateLog.RecoveryInfo, UpdateLog.State, UpdateLog.SyncLevel
-
-
Field Summary
Fields Modifier and Type Field Description static String
LOG_FILENAME_PATTERN
protected Map<CdcrUpdateLog.CdcrLogReader,org.apache.solr.update.CdcrUpdateLog.CdcrLogPointer>
logPointers
-
Fields inherited from class org.apache.solr.update.UpdateLog
ADD, applyingBufferedOpsMeter, BUFFER_TLOG_NAME, bufferedOpsGauge, bufferTlog, cancelApplyBufferUpdate, COMMIT, copyOverOldUpdatesMeter, dataDir, defaultSyncLevel, DELETE, DELETE_BY_QUERY, deleteByQueries, existOldBufferLog, FLAGS_IDX, globalStrings, lastDataDir, logs, map, maxNumLogsToKeep, maxVersionFromIndex, metricManager, newestLogsOnStartup, numDeletesByQueryToKeep, numDeletesToKeep, numOldRecords, numRecordsToKeep, numVersionBuckets, oldDeletes, OPERATION_MASK, PREV_POINTER_IDX, PREV_VERSION_IDX, prevMap, prevMap2, prevMapLog, prevMapLog2, prevTlog, prevTlogOnPrecommit, recoveryInfo, registryName, replayOpsMeter, state, testing_logReplayFinishHook, testing_logReplayHook, tlog, TLOG_NAME, tlogDir, tlogFiles, UPDATE_INPLACE, VERSION_IDX, versionInfo
-
-
Constructor Summary
Constructors Constructor Description CdcrUpdateLog()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
add(AddUpdateCommand cmd, boolean clearCaches)
protected void
addOldLog(TransactionLog oldLog, boolean removeOld)
void
close(boolean committed, boolean deleteOnClose)
void
delete(DeleteUpdateCommand cmd)
void
deleteByQuery(DeleteUpdateCommand cmd)
void
disableBuffer()
Disable the buffering of the tlogs.void
enableBuffer()
Enable the buffering of the tlogs.protected void
ensureLog(long startVersion)
CdcrUpdateLog.CdcrLogReader
getBufferToggle()
long
getLastLogId()
void
init(UpdateHandler uhandler, SolrCore core)
void
initForRecovery(File bufferedTlog, long offset)
expert: Initialise the update log with a tlog file containing buffered updates.boolean
isBuffering()
Is the update log buffering the tlogs ?CdcrUpdateLog.CdcrLogReader
newLogReader()
Creates a newCdcrUpdateLog.CdcrLogReader
initialised with the current list of tlogs.TransactionLog
newTransactionLog(File tlogFile, Collection<String> globalStrings, boolean openExisting)
Returns a newTransactionLog
.CdcrUpdateLog.BufferedUpdates
resetForRecovery()
expert: Reset the update log before initialisation.-
Methods inherited from class org.apache.solr.update.UpdateLog
add, applyBufferedUpdates, applyPartialUpdates, bufferUpdates, clearLog, close, commitAndSwitchToNewTlog, convertTlogEntryToAddUpdateCommand, copyOverBufferingUpdates, copyOverOldUpdates, copyOverOldUpdates, deleteAll, deleteBufferLogs, deleteFile, dropBufferedUpdates, ensureBufferTlog, ensureLog, existOldBufferLog, finish, getBufferLogList, getCurrentLogSizeFromStream, getCurrentMaxVersion, getDBQNewer, getFs, getLogDir, getLogList, getMaxNumLogsToKeep, getNumRecordsToKeep, getNumVersionBuckets, getRecentUpdates, getStartingVersions, getState, getTlogDir, getTotalLogsNumber, getTotalLogsSize, getVersionInfo, hasUncommittedChanges, init, initializeMetrics, lookup, lookupVersion, newMap, objToInt, openRealtimeSearcher, postCommit, postSoftCommit, preCommit, preSoftCommit, recoverFromCurrentLog, recoverFromLog, seedBucketsWithHighestVersion, seedBucketsWithHighestVersion, toString
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
-
Methods inherited from interface org.apache.solr.metrics.SolrMetricProducer
close, getSolrMetricsContext, initializeMetrics
-
-
-
-
Field Detail
-
logPointers
protected final Map<CdcrUpdateLog.CdcrLogReader,org.apache.solr.update.CdcrUpdateLog.CdcrLogPointer> logPointers
-
LOG_FILENAME_PATTERN
public static String LOG_FILENAME_PATTERN
-
-
Method Detail
-
init
public void init(UpdateHandler uhandler, SolrCore core)
-
newTransactionLog
public TransactionLog newTransactionLog(File tlogFile, Collection<String> globalStrings, boolean openExisting)
Description copied from class:UpdateLog
Returns a newTransactionLog
. Sub-classes can override this method to change the implementation of the transaction log.- Overrides:
newTransactionLog
in classUpdateLog
-
addOldLog
protected void addOldLog(TransactionLog oldLog, boolean removeOld)
-
getLastLogId
public long getLastLogId()
- Overrides:
getLastLogId
in classUpdateLog
-
add
public void add(AddUpdateCommand cmd, boolean clearCaches)
-
delete
public void delete(DeleteUpdateCommand cmd)
-
deleteByQuery
public void deleteByQuery(DeleteUpdateCommand cmd)
- Overrides:
deleteByQuery
in classUpdateLog
-
newLogReader
public CdcrUpdateLog.CdcrLogReader newLogReader()
Creates a newCdcrUpdateLog.CdcrLogReader
initialised with the current list of tlogs.
-
enableBuffer
public void enableBuffer()
Enable the buffering of the tlogs. When buffering is activated, the update logs will not remove any old transaction log files.
-
disableBuffer
public void disableBuffer()
Disable the buffering of the tlogs.
-
getBufferToggle
public CdcrUpdateLog.CdcrLogReader getBufferToggle()
-
isBuffering
public boolean isBuffering()
Is the update log buffering the tlogs ?
-
ensureLog
protected void ensureLog(long startVersion)
-
resetForRecovery
public CdcrUpdateLog.BufferedUpdates resetForRecovery()
expert: Reset the update log before initialisation. This is called byIndexFetcher.moveTlogFiles(File)
during a a Recovery operation in order to re-initialise the UpdateLog with a new set of tlog files.- See Also:
initForRecovery(File, long)
-
initForRecovery
public void initForRecovery(File bufferedTlog, long offset)
expert: Initialise the update log with a tlog file containing buffered updates. This is called by
IndexFetcher.moveTlogFiles(File)
during a Recovery operation. This is mainly a copy of the originalUpdateLog.init(UpdateHandler, SolrCore)
method, but modified to:- preserve the same
VersionInfo
instance in order to not "unblock" updates, since theIndexFetcher.moveTlogFiles(File)
acquired a write lock from this instance. - copy the buffered updates.
- See Also:
resetForRecovery()
- preserve the same
-
-