Package org.apache.solr.core
Class HdfsDirectoryFactory
- java.lang.Object
-
- org.apache.solr.core.DirectoryFactory
-
- org.apache.solr.core.CachingDirectoryFactory
-
- org.apache.solr.core.HdfsDirectoryFactory
-
- All Implemented Interfaces:
Closeable
,AutoCloseable
,SolrMetricProducer
,NamedListInitializedPlugin
,SolrCoreAware
public class HdfsDirectoryFactory extends CachingDirectoryFactory implements SolrCoreAware, SolrMetricProducer
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class org.apache.solr.core.CachingDirectoryFactory
CachingDirectoryFactory.CacheValue, CachingDirectoryFactory.CloseListener
-
Nested classes/interfaces inherited from class org.apache.solr.core.DirectoryFactory
DirectoryFactory.DirContext
-
-
Field Summary
Fields Modifier and Type Field Description static String
BLOCKCACHE_DIRECT_MEMORY_ALLOCATION
static String
BLOCKCACHE_ENABLED
static String
BLOCKCACHE_GLOBAL
static String
BLOCKCACHE_READ_ENABLED
static String
BLOCKCACHE_SLAB_COUNT
static String
BLOCKCACHE_WRITE_ENABLED
static String
CACHE_MERGES
static String
CACHE_READONCE
static String
CONFIG_DIRECTORY
static String
HDFS_HOME
static String
KERBEROS_ENABLED
static String
KERBEROS_KEYTAB
static String
KERBEROS_PRINCIPAL
static String
LOCALITYMETRICS_ENABLED
static Metrics
metrics
static String
NRTCACHINGDIRECTORY_ENABLE
static String
NRTCACHINGDIRECTORY_MAXCACHEMB
static String
NRTCACHINGDIRECTORY_MAXMERGESIZEMB
static String
NUMBEROFBLOCKSPERBANK
-
Fields inherited from class org.apache.solr.core.CachingDirectoryFactory
byDirectoryCache, byPathCache, closeListeners, removeEntries
-
Fields inherited from class org.apache.solr.core.DirectoryFactory
coreContainer, dataHomePath, INDEX_W_TIMESTAMP_REGEX, IOCONTEXT_NO_CACHE, LOCK_TYPE_HDFS, LOCK_TYPE_NATIVE, LOCK_TYPE_NONE, LOCK_TYPE_SIMPLE, LOCK_TYPE_SINGLE
-
-
Constructor Summary
Constructors Constructor Description HdfsDirectoryFactory()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
cleanupOldIndexDirectories(String dataDir, String currentIndexDir, boolean afterReload)
void
close()
Close the this and all of the Directories it contains.protected org.apache.lucene.store.Directory
create(String path, org.apache.lucene.store.LockFactory lockFactory, DirectoryFactory.DirContext dirContext)
Creates a new Directory for a given path.protected org.apache.lucene.store.LockFactory
createLockFactory(String rawLockType)
Creates a new LockFactory for a given path.boolean
exists(String path)
Returns true if a Directory exists for a given path in the underlying (stable) storage and contains at least one file.org.apache.hadoop.conf.Configuration
getConf()
String
getConfDir()
String
getDataHome(CoreDescriptor cd)
Get the data home folder.void
inform(SolrCore core)
void
init(NamedList args)
void
initializeMetrics(SolrMetricManager manager, String registry, String tag, String scope)
Initializes metrics specific to this producer.boolean
isAbsolute(String path)
boolean
isPersistent()
boolean
isSharedStorage()
void
move(org.apache.lucene.store.Directory fromDir, org.apache.lucene.store.Directory toDir, String fileName, org.apache.lucene.store.IOContext ioContext)
Override for more efficient moves.protected void
removeDirectory(CachingDirectoryFactory.CacheValue cacheValue)
void
renameWithOverwrite(org.apache.lucene.store.Directory dir, String fileName, String toName)
boolean
searchersReserveCommitPoints()
If your implementation can count on delete-on-last-close semantics or throws an exception when trying to remove a file in use, return false (eg NFS).long
size(String path)
long
size(org.apache.lucene.store.Directory directory)
-
Methods inherited from class org.apache.solr.core.CachingDirectoryFactory
addCloseListener, deleteOldIndexDirectory, doneWithDirectory, get, getLivePaths, getPath, incRef, normalize, release, remove, remove, remove, remove, stripTrailingSlash
-
Methods inherited from class org.apache.solr.core.DirectoryFactory
empty, getBaseDir, initCoreContainer, sizeOf, sizeOfDirectory
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.apache.solr.metrics.SolrMetricProducer
initializeMetrics
-
-
-
-
Field Detail
-
BLOCKCACHE_SLAB_COUNT
public static final String BLOCKCACHE_SLAB_COUNT
- See Also:
- Constant Field Values
-
BLOCKCACHE_DIRECT_MEMORY_ALLOCATION
public static final String BLOCKCACHE_DIRECT_MEMORY_ALLOCATION
- See Also:
- Constant Field Values
-
BLOCKCACHE_ENABLED
public static final String BLOCKCACHE_ENABLED
- See Also:
- Constant Field Values
-
BLOCKCACHE_GLOBAL
public static final String BLOCKCACHE_GLOBAL
- See Also:
- Constant Field Values
-
BLOCKCACHE_READ_ENABLED
public static final String BLOCKCACHE_READ_ENABLED
- See Also:
- Constant Field Values
-
BLOCKCACHE_WRITE_ENABLED
public static final String BLOCKCACHE_WRITE_ENABLED
- See Also:
- Constant Field Values
-
NRTCACHINGDIRECTORY_ENABLE
public static final String NRTCACHINGDIRECTORY_ENABLE
- See Also:
- Constant Field Values
-
NRTCACHINGDIRECTORY_MAXMERGESIZEMB
public static final String NRTCACHINGDIRECTORY_MAXMERGESIZEMB
- See Also:
- Constant Field Values
-
NRTCACHINGDIRECTORY_MAXCACHEMB
public static final String NRTCACHINGDIRECTORY_MAXCACHEMB
- See Also:
- Constant Field Values
-
NUMBEROFBLOCKSPERBANK
public static final String NUMBEROFBLOCKSPERBANK
- See Also:
- Constant Field Values
-
LOCALITYMETRICS_ENABLED
public static final String LOCALITYMETRICS_ENABLED
- See Also:
- Constant Field Values
-
KERBEROS_ENABLED
public static final String KERBEROS_ENABLED
- See Also:
- Constant Field Values
-
KERBEROS_KEYTAB
public static final String KERBEROS_KEYTAB
- See Also:
- Constant Field Values
-
KERBEROS_PRINCIPAL
public static final String KERBEROS_PRINCIPAL
- See Also:
- Constant Field Values
-
HDFS_HOME
public static final String HDFS_HOME
- See Also:
- Constant Field Values
-
CONFIG_DIRECTORY
public static final String CONFIG_DIRECTORY
- See Also:
- Constant Field Values
-
CACHE_MERGES
public static final String CACHE_MERGES
- See Also:
- Constant Field Values
-
CACHE_READONCE
public static final String CACHE_READONCE
- See Also:
- Constant Field Values
-
metrics
public static Metrics metrics
-
-
Method Detail
-
close
public void close() throws IOException
Description copied from class:DirectoryFactory
Close the this and all of the Directories it contains.- Specified by:
close
in interfaceAutoCloseable
- Specified by:
close
in interfaceCloseable
- Overrides:
close
in classCachingDirectoryFactory
- Throws:
IOException
- If there is a low-level I/O error.
-
init
public void init(NamedList args)
- Specified by:
init
in interfaceNamedListInitializedPlugin
- Overrides:
init
in classCachingDirectoryFactory
-
createLockFactory
protected org.apache.lucene.store.LockFactory createLockFactory(String rawLockType) throws IOException
Description copied from class:DirectoryFactory
Creates a new LockFactory for a given path.- Specified by:
createLockFactory
in classDirectoryFactory
- Parameters:
rawLockType
- A string value as passed in config. Every factory should at least support 'none' to disable locking.- Throws:
IOException
- If there is a low-level I/O error.
-
create
protected org.apache.lucene.store.Directory create(String path, org.apache.lucene.store.LockFactory lockFactory, DirectoryFactory.DirContext dirContext) throws IOException
Description copied from class:DirectoryFactory
Creates a new Directory for a given path.- Specified by:
create
in classDirectoryFactory
- Throws:
IOException
- If there is a low-level I/O error.
-
exists
public boolean exists(String path)
Description copied from class:DirectoryFactory
Returns true if a Directory exists for a given path in the underlying (stable) storage and contains at least one file. Note that the existence of aDirectory
Object as returned by a previous call to theDirectoryFactory.get(java.lang.String, org.apache.solr.core.DirectoryFactory.DirContext, java.lang.String)
method (on the specifiedpath
) is not enough to cause this method to return true. Some prior user of that Directory must have written & synced at least one file to that Directory (and at least one file must still exist)- Overrides:
exists
in classCachingDirectoryFactory
-
getConf
public org.apache.hadoop.conf.Configuration getConf()
-
removeDirectory
protected void removeDirectory(CachingDirectoryFactory.CacheValue cacheValue) throws IOException
- Overrides:
removeDirectory
in classCachingDirectoryFactory
- Throws:
IOException
-
isAbsolute
public boolean isAbsolute(String path)
- Overrides:
isAbsolute
in classDirectoryFactory
- Parameters:
path
- the path to check- Returns:
- true if absolute, as in not relative
-
isPersistent
public boolean isPersistent()
- Specified by:
isPersistent
in classDirectoryFactory
- Returns:
- true if data is kept after close.
-
isSharedStorage
public boolean isSharedStorage()
- Overrides:
isSharedStorage
in classDirectoryFactory
- Returns:
- true if storage is shared.
-
searchersReserveCommitPoints
public boolean searchersReserveCommitPoints()
Description copied from class:DirectoryFactory
If your implementation can count on delete-on-last-close semantics or throws an exception when trying to remove a file in use, return false (eg NFS). Otherwise, return true. Defaults to returning false.- Overrides:
searchersReserveCommitPoints
in classDirectoryFactory
- Returns:
- true if factory impl requires that Searcher's explicitly reserve commit points.
-
getDataHome
public String getDataHome(CoreDescriptor cd) throws IOException
Description copied from class:DirectoryFactory
Get the data home folder. If solr.data.home is set, that is used, else base on instanceDir- Overrides:
getDataHome
in classDirectoryFactory
- Parameters:
cd
- core descriptor instance- Returns:
- a String with absolute path to data direcotry
- Throws:
IOException
-
size
public long size(org.apache.lucene.store.Directory directory) throws IOException
- Overrides:
size
in classDirectoryFactory
- Parameters:
directory
- to calculate size of- Returns:
- size in bytes
- Throws:
IOException
- on low level IO error
-
size
public long size(String path) throws IOException
- Overrides:
size
in classDirectoryFactory
- Parameters:
path
- to calculate size of- Returns:
- size in bytes
- Throws:
IOException
- on low level IO error
-
getConfDir
public String getConfDir()
-
initializeMetrics
public void initializeMetrics(SolrMetricManager manager, String registry, String tag, String scope)
Description copied from interface:SolrMetricProducer
Initializes metrics specific to this producer.Note: for back-compatibility this method by default calls
SolrMetricProducer.initializeMetrics(SolrMetricManager, String, String)
.- Specified by:
initializeMetrics
in interfaceSolrMetricProducer
- Parameters:
manager
- an instance ofSolrMetricManager
registry
- registry name where metrics are registeredtag
- symbolic tag that represents a group of related instances that have the same life-cycle. Parent component can use thetag
when callingSolrMetricManager.unregisterGauges(String, String)
to unregister metrics created by this instance of the producer.scope
- scope of the metrics (eg. handler name) to separate metrics of instances of the same component executing in different contexts
-
inform
public void inform(SolrCore core)
- Specified by:
inform
in interfaceSolrCoreAware
-
cleanupOldIndexDirectories
public void cleanupOldIndexDirectories(String dataDir, String currentIndexDir, boolean afterReload)
- Overrides:
cleanupOldIndexDirectories
in classDirectoryFactory
-
renameWithOverwrite
public void renameWithOverwrite(org.apache.lucene.store.Directory dir, String fileName, String toName) throws IOException
- Overrides:
renameWithOverwrite
in classDirectoryFactory
- Throws:
IOException
-
move
public void move(org.apache.lucene.store.Directory fromDir, org.apache.lucene.store.Directory toDir, String fileName, org.apache.lucene.store.IOContext ioContext) throws IOException
Description copied from class:DirectoryFactory
Override for more efficient moves. Intended for use with replication - use carefully - some Directory wrappers will cache files for example.- Overrides:
move
in classDirectoryFactory
- Throws:
IOException
- If there is a low-level I/O error.
-
-