Package org.apache.solr.core

Class HdfsDirectoryFactory

java.lang.Object

org.apache.solr.core.DirectoryFactory

org.apache.solr.core.CachingDirectoryFactory

org.apache.solr.core.HdfsDirectoryFactory

All Implemented Interfaces:

Closeable, AutoCloseable, SolrMetricProducer, NamedListInitializedPlugin, SolrCoreAware

public class HdfsDirectoryFactory
extends CachingDirectoryFactory
implements SolrCoreAware, SolrMetricProducer

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.solr.core.CachingDirectoryFactory

CachingDirectoryFactory.CacheValue, CachingDirectoryFactory.CloseListener

Nested classes/interfaces inherited from class org.apache.solr.core.DirectoryFactory

DirectoryFactory.DirContext

Field Summary

Fields

Modifier and Type	Field	Description
static String	BLOCKCACHE_DIRECT_MEMORY_ALLOCATION
static String	BLOCKCACHE_ENABLED
static String	BLOCKCACHE_GLOBAL
static String	BLOCKCACHE_READ_ENABLED
static String	BLOCKCACHE_SLAB_COUNT
static String	BLOCKCACHE_WRITE_ENABLED
static String	CACHE_MERGES
static String	CACHE_READONCE
static String	CONFIG_DIRECTORY
static String	HDFS_HOME
static String	KERBEROS_ENABLED
static String	KERBEROS_KEYTAB
static String	KERBEROS_PRINCIPAL
static String	LOCALITYMETRICS_ENABLED
static Metrics	metrics
static String	NRTCACHINGDIRECTORY_ENABLE
static String	NRTCACHINGDIRECTORY_MAXCACHEMB
static String	NRTCACHINGDIRECTORY_MAXMERGESIZEMB
static String	NUMBEROFBLOCKSPERBANK

Fields inherited from class org.apache.solr.core.CachingDirectoryFactory

byDirectoryCache, byPathCache, closeListeners, removeEntries

Fields inherited from class org.apache.solr.core.DirectoryFactory

coreContainer, dataHomePath, INDEX_W_TIMESTAMP_REGEX, IOCONTEXT_NO_CACHE, LOCK_TYPE_HDFS, LOCK_TYPE_NATIVE, LOCK_TYPE_NONE, LOCK_TYPE_SIMPLE, LOCK_TYPE_SINGLE

Constructor Summary

Constructors

Constructor	Description
HdfsDirectoryFactory()

Method Summary

Modifier and Type	Method	Description
void	cleanupOldIndexDirectories(String dataDir, String currentIndexDir, boolean afterReload)
void	close()	Close the this and all of the Directories it contains.
protected org.apache.lucene.store.Directory	create(String path, org.apache.lucene.store.LockFactory lockFactory, DirectoryFactory.DirContext dirContext)	Creates a new Directory for a given path.
protected org.apache.lucene.store.LockFactory	createLockFactory(String rawLockType)	Creates a new LockFactory for a given path.
boolean	exists(String path)	Returns true if a Directory exists for a given path in the underlying (stable) storage and contains at least one file.
org.apache.hadoop.conf.Configuration	getConf(org.apache.hadoop.fs.Path path)
String	getConfDir()
String	getDataHome(CoreDescriptor cd)	Get the data home folder.
void	inform(SolrCore core)
void	init(NamedList args)
void	initializeMetrics(SolrMetricsContext parentContext, String scope)	Initialize metrics specific to this producer.
boolean	isAbsolute(String path)
boolean	isPersistent()
boolean	isSharedStorage()
void	move(org.apache.lucene.store.Directory fromDir, org.apache.lucene.store.Directory toDir, String fileName, org.apache.lucene.store.IOContext ioContext)	Override for more efficient moves.
protected void	removeDirectory(CachingDirectoryFactory.CacheValue cacheValue)
void	renameWithOverwrite(org.apache.lucene.store.Directory dir, String fileName, String toName)
boolean	searchersReserveCommitPoints()	If your implementation can count on delete-on-last-close semantics or throws an exception when trying to remove a file in use, return false (eg NFS).
long	size(String path)
long	size(org.apache.lucene.store.Directory directory)

Methods inherited from class org.apache.solr.core.CachingDirectoryFactory

addCloseListener, deleteOldIndexDirectory, doneWithDirectory, get, getLivePaths, getPath, incRef, normalize, release, remove, remove, remove, remove, stripTrailingSlash

Methods inherited from class org.apache.solr.core.DirectoryFactory

empty, getBaseDir, initCoreContainer, sizeOf, sizeOfDirectory

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.solr.metrics.SolrMetricProducer

getSolrMetricsContext, initializeMetrics

Field Detail

BLOCKCACHE_SLAB_COUNT

public static final String BLOCKCACHE_SLAB_COUNT

See Also:

Constant Field Values

BLOCKCACHE_DIRECT_MEMORY_ALLOCATION

public static final String BLOCKCACHE_DIRECT_MEMORY_ALLOCATION

See Also:

Constant Field Values

BLOCKCACHE_ENABLED

public static final String BLOCKCACHE_ENABLED

See Also:

Constant Field Values

BLOCKCACHE_GLOBAL

public static final String BLOCKCACHE_GLOBAL

See Also:

Constant Field Values

BLOCKCACHE_READ_ENABLED

public static final String BLOCKCACHE_READ_ENABLED

See Also:

Constant Field Values

BLOCKCACHE_WRITE_ENABLED

public static final String BLOCKCACHE_WRITE_ENABLED

See Also:

Constant Field Values

NRTCACHINGDIRECTORY_ENABLE

public static final String NRTCACHINGDIRECTORY_ENABLE

See Also:

Constant Field Values

NRTCACHINGDIRECTORY_MAXMERGESIZEMB

public static final String NRTCACHINGDIRECTORY_MAXMERGESIZEMB

See Also:

Constant Field Values

NRTCACHINGDIRECTORY_MAXCACHEMB

public static final String NRTCACHINGDIRECTORY_MAXCACHEMB

See Also:

Constant Field Values

NUMBEROFBLOCKSPERBANK

public static final String NUMBEROFBLOCKSPERBANK

See Also:

Constant Field Values

LOCALITYMETRICS_ENABLED

public static final String LOCALITYMETRICS_ENABLED

See Also:

Constant Field Values

KERBEROS_ENABLED

public static final String KERBEROS_ENABLED

See Also:

Constant Field Values

KERBEROS_KEYTAB

public static final String KERBEROS_KEYTAB

See Also:

Constant Field Values

KERBEROS_PRINCIPAL

public static final String KERBEROS_PRINCIPAL

See Also:

Constant Field Values

HDFS_HOME

public static final String HDFS_HOME

See Also:

Constant Field Values

CONFIG_DIRECTORY

public static final String CONFIG_DIRECTORY

See Also:

Constant Field Values

CACHE_MERGES

public static final String CACHE_MERGES

See Also:

Constant Field Values

CACHE_READONCE

public static final String CACHE_READONCE

See Also:

Constant Field Values

metrics

public static Metrics metrics

Constructor Detail

HdfsDirectoryFactory

public HdfsDirectoryFactory()

Method Detail

close

public void close()
           throws IOException

Description copied from class: DirectoryFactory

Close the this and all of the Directories it contains.

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Specified by:

close in interface SolrMetricProducer

Overrides:

close in class CachingDirectoryFactory

Throws:

IOException - If there is a low-level I/O error.

init

public void init(NamedList args)

Specified by:

init in interface NamedListInitializedPlugin

Overrides:

init in class CachingDirectoryFactory

createLockFactory

protected org.apache.lucene.store.LockFactory createLockFactory(String rawLockType)
                                                         throws IOException

Description copied from class: DirectoryFactory

Creates a new LockFactory for a given path.

Specified by:

createLockFactory in class DirectoryFactory

Parameters:

rawLockType - A string value as passed in config. Every factory should at least support 'none' to disable locking.

Throws:

IOException - If there is a low-level I/O error.

create

protected org.apache.lucene.store.Directory create(String path,
                                                   org.apache.lucene.store.LockFactory lockFactory,
                                                   DirectoryFactory.DirContext dirContext)
                                            throws IOException

Description copied from class: DirectoryFactory

Creates a new Directory for a given path.

Specified by:

create in class DirectoryFactory

Throws:

IOException - If there is a low-level I/O error.

exists

public boolean exists(String path)

Description copied from class: DirectoryFactory

Returns true if a Directory exists for a given path in the underlying (stable) storage and contains at least one file. Note that the existence of a Directory Object as returned by a previous call to the DirectoryFactory.get(java.lang.String, org.apache.solr.core.DirectoryFactory.DirContext, java.lang.String) method (on the specified path) is not enough to cause this method to return true. Some prior user of that Directory must have written & synced at least one file to that Directory (and at least one file must still exist)

Overrides:

exists in class CachingDirectoryFactory

getConf

public org.apache.hadoop.conf.Configuration getConf(org.apache.hadoop.fs.Path path)

removeDirectory

protected void removeDirectory(CachingDirectoryFactory.CacheValue cacheValue)

Overrides:

removeDirectory in class CachingDirectoryFactory

isAbsolute

public boolean isAbsolute(String path)

Overrides:

isAbsolute in class DirectoryFactory

Parameters:

path - the path to check

Returns:

true if absolute, as in not relative

isPersistent

public boolean isPersistent()

Specified by:

isPersistent in class DirectoryFactory

Returns:

true if data is kept after close.

isSharedStorage

public boolean isSharedStorage()

Overrides:

isSharedStorage in class DirectoryFactory

Returns:

true if storage is shared.

searchersReserveCommitPoints

public boolean searchersReserveCommitPoints()

Description copied from class: DirectoryFactory

If your implementation can count on delete-on-last-close semantics or throws an exception when trying to remove a file in use, return false (eg NFS). Otherwise, return true. Defaults to returning false.

Overrides:

searchersReserveCommitPoints in class DirectoryFactory

Returns:

true if factory impl requires that Searcher's explicitly reserve commit points.

getDataHome

public String getDataHome(CoreDescriptor cd)
                   throws IOException

Description copied from class: DirectoryFactory

Get the data home folder. If solr.data.home is set, that is used, else base on instanceDir

Overrides:

getDataHome in class DirectoryFactory

Parameters:

cd - core descriptor instance

Returns:

a String with absolute path to data direcotry

Throws:

IOException

size

public long size(org.apache.lucene.store.Directory directory)
          throws IOException

Overrides:

size in class DirectoryFactory

Parameters:

directory - to calculate size of

Returns:

size in bytes

Throws:

IOException - on low level IO error

size

public long size(String path)
          throws IOException

Overrides:

size in class DirectoryFactory

Parameters:

path - to calculate size of

Returns:

size in bytes

Throws:

IOException - on low level IO error

getConfDir

public String getConfDir()

initializeMetrics

public void initializeMetrics(SolrMetricsContext parentContext,
                              String scope)

Description copied from interface: SolrMetricProducer

Initialize metrics specific to this producer.

Specified by:

initializeMetrics in interface SolrMetricProducer

Parameters:

parentContext - parent metrics context. If this component has the same life-cycle as the parent it can simply use the parent context, otherwise it should obtain a child context using SolrMetricsContext.getChildContext(Object) passing this as the child.

scope - component scope

inform

public void inform(SolrCore core)

Specified by:

inform in interface SolrCoreAware

cleanupOldIndexDirectories

public void cleanupOldIndexDirectories(String dataDir,
                                       String currentIndexDir,
                                       boolean afterReload)

Overrides:

cleanupOldIndexDirectories in class DirectoryFactory

renameWithOverwrite

public void renameWithOverwrite(org.apache.lucene.store.Directory dir,
                                String fileName,
                                String toName)
                         throws IOException

Overrides:

renameWithOverwrite in class DirectoryFactory

Throws:

IOException

move

public void move(org.apache.lucene.store.Directory fromDir,
                 org.apache.lucene.store.Directory toDir,
                 String fileName,
                 org.apache.lucene.store.IOContext ioContext)
          throws IOException

Description copied from class: DirectoryFactory

Override for more efficient moves. Intended for use with replication - use carefully - some Directory wrappers will cache files for example.

Overrides:

move in class DirectoryFactory

Throws:

IOException - If there is a low-level I/O error.