Package org.elasticsearch.env

Class NodeEnvironment

java.lang.Object
org.elasticsearch.env.NodeEnvironment
All Implemented Interfaces:
Closeable, AutoCloseable
public final class NodeEnvironment extends Object implements Closeable
A component that holds all data paths for a single node.

  • Field Details

    • MAX_LOCAL_STORAGE_NODES_SETTING

      public static final Setting<Integer> MAX_LOCAL_STORAGE_NODES_SETTING
      Maximum number of data nodes that should run in an environment.

    • NODE_ID_SEED_SETTING

      public static final Setting<Long> NODE_ID_SEED_SETTING
      Seed for determining a persisted unique uuid of this node. If the node has already a persisted uuid on disk, this seed will be ignored and the uuid from disk will be reused.

    • ENABLE_LUCENE_SEGMENT_INFOS_TRACE_SETTING

      public static final Setting<Boolean> ENABLE_LUCENE_SEGMENT_INFOS_TRACE_SETTING
      If true the [verbose] SegmentInfos.infoStream logging is sent to System.out.

    • NODES_FOLDER

      public static final String NODES_FOLDER
      See Also:
      Constant Field Values

    • INDICES_FOLDER

      public static final String INDICES_FOLDER
      See Also:
      Constant Field Values

    • NODE_LOCK_FILENAME

      public static final String NODE_LOCK_FILENAME
      See Also:
      Constant Field Values

  • Constructor Details

  • Method Details

    • resolveNodePath

      public static Path resolveNodePath(Path path, int nodeLockId)
      Resolve a specific nodes/{node.id} path for the specified path and node lock id.
      Parameters:
      path - the path
      nodeLockId - the node lock id
      Returns:
      the resolved path

    • generateNodeId

      public static String generateNodeId(Settings settings)

    • deleteShardDirectorySafe

      public void deleteShardDirectorySafe(ShardId shardId, IndexSettings indexSettings, Consumer<Path[]> listener) throws IOException, ShardLockObtainFailedException
      Deletes a shard data directory iff the shards locks were successfully acquired.
      Parameters:
      shardId - the id of the shard to delete to delete
      Throws:
      IOException - if an IOException occurs
      ShardLockObtainFailedException

    • acquireFSLockForPaths

      public static void acquireFSLockForPaths(IndexSettings indexSettings, Path... shardPaths) throws IOException
      Acquires, then releases, all write.lock files in the given shard paths. The "write.lock" file is assumed to be under the shard path's "index" directory as used by Elasticsearch.
      Throws:
      org.apache.lucene.store.LockObtainFailedException - if any of the locks could not be acquired
      IOException

    • deleteShardDirectoryUnderLock

      public void deleteShardDirectoryUnderLock(ShardLock lock, IndexSettings indexSettings, Consumer<Path[]> listener) throws IOException
      Deletes a shard data directory. Note: this method assumes that the shard lock is acquired. This method will also attempt to acquire the write locks for the shard's paths before deleting the data, but this is best effort, as the lock is released before the deletion happens in order to allow the folder to be deleted
      Parameters:
      lock - the shards lock
      Throws:
      IOException - if an IOException occurs
      ElasticsearchException - if the write.lock is not acquirable

    • deleteIndexDirectorySafe

      public void deleteIndexDirectorySafe(Index index, long lockTimeoutMS, IndexSettings indexSettings, Consumer<Path[]> listener) throws IOException, ShardLockObtainFailedException
      Deletes an indexes data directory recursively iff all of the indexes shards locks were successfully acquired. If any of the indexes shard directories can't be locked non of the shards will be deleted
      Parameters:
      index - the index to delete
      lockTimeoutMS - how long to wait for acquiring the indices shard locks
      indexSettings - settings for the index being deleted
      Throws:
      IOException - if any of the shards data directories can't be locked or deleted
      ShardLockObtainFailedException

    • deleteIndexDirectoryUnderLock

      public void deleteIndexDirectoryUnderLock(Index index, IndexSettings indexSettings, Consumer<Path[]> listener) throws IOException
      Deletes an indexes data directory recursively. Note: this method assumes that the shard lock is acquired
      Parameters:
      index - the index to delete
      indexSettings - settings for the index being deleted
      Throws:
      IOException

    • lockAllForIndex

      public List<ShardLock> lockAllForIndex(Index index, IndexSettings settings, String lockDetails, long lockTimeoutMS) throws ShardLockObtainFailedException
      Tries to lock all local shards for the given index. If any of the shard locks can't be acquired a ShardLockObtainFailedException is thrown and all previously acquired locks are released.
      Parameters:
      index - the index to lock shards for
      lockTimeoutMS - how long to wait for acquiring the indices shard locks
      Returns:
      the ShardLock instances for this index.
      Throws:
      ShardLockObtainFailedException

    • shardLock

      public ShardLock shardLock(ShardId id, String details) throws ShardLockObtainFailedException
      Tries to lock the given shards ID. A shard lock is required to perform any kind of write operation on a shards data directory like deleting files, creating a new index writer or recover from a different shard instance into it. If the shard lock can not be acquired a ShardLockObtainFailedException is thrown. Note: this method will return immediately if the lock can't be acquired.
      Parameters:
      id - the shard ID to lock
      details - information about why the shard is being locked
      Returns:
      the shard lock. Call ShardLock.close() to release the lock
      Throws:
      ShardLockObtainFailedException

    • shardLock

      public ShardLock shardLock(ShardId shardId, String details, long lockTimeoutMS) throws ShardLockObtainFailedException
      Tries to lock the given shards ID. A shard lock is required to perform any kind of write operation on a shards data directory like deleting files, creating a new index writer or recover from a different shard instance into it. If the shard lock can not be acquired a ShardLockObtainFailedException is thrown
      Parameters:
      shardId - the shard ID to lock
      details - information about why the shard is being locked
      lockTimeoutMS - the lock timeout in milliseconds
      Returns:
      the shard lock. Call ShardLock.close() to release the lock
      Throws:
      ShardLockObtainFailedException

    • lockedShards

      public Set<ShardId> lockedShards()
      Returns all currently lock shards. Note: the shard ids return do not contain a valid Index UUID

    • hasNodeFile

      public boolean hasNodeFile()

    • nodeDataPaths

      public Path[] nodeDataPaths()
      Returns an array of all of the nodes data locations.
      Throws:
      IllegalStateException - if the node is not configured to store local locations

    • sharedDataPath

      public Path sharedDataPath()
      Returns shared data path for this node environment

    • nodeId

      public String nodeId()
      returns the unique uuid describing this node. The uuid is persistent in the data folder of this node and remains across restarts.

    • nodePaths

      public NodeEnvironment.NodePath[] nodePaths()
      Returns an array of all of the NodeEnvironment.NodePaths.

    • getNodeLockId

      public int getNodeLockId()

    • indexPaths

      public Path[] indexPaths(Index index)
      Returns all index paths.

    • availableShardPaths

      public Path[] availableShardPaths(ShardId shardId)
      Returns all shard paths excluding custom shard path. Note: Shards are only allocated on one of the returned paths. The returned array may contain paths to non-existing directories.
      See Also:
      IndexSettings.hasCustomDataPath(), resolveCustomLocation(String, ShardId)

    • availableIndexFolders

      public Set<String> availableIndexFolders() throws IOException
      Returns all folder names in ${data.paths}/nodes/{node.id}/indices folder
      Throws:
      IOException

    • availableIndexFolders

      public Set<String> availableIndexFolders(Predicate<String> excludeIndexPathIdsPredicate) throws IOException
      Returns folder names in ${data.paths}/nodes/{node.id}/indices folder that don't match the given predicate.
      Parameters:
      excludeIndexPathIdsPredicate - folder names to exclude
      Throws:
      IOException

    • availableIndexFoldersForPath

      public Set<String> availableIndexFoldersForPath(NodeEnvironment.NodePath nodePath) throws IOException
      Return all directory names in the nodes/{node.id}/indices directory for the given node path.
      Parameters:
      nodePath - the path
      Returns:
      all directories that could be indices for the given node path.
      Throws:
      IOException - if an I/O exception occurs traversing the filesystem

    • availableIndexFoldersForPath

      public Set<String> availableIndexFoldersForPath(NodeEnvironment.NodePath nodePath, Predicate<String> excludeIndexPathIdsPredicate) throws IOException
      Return directory names in the nodes/{node.id}/indices directory for the given node path that don't match the given predicate.
      Parameters:
      nodePath - the path
      excludeIndexPathIdsPredicate - folder names to exclude
      Returns:
      all directories that could be indices for the given node path.
      Throws:
      IOException - if an I/O exception occurs traversing the filesystem

    • resolveIndexFolder

      public Path[] resolveIndexFolder(String indexFolderName)
      Resolves all existing paths to indexFolderName in ${data.paths}/nodes/{node.id}/indices

    • findAllShardIds

      public Set<ShardId> findAllShardIds(Index index) throws IOException
      Tries to find all allocated shards for the given index on the current node. NOTE: This methods is prone to race-conditions on the filesystem layer since it might not see directories created concurrently or while it's traversing.
      Parameters:
      index - the index to filter shards
      Returns:
      a set of shard IDs
      Throws:
      IOException - if an IOException occurs

    • shardCountPerPath

      public Map<NodeEnvironment.NodePath,Long> shardCountPerPath(Index index) throws IOException
      Find all the shards for this index, returning a map of the NodePath to the number of shards on that path
      Parameters:
      index - the index by which to filter shards
      Returns:
      a map of NodePath to count of the shards for the index on that path
      Throws:
      IOException - if an IOException occurs

    • close

      public void close()
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable

    • resolveBaseCustomLocation

      public static Path resolveBaseCustomLocation(String customDataPath, Path sharedDataPath, int nodeLockId)
      Resolve the custom path for a index's shard.

    • resolveCustomLocation

      public Path resolveCustomLocation(String customDataPath, ShardId shardId)
      Resolve the custom path for a index's shard. Uses the IndexMetadata.SETTING_DATA_PATH setting to determine the root path for the index.
      Parameters:
      customDataPath - the custom data path
      shardId - shard to resolve the path to

    • resolveCustomLocation

      public static Path resolveCustomLocation(String customDataPath, ShardId shardId, Path sharedDataPath, int nodeLockId)

    • shardStatePathToDataPath

      public static Path shardStatePathToDataPath(Path shardPath)
      Returns the NodePath.path for this shard.