org.apache.lucene.index

Class FilterMergePolicy

java.lang.Object

org.apache.lucene.index.MergePolicy

org.apache.lucene.index.FilterMergePolicy

Direct Known Subclasses:

OneMergeWrappingMergePolicy, UpgradeIndexMergePolicy

public class FilterMergePolicy
extends MergePolicy

A wrapper for MergePolicy instances.

WARNING: This API is experimental and might change in incompatible ways in the next release.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.lucene.index.MergePolicy

MergePolicy.MergeAbortedException, MergePolicy.MergeContext, MergePolicy.MergeException, MergePolicy.MergeSpecification, MergePolicy.OneMerge, MergePolicy.OneMergeProgress

Field Summary

Fields

Modifier and Type	Field and Description
protected MergePolicy	in The wrapped MergePolicy.

Fields inherited from class org.apache.lucene.index.MergePolicy

DEFAULT_MAX_CFS_SEGMENT_SIZE, DEFAULT_NO_CFS_RATIO, maxCFSSegmentSize, noCFSRatio

Constructor Summary

Constructors

Constructor and Description
FilterMergePolicy(MergePolicy in) Creates a new filter merge policy instance wrapping another.

Method Summary

Modifier and Type	Method and Description
MergePolicy.MergeSpecification	findForcedDeletesMerges(SegmentInfos segmentInfos, MergePolicy.MergeContext mergeContext) Determine what set of merge operations is necessary in order to expunge all deletes from the index.
MergePolicy.MergeSpecification	findForcedMerges(SegmentInfos segmentInfos, int maxSegmentCount, Map<SegmentCommitInfo,Boolean> segmentsToMerge, MergePolicy.MergeContext mergeContext) Determine what set of merge operations is necessary in order to merge to <= the specified segment count.
MergePolicy.MergeSpecification	findFullFlushMerges(MergeTrigger mergeTrigger, SegmentInfos segmentInfos, MergePolicy.MergeContext mergeContext) Identifies merges that we want to execute (synchronously) on commit.
MergePolicy.MergeSpecification	findMerges(MergeTrigger mergeTrigger, SegmentInfos segmentInfos, MergePolicy.MergeContext mergeContext) Determine what set of merge operations are now necessary on the index.
double	getMaxCFSSegmentSizeMB() Returns the largest size allowed for a compound file segment
double	getNoCFSRatio() Returns current noCFSRatio.
boolean	keepFullyDeletedSegment(IOSupplier<CodecReader> readerIOSupplier) Returns true if the segment represented by the given CodecReader should be keep even if it's fully deleted.
int	numDeletesToMerge(SegmentCommitInfo info, int delCount, IOSupplier<CodecReader> readerSupplier) Returns the number of deletes that a merge would claim on the given segment.
void	setMaxCFSSegmentSizeMB(double v) If a merged segment will be more than this value, leave the segment as non-compound file even if compound file is enabled.
void	setNoCFSRatio(double noCFSRatio) If a merged segment will be more than this percentage of the total size of the index, leave the segment as non-compound file even if compound file is enabled.
protected long	size(SegmentCommitInfo info, MergePolicy.MergeContext context) Return the byte size of the provided SegmentCommitInfo, pro-rated by percentage of non-deleted documents is set.
String	toString()
boolean	useCompoundFile(SegmentInfos infos, SegmentCommitInfo mergedInfo, MergePolicy.MergeContext mergeContext) Returns true if a new segment (regardless of its origin) should use the compound file format.

Methods inherited from class org.apache.lucene.index.MergePolicy

assertDelCount, isMerged, message, segString, verbose

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

in

protected final MergePolicy in

The wrapped MergePolicy.

Constructor Detail

FilterMergePolicy

public FilterMergePolicy(MergePolicy in)

Creates a new filter merge policy instance wrapping another.

Parameters:

in - the wrapped MergePolicy

Method Detail

findMerges

public MergePolicy.MergeSpecification findMerges(MergeTrigger mergeTrigger,
                                                 SegmentInfos segmentInfos,
                                                 MergePolicy.MergeContext mergeContext)
                                          throws IOException

Description copied from class: MergePolicy

Determine what set of merge operations are now necessary on the index. IndexWriter calls this whenever there is a change to the segments. This call is always synchronized on the IndexWriter instance so only one thread at a time will call this method.

Specified by:

findMerges in class MergePolicy

Parameters:

mergeTrigger - the event that triggered the merge

segmentInfos - the total set of segments in the index

mergeContext - the IndexWriter to find the merges on

Throws:

IOException

findForcedMerges

public MergePolicy.MergeSpecification findForcedMerges(SegmentInfos segmentInfos,
                                                       int maxSegmentCount,
                                                       Map<SegmentCommitInfo,Boolean> segmentsToMerge,
                                                       MergePolicy.MergeContext mergeContext)
                                                throws IOException

Description copied from class: MergePolicy

Determine what set of merge operations is necessary in order to merge to <= the specified segment count. IndexWriter calls this when its IndexWriter.forceMerge(int) method is called. This call is always synchronized on the IndexWriter instance so only one thread at a time will call this method.

Specified by:

findForcedMerges in class MergePolicy

Parameters:

segmentInfos - the total set of segments in the index

maxSegmentCount - requested maximum number of segments in the index (currently this is always 1)

segmentsToMerge - contains the specific SegmentInfo instances that must be merged away. This may be a subset of all SegmentInfos. If the value is True for a given SegmentInfo, that means this segment was an original segment present in the to-be-merged index; else, it was a segment produced by a cascaded merge.

mergeContext - the MergeContext to find the merges on

Throws:

IOException

findForcedDeletesMerges

public MergePolicy.MergeSpecification findForcedDeletesMerges(SegmentInfos segmentInfos,
                                                              MergePolicy.MergeContext mergeContext)
                                                       throws IOException

Description copied from class: MergePolicy

Determine what set of merge operations is necessary in order to expunge all deletes from the index.

Specified by:

findForcedDeletesMerges in class MergePolicy

Parameters:

segmentInfos - the total set of segments in the index

mergeContext - the MergeContext to find the merges on

Throws:

IOException

findFullFlushMerges

public MergePolicy.MergeSpecification findFullFlushMerges(MergeTrigger mergeTrigger,
                                                          SegmentInfos segmentInfos,
                                                          MergePolicy.MergeContext mergeContext)
                                                   throws IOException

Description copied from class: MergePolicy

Identifies merges that we want to execute (synchronously) on commit. By default, this will do no merging on commit. If you implement this method in your MergePolicy you must also set a non-zero timeout using IndexWriterConfig.setMaxFullFlushMergeWaitMillis(long). Any merges returned here will make IndexWriter.commit(), IndexWriter.prepareCommit() or IndexWriter.getReader(boolean, boolean) block until the merges complete or until LiveIndexWriterConfig.getMaxFullFlushMergeWaitMillis() has elapsed. This may be used to merge small segments that have just been flushed, reducing the number of segments in the point in time snapshot. If a merge does not complete in the allotted time, it will continue to execute, and eventually finish and apply to future point in time snapshot, but will not be reflected in the current one. If a MergePolicy.OneMerge in the returned MergePolicy.MergeSpecification includes a segment already included in a registered merge, then IndexWriter.commit() or IndexWriter.prepareCommit() will throw a IllegalStateException. Use MergePolicy.MergeContext.getMergingSegments() to determine which segments are currently registered to merge.

Overrides:

findFullFlushMerges in class MergePolicy

Parameters:

mergeTrigger - the event that triggered the merge (COMMIT or GET_READER).

segmentInfos - the total set of segments in the index (while preparing the commit)

mergeContext - the MergeContext to find the merges on, which should be used to determine which segments are already in a registered merge (see MergePolicy.MergeContext.getMergingSegments()).

Throws:

IOException

useCompoundFile

public boolean useCompoundFile(SegmentInfos infos,
                               SegmentCommitInfo mergedInfo,
                               MergePolicy.MergeContext mergeContext)
                        throws IOException

Description copied from class: MergePolicy

Returns true if a new segment (regardless of its origin) should use the compound file format. The default implementation returns true iff the size of the given mergedInfo is less or equal to MergePolicy.getMaxCFSSegmentSizeMB() and the size is less or equal to the TotalIndexSize * MergePolicy.getNoCFSRatio() otherwise false.

Overrides:

useCompoundFile in class MergePolicy

Throws:

IOException

size

protected long size(SegmentCommitInfo info,
                    MergePolicy.MergeContext context)
             throws IOException

Description copied from class: MergePolicy

Return the byte size of the provided SegmentCommitInfo, pro-rated by percentage of non-deleted documents is set.

Overrides:

size in class MergePolicy

Throws:

IOException

getNoCFSRatio

public double getNoCFSRatio()

Description copied from class: MergePolicy

Returns current noCFSRatio.

Overrides:

getNoCFSRatio in class MergePolicy

See Also:

MergePolicy.setNoCFSRatio(double)

setNoCFSRatio

public final void setNoCFSRatio(double noCFSRatio)

Description copied from class: MergePolicy

If a merged segment will be more than this percentage of the total size of the index, leave the segment as non-compound file even if compound file is enabled. Set to 1.0 to always use CFS regardless of merge size.

Overrides:

setNoCFSRatio in class MergePolicy

setMaxCFSSegmentSizeMB

public final void setMaxCFSSegmentSizeMB(double v)

Description copied from class: MergePolicy

If a merged segment will be more than this value, leave the segment as non-compound file even if compound file is enabled. Set this to Double.POSITIVE_INFINITY (default) and noCFSRatio to 1.0 to always use CFS regardless of merge size.

Overrides:

setMaxCFSSegmentSizeMB in class MergePolicy

getMaxCFSSegmentSizeMB

public final double getMaxCFSSegmentSizeMB()

Description copied from class: MergePolicy

Returns the largest size allowed for a compound file segment

Overrides:

getMaxCFSSegmentSizeMB in class MergePolicy

toString

public String toString()

Overrides:

toString in class Object

keepFullyDeletedSegment

public boolean keepFullyDeletedSegment(IOSupplier<CodecReader> readerIOSupplier)
                                throws IOException

Description copied from class: MergePolicy

Returns true if the segment represented by the given CodecReader should be keep even if it's fully deleted. This is useful for testing of for instance if the merge policy implements retention policies for soft deletes.

Overrides:

keepFullyDeletedSegment in class MergePolicy

Throws:

IOException

numDeletesToMerge

public int numDeletesToMerge(SegmentCommitInfo info,
                             int delCount,
                             IOSupplier<CodecReader> readerSupplier)
                      throws IOException

Description copied from class: MergePolicy

Returns the number of deletes that a merge would claim on the given segment. This method will by default return the sum of the del count on disk and the pending delete count. Yet, subclasses that wrap merge readers might modify this to reflect deletes that are carried over to the target segment in the case of soft deletes. Soft deletes all deletes to survive across merges in order to control when the soft-deleted data is claimed.

Overrides:

numDeletesToMerge in class MergePolicy

Parameters:

info - the segment info that identifies the segment

delCount - the number deleted documents for this segment

readerSupplier - a supplier that allows to obtain a CodecReader for this segment

Throws:

IOException

See Also:

IndexWriter.softUpdateDocument(Term, Iterable, Field...), IndexWriterConfig.setSoftDeletesField(String)