org.apache.lucene.codecs.uniformsplit

Class BlockReader

java.lang.Object

org.apache.lucene.index.TermsEnum

org.apache.lucene.index.BaseTermsEnum

org.apache.lucene.codecs.uniformsplit.BlockReader

All Implemented Interfaces:

Accountable, BytesRefIterator

Direct Known Subclasses:

IntersectBlockReader, STBlockReader

public class BlockReader
extends BaseTermsEnum
implements Accountable

Seeks the block corresponding to a given term, read the block bytes, and scans the block terms.

Reads fully the block in blockReadBuffer. Then scans the block terms in memory. The details region is lazily decoded with termStatesReadBuffer which shares the same byte array with blockReadBuffer. See BlockWriter and BlockLine for the block format.

WARNING: This API is experimental and might change in incompatible ways in the next release.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.lucene.index.TermsEnum

TermsEnum.SeekStatus

Field Summary

Fields

Modifier and Type	Field and Description
protected BlockDecoder	blockDecoder
protected int	blockFirstLineStart Offset of the start of the first line of the current block (just after the header), relative to the block start.
protected BlockHeader	blockHeader Current block header.
protected BlockHeader.Serializer	blockHeaderReader
protected IndexInput	blockInput IndexInput on the block file.
protected BlockLine	blockLine Current block line.
protected BlockLine.Serializer	blockLineReader
protected ByteArrayDataInput	blockReadBuffer In-memory read buffer for the current block.
protected long	blockStartFP Current block start file pointer, absolute in the block file.
protected IndexDictionary.Browser	dictionaryBrowser Holds the IndexDictionary.Browser once loaded.
protected IndexDictionary.BrowserSupplier	dictionaryBrowserSupplier IndexDictionary.Browser supplier for lazy loading.
protected FieldMetadata	fieldMetadata
protected BytesRefBuilder	forcedTerm Set when seekExact(BytesRef, TermState) is called.
protected int	lineIndexInBlock Current line index in the block.
protected PostingsReaderBase	postingsReader
protected BytesRef	scratchBlockBytes
protected BlockLine	scratchBlockLine
protected BlockTermState	scratchTermState
protected BlockTermState	termState Current block line details.
protected boolean	termStateForced Whether the current TermState has been forced with a call to seekExact(BytesRef, TermState).
protected DeltaBaseTermStateSerializer	termStateSerializer
protected ByteArrayDataInput	termStatesReadBuffer In-memory read buffer for the details region of the current block.

Fields inherited from class org.apache.lucene.index.TermsEnum

EMPTY

Fields inherited from interface org.apache.lucene.util.Accountable

NULL_ACCOUNTABLE

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	BlockReader(IndexDictionary.BrowserSupplier dictionaryBrowserSupplier, IndexInput blockInput, PostingsReaderBase postingsReader, FieldMetadata fieldMetadata, BlockDecoder blockDecoder)

Method Summary

Modifier and Type	Method and Description
protected void	clearTermState() Called by the primary TermsEnum methods to clear the previous TermState.
protected int	compareToMiddleAndJump(BytesRef searchedTerm) Compares the searched term to the middle term of the block.
protected BlockHeader.Serializer	createBlockHeaderSerializer()
protected BlockLine.Serializer	createBlockLineSerializer()
protected DeltaBaseTermStateSerializer	createDeltaBaseTermStateSerializer()
protected BytesRef	decodeBlockBytesIfNeeded(int numBlockBytes)
int	docFreq()
protected IndexDictionary.Browser	getOrCreateDictionaryBrowser()
ImpactsEnum	impacts(int flags)
protected void	initializeBlockReadLazily()
protected void	initializeHeader(BytesRef searchedTerm, long targetBlockStartFP) Reads and sets blockHeader.
protected boolean	isBeyondLastTerm(BytesRef searchedTerm, long blockStartFP) Indicates whether the searched term is beyond the last term of the field.
protected boolean	isCurrentTerm(BytesRef searchedTerm)
protected CorruptIndexException	newCorruptIndexException(String msg, Long fp)
BytesRef	next()
protected BytesRef	nextTerm() Moves to the next term line and reads it, it may be in the next block.
long	ord()
PostingsEnum	postings(PostingsEnum reuse, int flags)
long	ramBytesUsed()
protected BlockHeader	readHeader() Reads the block header.
protected BlockLine	readLineInBlock() Reads the current block line.
protected BlockTermState	readTermState() Reads the BlockTermState on the current line.
protected BlockTermState	readTermStateIfNotRead() Reads the BlockTermState if it is not already set.
TermsEnum.SeekStatus	seekCeil(BytesRef searchedTerm)
boolean	seekExact(BytesRef searchedTerm)
void	seekExact(BytesRef term, TermState state) Positions this BlockReader without re-seeking the term dictionary.
void	seekExact(long ord) Not supported.
protected TermsEnum.SeekStatus	seekInBlock(BytesRef searchedTerm) Seeks to the provided term in this block.
protected TermsEnum.SeekStatus	seekInBlock(BytesRef searchedTerm, long blockStartFP) Seeks to the provided term in the block starting at the provided file pointer.
BytesRef	term()
TermState	termState()
long	totalTermFreq()

Methods inherited from class org.apache.lucene.index.BaseTermsEnum

attributes

Methods inherited from class org.apache.lucene.index.TermsEnum

postings

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.lucene.util.Accountable

getChildResources

Field Detail

blockInput

protected IndexInput blockInput

IndexInput on the block file.

postingsReader

protected final PostingsReaderBase postingsReader

fieldMetadata

protected final FieldMetadata fieldMetadata

blockDecoder

protected final BlockDecoder blockDecoder

blockHeaderReader

protected BlockHeader.Serializer blockHeaderReader

blockLineReader

protected BlockLine.Serializer blockLineReader

blockReadBuffer

protected ByteArrayDataInput blockReadBuffer

In-memory read buffer for the current block.

termStatesReadBuffer

protected ByteArrayDataInput termStatesReadBuffer

In-memory read buffer for the details region of the current block. It shares the same byte array as blockReadBuffer, with a different position.

termStateSerializer

protected DeltaBaseTermStateSerializer termStateSerializer

dictionaryBrowserSupplier

protected final IndexDictionary.BrowserSupplier dictionaryBrowserSupplier

IndexDictionary.Browser supplier for lazy loading.

dictionaryBrowser

protected IndexDictionary.Browser dictionaryBrowser

Holds the IndexDictionary.Browser once loaded.

blockStartFP

protected long blockStartFP

Current block start file pointer, absolute in the block file.

blockHeader

protected BlockHeader blockHeader

Current block header.

blockLine

protected BlockLine blockLine

Current block line.

termState

protected BlockTermState termState

Current block line details.

blockFirstLineStart

protected int blockFirstLineStart

Offset of the start of the first line of the current block (just after the header), relative to the block start.

lineIndexInBlock

protected int lineIndexInBlock

Current line index in the block.

termStateForced

protected boolean termStateForced

Whether the current TermState has been forced with a call to seekExact(BytesRef, TermState).

See Also:

forcedTerm

forcedTerm

protected BytesRefBuilder forcedTerm

Set when seekExact(BytesRef, TermState) is called.

This optimizes the use-case when the caller calls first seekExact(BytesRef, TermState) and then postings(PostingsEnum, int). In this case we don't access the terms block file (we don't seek) but directly the postings file because we already have the TermState with the file pointers to the postings file.

scratchBlockBytes

protected BytesRef scratchBlockBytes

scratchTermState

protected final BlockTermState scratchTermState

scratchBlockLine

protected BlockLine scratchBlockLine

Constructor Detail

BlockReader

protected BlockReader(IndexDictionary.BrowserSupplier dictionaryBrowserSupplier,
                      IndexInput blockInput,
                      PostingsReaderBase postingsReader,
                      FieldMetadata fieldMetadata,
                      BlockDecoder blockDecoder)
               throws IOException

Parameters:

dictionaryBrowserSupplier - to load the IndexDictionary.Browser lazily in seekCeil(BytesRef).

blockDecoder - Optional block decoder, may be null if none. It can be used for decompression or decryption.

Throws:

IOException

Method Detail

seekCeil

public TermsEnum.SeekStatus seekCeil(BytesRef searchedTerm)
                              throws IOException

Specified by:

seekCeil in class TermsEnum

Throws:

IOException

seekExact

public boolean seekExact(BytesRef searchedTerm)
                  throws IOException

Overrides:

seekExact in class BaseTermsEnum

Throws:

IOException

isCurrentTerm

protected boolean isCurrentTerm(BytesRef searchedTerm)

isBeyondLastTerm

protected boolean isBeyondLastTerm(BytesRef searchedTerm,
                                   long blockStartFP)

Indicates whether the searched term is beyond the last term of the field.

Parameters:

blockStartFP - The current block start file pointer.

seekInBlock

protected TermsEnum.SeekStatus seekInBlock(BytesRef searchedTerm,
                                           long blockStartFP)
                                    throws IOException

Seeks to the provided term in the block starting at the provided file pointer. Does not exceed the block.

Throws:

IOException

seekInBlock

protected TermsEnum.SeekStatus seekInBlock(BytesRef searchedTerm)
                                    throws IOException

Seeks to the provided term in this block.

Does not exceed this block; TermsEnum.SeekStatus.END is returned if it follows the block.

Compares the line terms with the searchedTerm, taking advantage of the incremental encoding properties.

Scans linearly the terms. Updates the current block line with the current term.

Throws:

IOException

compareToMiddleAndJump

protected int compareToMiddleAndJump(BytesRef searchedTerm)
                              throws IOException

Compares the searched term to the middle term of the block. If the searched term is lexicographically equal or after the middle term then jumps to the second half of the block directly.

Returns:

The comparison between the searched term and the middle term.

Throws:

IOException

readLineInBlock

protected BlockLine readLineInBlock()
                             throws IOException

Reads the current block line. Sets blockLine and increments lineIndexInBlock.

Returns:

The BlockLine; or null if there no more line in the block.

Throws:

IOException

seekExact

public void seekExact(BytesRef term,
                      TermState state)

Positions this BlockReader without re-seeking the term dictionary.

The block containing the term is not read by this method. It will be read lazily only if needed, for example if next() is called. Calling postings(org.apache.lucene.index.PostingsEnum, int) after this method does require the block to be read.

Overrides:

seekExact in class BaseTermsEnum

seekExact

public void seekExact(long ord)

Not supported.

Specified by:

seekExact in class TermsEnum

next

public BytesRef next()
              throws IOException

Specified by:

next in interface BytesRefIterator

Throws:

IOException

nextTerm

protected BytesRef nextTerm()
                     throws IOException

Moves to the next term line and reads it, it may be in the next block. The term details are not read yet. They will be read only when needed with readTermStateIfNotRead().

Returns:

The read term bytes; or null if there is no more term for the field.

Throws:

IOException

initializeHeader

protected void initializeHeader(BytesRef searchedTerm,
                                long targetBlockStartFP)
                         throws IOException

Reads and sets blockHeader. Sets null if there is no block for the field anymore.

Parameters:

searchedTerm - The searched term; or null if none.

targetBlockStartFP - The file pointer of the block to read.

Throws:

IOException

initializeBlockReadLazily

protected void initializeBlockReadLazily()
                                  throws IOException

Throws:

IOException

createBlockHeaderSerializer

protected BlockHeader.Serializer createBlockHeaderSerializer()

createBlockLineSerializer

protected BlockLine.Serializer createBlockLineSerializer()

createDeltaBaseTermStateSerializer

protected DeltaBaseTermStateSerializer createDeltaBaseTermStateSerializer()

readHeader

protected BlockHeader readHeader()
                          throws IOException

Reads the block header. Sets blockHeader.

Returns:

The block header; or null if there is no block for the field anymore.

Throws:

IOException

decodeBlockBytesIfNeeded

protected BytesRef decodeBlockBytesIfNeeded(int numBlockBytes)
                                     throws IOException

Throws:

IOException

readTermStateIfNotRead

protected BlockTermState readTermStateIfNotRead()
                                         throws IOException

Reads the BlockTermState if it is not already set. Sets termState.

Throws:

IOException

readTermState

protected BlockTermState readTermState()
                                throws IOException

Reads the BlockTermState on the current line. Sets termState.

Overriding method may return null if there is no BlockTermState (in this case the extending class must support a null termState).

Returns:

The BlockTermState; or null if none.

Throws:

IOException

term

public BytesRef term()

Specified by:

term in class TermsEnum

ord

public long ord()

Specified by:

ord in class TermsEnum

docFreq

public int docFreq()
            throws IOException

Specified by:

docFreq in class TermsEnum

Throws:

IOException

totalTermFreq

public long totalTermFreq()
                   throws IOException

Specified by:

totalTermFreq in class TermsEnum

Throws:

IOException

termState

public TermState termState()
                    throws IOException

Overrides:

termState in class BaseTermsEnum

Throws:

IOException

postings

public PostingsEnum postings(PostingsEnum reuse,
                             int flags)
                      throws IOException

Specified by:

postings in class TermsEnum

Throws:

IOException

impacts

public ImpactsEnum impacts(int flags)
                    throws IOException

Specified by:

impacts in class TermsEnum

Throws:

IOException

ramBytesUsed

public long ramBytesUsed()

Specified by:

ramBytesUsed in interface Accountable

getOrCreateDictionaryBrowser

protected IndexDictionary.Browser getOrCreateDictionaryBrowser()
                                                        throws IOException

Throws:

IOException

clearTermState

protected void clearTermState()

Called by the primary TermsEnum methods to clear the previous TermState.

newCorruptIndexException

protected CorruptIndexException newCorruptIndexException(String msg,
                                                         Long fp)