Closeable
, AutoCloseable
public final class ShingleAnalyzerWrapper
extends org.apache.lucene.analysis.Analyzer
ShingleFilter
around another Analyzer
.
A shingle is another name for a token based n-gram.
Constructor | Description |
---|---|
ShingleAnalyzerWrapper(org.apache.lucene.analysis.Analyzer defaultAnalyzer) |
|
ShingleAnalyzerWrapper(org.apache.lucene.analysis.Analyzer defaultAnalyzer,
int maxShingleSize) |
|
ShingleAnalyzerWrapper(org.apache.lucene.analysis.Analyzer defaultAnalyzer,
int minShingleSize,
int maxShingleSize) |
|
ShingleAnalyzerWrapper(org.apache.lucene.analysis.Analyzer defaultAnalyzer,
int minShingleSize,
int maxShingleSize,
String tokenSeparator,
boolean outputUnigrams,
boolean outputUnigramsIfNoShingles) |
Creates a new ShingleAnalyzerWrapper
|
ShingleAnalyzerWrapper(org.apache.lucene.util.Version matchVersion) |
Wraps
StandardAnalyzer . |
ShingleAnalyzerWrapper(org.apache.lucene.util.Version matchVersion,
int minShingleSize,
int maxShingleSize) |
Wraps
StandardAnalyzer . |
Modifier and Type | Method | Description |
---|---|---|
int |
getMaxShingleSize() |
The max shingle (token ngram) size
|
int |
getMinShingleSize() |
The min shingle (token ngram) size
|
String |
getTokenSeparator() |
|
boolean |
isOutputUnigrams() |
|
boolean |
isOutputUnigramsIfNoShingles() |
|
org.apache.lucene.analysis.TokenStream |
reusableTokenStream(String fieldName,
Reader reader) |
|
void |
setMaxShingleSize(int maxShingleSize) |
Deprecated.
Setting maxShingleSize after Analyzer instantiation prevents reuse.
|
void |
setMinShingleSize(int minShingleSize) |
Deprecated.
Setting minShingleSize after Analyzer instantiation prevents reuse.
|
void |
setOutputUnigrams(boolean outputUnigrams) |
Deprecated.
Setting outputUnigrams after Analyzer instantiation prevents reuse.
|
void |
setOutputUnigramsIfNoShingles(boolean outputUnigramsIfNoShingles) |
Deprecated.
Setting outputUnigramsIfNoShingles after Analyzer instantiation prevents reuse.
|
void |
setTokenSeparator(String tokenSeparator) |
Deprecated.
Setting tokenSeparator after Analyzer instantiation prevents reuse.
|
org.apache.lucene.analysis.TokenStream |
tokenStream(String fieldName,
Reader reader) |
public ShingleAnalyzerWrapper(org.apache.lucene.analysis.Analyzer defaultAnalyzer)
public ShingleAnalyzerWrapper(org.apache.lucene.analysis.Analyzer defaultAnalyzer, int maxShingleSize)
public ShingleAnalyzerWrapper(org.apache.lucene.analysis.Analyzer defaultAnalyzer, int minShingleSize, int maxShingleSize)
public ShingleAnalyzerWrapper(org.apache.lucene.analysis.Analyzer defaultAnalyzer, int minShingleSize, int maxShingleSize, String tokenSeparator, boolean outputUnigrams, boolean outputUnigramsIfNoShingles)
defaultAnalyzer
- Analyzer whose TokenStream is to be filteredminShingleSize
- Min shingle (token ngram) sizemaxShingleSize
- Max shingle sizetokenSeparator
- Used to separate input stream tokens in output shinglesoutputUnigrams
- Whether or not the filter shall pass the original
tokens to the output streamoutputUnigramsIfNoShingles
- Overrides the behavior of outputUnigrams==false for those
times when no shingles are available (because there are fewer than
minShingleSize tokens in the input stream)?
Note that if outputUnigrams==true, then unigrams are always output,
regardless of whether any shingles are available.public ShingleAnalyzerWrapper(org.apache.lucene.util.Version matchVersion)
StandardAnalyzer
.public ShingleAnalyzerWrapper(org.apache.lucene.util.Version matchVersion, int minShingleSize, int maxShingleSize)
StandardAnalyzer
.public int getMaxShingleSize()
@Deprecated public void setMaxShingleSize(int maxShingleSize)
maxShingleSize
- max shingle sizepublic int getMinShingleSize()
@Deprecated public void setMinShingleSize(int minShingleSize)
Set the min shingle size (default: 2).
This method requires that the passed in minShingleSize is not greater than maxShingleSize, so make sure that maxShingleSize is set before calling this method.
minShingleSize
- min size of output shinglespublic String getTokenSeparator()
@Deprecated public void setTokenSeparator(String tokenSeparator)
tokenSeparator
- used to separate input stream tokens in output shinglespublic boolean isOutputUnigrams()
@Deprecated public void setOutputUnigrams(boolean outputUnigrams)
outputUnigrams
- Whether or not the filter shall pass the original
tokens to the output streampublic boolean isOutputUnigramsIfNoShingles()
@Deprecated public void setOutputUnigramsIfNoShingles(boolean outputUnigramsIfNoShingles)
Shall we override the behavior of outputUnigrams==false for those times when no shingles are available (because there are fewer than minShingleSize tokens in the input stream)? (default: false.)
Note that if outputUnigrams==true, then unigrams are always output, regardless of whether any shingles are available.
outputUnigramsIfNoShingles
- Whether or not to output a single
unigram when no shingles are available.public org.apache.lucene.analysis.TokenStream tokenStream(String fieldName, Reader reader)
tokenStream
in class org.apache.lucene.analysis.Analyzer
public org.apache.lucene.analysis.TokenStream reusableTokenStream(String fieldName, Reader reader) throws IOException
reusableTokenStream
in class org.apache.lucene.analysis.Analyzer
IOException
Copyright © 2000-2018 Apache Software Foundation. All Rights Reserved.