public class SegTokenFilter extends Object
Filters a SegToken
by converting full-width latin to half-width, then lowercasing latin.
Additionally, all punctuation is converted into Utility.COMMON_DELIMITER
Constructor | Description |
---|---|
SegTokenFilter() |
Copyright © 2000-2018 Apache Software Foundation. All Rights Reserved.