com.cybozu.labs.langdetect

Class Detector

java.lang.Object

com.cybozu.labs.langdetect.Detector

public class Detector
extends java.lang.Object

Detector class is to detect language from specified text. Its instance is able to be constructed via the factory class DetectorFactory.

After appending a target text to the Detector instance with append(Reader) or append(String), the detector provides the language detection results for target text via detect() or getProbabilities(). detect() method returns a single language name which has the highest probability. getProbabilities() methods returns a list of multiple languages and their probabilities.

The detector has some parameters for language detection. See setAlpha(double), setMaxTextLength(int) and setPriorMap(HashMap).

 import java.util.ArrayList;
 import com.cybozu.labs.langdetect.Detector;
 import com.cybozu.labs.langdetect.DetectorFactory;
 import com.cybozu.labs.langdetect.Language;

 class LangDetectSample {
     public void init(String profileDirectory) throws LangDetectException {
         DetectorFactory.loadProfile(profileDirectory);
     }
     public String detect(String text) throws LangDetectException {
         Detector detector = DetectorFactory.create();
         detector.append(text);
         return detector.detect();
     }
     public ArrayList detectLangs(String text) throws LangDetectException {
         Detector detector = DetectorFactory.create();
         detector.append(text);
         return detector.getProbabilities();
     }
 }
 

• 4x faster improvement based on Elmer Garduno's code. Thanks!

Author:

Nakatani Shuyo

See Also:

DetectorFactory

Constructor Summary

Constructors

Constructor and Description
Detector(DetectorFactory factory) Constructor.

Method Summary

Methods

Modifier and Type	Method and Description
void	append(java.io.Reader reader) Append the target text for language detection.
void	append(java.lang.String text) Append the target text for language detection.
java.lang.String	detect() Detect language of the target text and return the language name which has the highest probability.
java.util.ArrayList<Language>	getProbabilities() Get language candidates which have high probabilities
void	setAlpha(double alpha) Set smoothing parameter.
void	setMaxTextLength(int max_text_length) Specify max size of target text to use for language detection.
void	setPriorMap(java.util.HashMap<java.lang.String,java.lang.Double> priorMap) Set prior information about language probabilities.
void	setVerbose() Set Verbose Mode(use for debug).

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Detector

public Detector(DetectorFactory factory)

Constructor. Detector instance can be constructed via DetectorFactory.create().

Parameters:

factory - DetectorFactory instance (only DetectorFactory inside)

Method Detail

setVerbose

public void setVerbose()

Set Verbose Mode(use for debug).

setAlpha

public void setAlpha(double alpha)

Set smoothing parameter. The default value is 0.5(i.e. Expected Likelihood Estimate).

Parameters:

alpha - the smoothing parameter

setPriorMap

public void setPriorMap(java.util.HashMap<java.lang.String,java.lang.Double> priorMap)
                 throws LangDetectException

Set prior information about language probabilities.

Parameters:

priorMap - the priorMap to set

Throws:

LangDetectException

setMaxTextLength

public void setMaxTextLength(int max_text_length)

Specify max size of target text to use for language detection. The default value is 10000(10KB).

Parameters:

max_text_length - the max_text_length to set

append

public void append(java.io.Reader reader)
            throws java.io.IOException

Append the target text for language detection. This method read the text from specified input reader. If the total size of target text exceeds the limit size specified by setMaxTextLength(int), the rest is cut down.

Parameters:

reader - the input reader (BufferedReader as usual)

Throws:

java.io.IOException - Can't read the reader.

append

public void append(java.lang.String text)

Append the target text for language detection. If the total size of target text exceeds the limit size specified by setMaxTextLength(int), the rest is cut down.

Parameters:

text - the target text to append

detect

public java.lang.String detect()
                        throws LangDetectException

Detect language of the target text and return the language name which has the highest probability.

Returns:

detected language name which has most probability.

Throws:

LangDetectException - code = ErrorCode.CantDetectError : Can't detect because of no valid features in text

getProbabilities

public java.util.ArrayList<Language> getProbabilities()
                                               throws LangDetectException

Get language candidates which have high probabilities

Returns:

possible languages list (whose probabilities are over PROB_THRESHOLD, ordered by probabilities descendently

Throws:

LangDetectException - code = ErrorCode.CantDetectError : Can't detect because of no valid features in text