public class EvaluateModel
extends java.lang.Object
Constructor and Description |
---|
EvaluateModel(M5Instances data)
Initializes all the counters for the evaluation and also takes a
cost matrix as parameter.
|
Modifier and Type | Method and Description |
---|---|
double |
avgCost()
Gets the average cost, that is, total cost of misclassifications
(incorrect plus unclassified) over the total number of instances.
|
double[][] |
confusionMatrix()
Returns a copy of the confusion matrix.
|
double |
correct()
Gets the number of instances correctly classified (that is, for
which a correct prediction was made).
|
double |
correlationCoefficient()
Returns the correlation coefficient if the class is numeric.
|
void |
crossValidateModel(M5 classifier,
M5Instances data,
int numFolds)
Performs a (stratified if class is nominal) cross-validation
for a classifier on a set of instances.
|
void |
crossValidateModel(java.lang.String classifierString,
M5Instances data,
int numFolds,
java.lang.String[] options)
Performs a (stratified if class is nominal) cross-validation
for a classifier on a set of instances.
|
boolean |
equals(java.lang.Object obj)
Tests whether the current evaluation object is equal to another
evaluation object
|
double |
errorRate()
Returns the estimated error rate or the root mean squared error
(if the class is numeric).
|
void |
evaluateModel(M5 classifier,
M5Instances data)
Evaluates the classifier on a given set of instances.
|
static java.lang.String |
evaluateModel(M5 classifier,
java.lang.String[] options)
Evaluates a classifier with the options given in an array of
strings.
|
static java.lang.String |
evaluateModel(java.lang.String classifierString,
java.lang.String[] options)
Evaluate the model M5 by executing the function evaluateModel(M5 classifier,
String[] options).
|
double |
evaluateModelOnce(double[] dist,
M5Instance instance)
Evaluates the supplied distribution on a single instance.
|
void |
evaluateModelOnce(double prediction,
M5Instance instance)
Evaluates the supplied prediction on a single instance.
|
double |
evaluateModelOnce(M5 classifier,
M5Instance instance)
Evaluates the classifier on a single instance.
|
double |
falseNegativeRate(int classIndex)
Calculate the false negative rate with respect to a particular class.
|
double |
falsePositiveRate(int classIndex)
Calculate the false positive rate with respect to a particular class.
|
double |
fMeasure(int classIndex)
Calculate the F-Measure with respect to a particular class.
|
double |
incorrect()
Gets the number of instances incorrectly classified (that is, for
which an incorrect prediction was made).
|
double |
kappa()
Returns value of kappa statistic if class is nominal.
|
double |
KBInformation()
Return the total Kononenko & Bratko Information score in bits
|
double |
KBMeanInformation()
Return the Kononenko & Bratko Information score in bits per
instance.
|
double |
KBRelativeInformation()
Return the Kononenko & Bratko Relative Information score
|
static void |
main(java.lang.String[] args)
A test method for this class.
|
double |
meanAbsoluteError()
Returns the mean absolute error.
|
double |
meanPriorAbsoluteError()
Returns the mean absolute error of the prior.
|
double |
numFalseNegatives(int classIndex)
Calculate number of false negatives with respect to a particular class.
|
double |
numFalsePositives(int classIndex)
Calculate number of false positives with respect to a particular class.
|
double |
numInstances()
Gets the number of test instances that had a known class value
(actually the sum of the weights of test instances with known
class value).
|
double |
numTrueNegatives(int classIndex)
Calculate the number of true negatives with respect to a particular class.
|
double |
numTruePositives(int classIndex)
Calculate the number of true positives with respect to a particular class.
|
double |
pctCorrect()
Gets the percentage of instances correctly classified (that is, for
which a correct prediction was made).
|
double |
pctIncorrect()
Gets the percentage of instances incorrectly classified (that is, for
which an incorrect prediction was made).
|
double |
pctUnclassified()
Gets the percentage of instances not classified (that is, for
which no prediction was made by the classifier).
|
double |
precision(int classIndex)
Calculate the precision with respect to a particular class.
|
double |
priorEntropy()
Calculate the entropy of the prior distribution
|
double |
recall(int classIndex)
Calculate the recall with respect to a particular class.
|
double |
relativeAbsoluteError()
Returns the relative absolute error.
|
double |
rootMeanPriorSquaredError()
Returns the root mean prior squared error.
|
double |
rootMeanSquaredError()
Returns the root mean squared error.
|
double |
rootRelativeSquaredError()
Returns the root relative squared error if the class is numeric.
|
static double |
roundNum(double num)
To round to 4 decimal in a double.
|
void |
setPriors(M5Instances train)
Sets the class prior probabilities
|
double |
SFEntropyGain()
Returns the total SF, which is the null model entropy minus
the scheme entropy.
|
double |
SFMeanEntropyGain()
Returns the SF per instance, which is the null model entropy
minus the scheme entropy, per instance.
|
double |
SFMeanPriorEntropy()
Returns the entropy per instance for the null model
|
double |
SFMeanSchemeEntropy()
Returns the entropy per instance for the scheme
|
double |
SFPriorEntropy()
Returns the total entropy for the null model
|
double |
SFSchemeEntropy()
Returns the total entropy for the scheme
|
java.lang.String |
toClassDetailsString()
Returns a string with the deatails of this class.
|
java.lang.String |
toClassDetailsString(java.lang.String title)
Generates a breakdown of the accuracy for each class,
incorporating various information-retrieval statistics, such as
true/false positive rate, precision/recall/F-Measure.
|
java.lang.String |
toCumulativeMarginDistributionString()
Output the cumulative margin distribution as a string suitable
for input for gnuplot or similar package.
|
java.lang.String |
toMatrixString()
Calls toMatrixString() with a default title.
|
java.lang.String |
toMatrixString(java.lang.String title)
Outputs the performance statistics as a classification confusion
matrix.
|
java.lang.String |
toSummaryString()
Calls toSummaryString() with no title and no complexity stats
|
java.lang.String |
toSummaryString(boolean printComplexityStatistics)
Calls toSummaryString() with a default title.
|
java.lang.String |
toSummaryString(java.lang.String title,
boolean printComplexityStatistics)
Outputs the performance statistics in summary form.
|
double |
totalCost()
Gets the total cost, that is, the cost of each prediction times the
weight of the instance, summed over all instances.
|
double |
trueNegativeRate(int classIndex)
Calculate the true negative rate with respect to a particular class.
|
double |
truePositiveRate(int classIndex)
Calculate the true positive rate with respect to a particular class.
|
double |
unclassified()
Gets the number of instances not classified (that is, for
which no prediction was made by the classifier).
|
void |
updatePriors(M5Instance instance)
Updates the class prior probabilities (when incrementally
training)
|
public EvaluateModel(M5Instances data) throws java.lang.Exception
data
- set of instances, to get some header informationjava.lang.Exception
- if cost matrix is not compatible with
data, the class is not defined or the class is numericpublic double[][] confusionMatrix()
public void crossValidateModel(M5 classifier, M5Instances data, int numFolds) throws java.lang.Exception
classifier
- the classifier with any options set.data
- the data on which the cross-validation is to be
performednumFolds
- the number of folds for the cross-validationjava.lang.Exception
- if a classifier could not be generated
successfully or the class is not definedpublic void crossValidateModel(java.lang.String classifierString, M5Instances data, int numFolds, java.lang.String[] options) throws java.lang.Exception
classifierString
- a string naming the class of the classifierdata
- the data on which the cross-validation is to be
performednumFolds
- the number of folds for the cross-validationoptions
- the options to the classifier. Any options
accepted by the classifier will be removed from this array.java.lang.Exception
- if a classifier could not be generated
successfully or the class is not definedpublic static java.lang.String evaluateModel(java.lang.String classifierString, java.lang.String[] options) throws java.lang.Exception
classifierString
- classifier name.options
- the array of string containing the optionsjava.lang.Exception
- if model could not be evaluated successfullypublic static void main(java.lang.String[] args)
args
- an array of command line arguments, the first of which
must be the class name of a classifier.public static java.lang.String evaluateModel(M5 classifier, java.lang.String[] options) throws java.lang.Exception
Valid options are:
-t name of training file
Name of the file with the training data. (required)
-T name of test file
Name of the file with the test data. If missing a cross-validation
is performed.
-c class index
Index of the class attribute (1, 2, ...; default: last).
-x number of folds
The number of folds for the cross-validation (default: 10).
-s random number seed
Random number seed for the cross-validation (default: 1).
-m file with cost matrix
The name of a file containing a cost matrix.
-l name of model input file
Loads classifier from the given file.
-d name of model output file
Saves classifier built from the training data into the given file.
-v
Outputs no statistics for the training data.
-o
Outputs statistics only, not the classifier.
-i
Outputs detailed information-retrieval statistics per class.
-k
Outputs information-theoretic statistics.
-p
Outputs predictions for test instances (and nothing else).
-r
Outputs cumulative margin distribution (and nothing else).
-g
Only for classifiers that implement "Graphable." Outputs
the graph representation of the classifier (and nothing
else).
classifier
- machine learning classifieroptions
- the array of string containing the optionsjava.lang.Exception
- if model could not be evaluated successfullypublic void evaluateModel(M5 classifier, M5Instances data) throws java.lang.Exception
classifier
- machine learning classifierdata
- set of test instances for evaluationjava.lang.Exception
- if model could not be evaluated
successfullypublic double evaluateModelOnce(M5 classifier, M5Instance instance) throws java.lang.Exception
classifier
- machine learning classifierinstance
- the test instance to be classifiedjava.lang.Exception
- if model could not be evaluated
successfully or the data contains string attributespublic double evaluateModelOnce(double[] dist, M5Instance instance) throws java.lang.Exception
dist
- the supplied distributioninstance
- the test instance to be classifiedjava.lang.Exception
- if model could not be evaluated
successfullypublic void evaluateModelOnce(double prediction, M5Instance instance) throws java.lang.Exception
prediction
- the supplied predictioninstance
- the test instance to be classifiedjava.lang.Exception
- if model could not be evaluated
successfullypublic final double numInstances()
public final double incorrect()
public final double pctIncorrect()
public final double totalCost()
public final double avgCost()
public final double correct()
public final double pctCorrect()
public final double unclassified()
public final double pctUnclassified()
public final double errorRate()
public final double kappa()
public final double correlationCoefficient() throws java.lang.Exception
java.lang.Exception
- if class is not numericpublic final double meanAbsoluteError()
public final double meanPriorAbsoluteError()
public final double relativeAbsoluteError() throws java.lang.Exception
java.lang.Exception
- if it can't be computedpublic final double rootMeanSquaredError()
public final double rootMeanPriorSquaredError()
public final double rootRelativeSquaredError()
public final double priorEntropy() throws java.lang.Exception
java.lang.Exception
- if the class is not nominalpublic final double KBInformation() throws java.lang.Exception
java.lang.Exception
- if the class is not nominalpublic final double KBMeanInformation() throws java.lang.Exception
java.lang.Exception
- if the class is not nominalpublic final double KBRelativeInformation() throws java.lang.Exception
java.lang.Exception
- if the class is not nominalpublic final double SFPriorEntropy()
public final double SFMeanPriorEntropy()
public final double SFSchemeEntropy()
public final double SFMeanSchemeEntropy()
public final double SFEntropyGain()
public final double SFMeanEntropyGain()
public java.lang.String toCumulativeMarginDistributionString() throws java.lang.Exception
java.lang.Exception
- if the class attribute is nominalpublic java.lang.String toSummaryString()
public java.lang.String toSummaryString(boolean printComplexityStatistics)
printComplexityStatistics
- if true, complexity statistics are
returned as wellpublic java.lang.String toSummaryString(java.lang.String title, boolean printComplexityStatistics)
title
- the title for the statisticsprintComplexityStatistics
- if true, complexity statistics are
returned as wellpublic java.lang.String toMatrixString() throws java.lang.Exception
java.lang.Exception
- if the class is numericpublic java.lang.String toMatrixString(java.lang.String title) throws java.lang.Exception
title
- the title for the confusion matrixjava.lang.Exception
- if the class is numericpublic java.lang.String toClassDetailsString() throws java.lang.Exception
java.lang.Exception
public java.lang.String toClassDetailsString(java.lang.String title) throws java.lang.Exception
title
- the title to prepend the stats string withjava.lang.Exception
- if the class is not nominal.public double numTruePositives(int classIndex)
correctly classified positives
classIndex
- the index of the class to consider as "positive"public double truePositiveRate(int classIndex)
correctly classified positives ------------------------------ total positives
classIndex
- the index of the class to consider as "positive"public double numTrueNegatives(int classIndex)
correctly classified negatives
classIndex
- the index of the class to consider as "positive"public double trueNegativeRate(int classIndex)
correctly classified negatives ------------------------------ total negatives
classIndex
- the index of the class to consider as "positive"public double numFalsePositives(int classIndex)
incorrectly classified negatives
classIndex
- the index of the class to consider as "positive"public double falsePositiveRate(int classIndex)
incorrectly classified negatives -------------------------------- total negatives
classIndex
- the index of the class to consider as "positive"public double numFalseNegatives(int classIndex)
incorrectly classified positives
classIndex
- the index of the class to consider as "positive"public double falseNegativeRate(int classIndex)
incorrectly classified positives -------------------------------- total positives
classIndex
- the index of the class to consider as "positive"public double recall(int classIndex)
correctly classified positives ------------------------------ total positives
(Which is also the same as the truePositiveRate.)
classIndex
- the index of the class to consider as "positive"public double precision(int classIndex)
correctly classified positives ------------------------------ total predicted as positive
classIndex
- the index of the class to consider as "positive"public double fMeasure(int classIndex)
2 * recall * precision ---------------------- recall + precision
classIndex
- the index of the class to consider as "positive"public void setPriors(M5Instances train) throws java.lang.Exception
train
- the training instances used to determine
the prior probabilitiesjava.lang.Exception
- if the class attribute of the instances is not
setpublic void updatePriors(M5Instance instance) throws java.lang.Exception
instance
- the new training instance seenjava.lang.Exception
- if the class of the instance is not
setpublic boolean equals(java.lang.Object obj)
equals
in class java.lang.Object
obj
- the object to compare againstpublic static double roundNum(double num)
num
- number to round