hex

Class ModelMetricsMultinomial

java.lang.Object

water.Iced<T>

water.Keyed<ModelMetrics>

hex.ModelMetrics

hex.ModelMetricsSupervised

hex.ModelMetricsMultinomial

All Implemented Interfaces:

java.io.Externalizable, java.io.Serializable, java.lang.Cloneable, Freezable<ModelMetrics>

Direct Known Subclasses:

ModelMetricsBinomialGLM.ModelMetricsMultinomialGLM, ModelMetricsMultinomialGeneric

public class ModelMetricsMultinomial
extends ModelMetricsSupervised

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ModelMetricsMultinomial.MetricBuilderMultinomial<T extends ModelMetricsMultinomial.MetricBuilderMultinomial<T>>

Nested classes/interfaces inherited from class hex.ModelMetricsSupervised

ModelMetricsSupervised.MetricBuilderSupervised<T extends ModelMetricsSupervised.MetricBuilderSupervised<T>>

Nested classes/interfaces inherited from class hex.ModelMetrics

ModelMetrics.MetricBuilder<T extends ModelMetrics.MetricBuilder<T>>

Field Summary

Fields

Modifier and Type	Field and Description
double	_aic
MultinomialAUC	_auc
ConfusionMatrix	_cm
float[]	_hit_ratios
double	_loglikelihood
double	_logloss
double	_mean_per_class_error

Fields inherited from class hex.ModelMetricsSupervised

_domain, _sigma

Fields inherited from class hex.ModelMetrics

_custom_metric, _description, _MSE, _nobs, _scoring_time

Fields inherited from class water.Keyed

_key

Constructor Summary

Constructors

Constructor and Description
ModelMetricsMultinomial(Model model, Frame frame, long nobs, double mse, java.lang.String[] domain, double sigma, ConfusionMatrix cm, float[] hr, double logloss, double loglikelihood, double aic, MultinomialAUC auc, CustomMetric customMetric)
ModelMetricsMultinomial(Model model, Frame frame, long nobs, double mse, java.lang.String[] domain, double sigma, ConfusionMatrix cm, float[] hr, double logloss, MultinomialAUC auc, CustomMetric customMetric)

Method Summary

Modifier and Type	Method and Description
double	aic()
double	auc()
double	aucpr()
ConfusionMatrix	cm()
static ModelMetricsMultinomial	getFromDKV(Model model, Frame frame)
static TwoDimTable	getHitRatioTable(float[] hits)
float[]	hr()
double	loglikelihood()
double	logloss()
static ModelMetricsMultinomial	make(Frame perClassProbs, Vec actualLabels, MultinomialAucType aucType) Build a Multinomial ModelMetrics object from per-class probabilities (in Frame preds - no labels!), from actual labels, and a given domain for all possible labels (maybe more than what's in labels)
static ModelMetricsMultinomial	make(Frame perClassProbs, Vec actualLabels, java.lang.String[] domain, MultinomialAucType aucType)
static ModelMetricsMultinomial	make(Frame perClassProbs, Vec actualLabels, Vec weights, java.lang.String[] domain, MultinomialAucType aucType) Build a Multinomial ModelMetrics object from per-class probabilities (in Frame preds - no labels!), from actual labels, and a given domain for all possible labels (maybe more than what's in labels)
double	mean_per_class_error()
double	pr_auc()
java.lang.String	toString()
static void	updateHits(double w, int iact, double[] ds, double[] hits)
static void	updateHits(double w, int iact, double[] ds, double[] hits, double[] priorClassDistribution)

Methods inherited from class hex.ModelMetricsSupervised

r2

Methods inherited from class hex.ModelMetrics

appendToStringMetrics, auc_obj, buildKey, buildKey, calcVarImp, calcVarImp, calcVarImp, calcVarImp, checksum_impl, custom_increasing, custom, deepCloneWithDifferentModelAndFrame, defaultModelMetrics, frame, getAllowedMetrics, getMetricFromModel, getMetricFromModelMetric, isForFrame, isForModel, model, mse, residual_degrees_of_freedom, rmse, setModelKey, sortModelsByMetric, sortModelsByMetric, withDescription, withModelAndFrame

Methods inherited from class water.Keyed

checksum_impl, checksum, checksum, getKey, makeSchema, readAll_impl, readAll, remove_impl, remove_impl, remove_self_key_impl, remove, remove, remove, remove, remove, remove, removeQuietly, writeAll_impl, writeAll

Methods inherited from class water.Iced

asBytes, clone, copyOver, frozenType, read, readExternal, readJSON, reloadFromBytes, toJsonBytes, toJsonString, write, writeExternal, writeJSON

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

_hit_ratios

public final float[] _hit_ratios

_cm

public final ConfusionMatrix _cm

_logloss

public final double _logloss

_loglikelihood

public final double _loglikelihood

_aic

public final double _aic

_mean_per_class_error

public double _mean_per_class_error

_auc

public MultinomialAUC _auc

Constructor Detail

ModelMetricsMultinomial

public ModelMetricsMultinomial(Model model,
                               Frame frame,
                               long nobs,
                               double mse,
                               java.lang.String[] domain,
                               double sigma,
                               ConfusionMatrix cm,
                               float[] hr,
                               double logloss,
                               double loglikelihood,
                               double aic,
                               MultinomialAUC auc,
                               CustomMetric customMetric)

ModelMetricsMultinomial

public ModelMetricsMultinomial(Model model,
                               Frame frame,
                               long nobs,
                               double mse,
                               java.lang.String[] domain,
                               double sigma,
                               ConfusionMatrix cm,
                               float[] hr,
                               double logloss,
                               MultinomialAUC auc,
                               CustomMetric customMetric)

Method Detail

toString

public java.lang.String toString()

Overrides:

toString in class ModelMetricsSupervised

logloss

public double logloss()

loglikelihood

public double loglikelihood()

aic

public double aic()

mean_per_class_error

public double mean_per_class_error()

cm

public ConfusionMatrix cm()

Overrides:

cm in class ModelMetrics

hr

public float[] hr()

Overrides:

hr in class ModelMetrics

auc

public double auc()

pr_auc

public double pr_auc()

aucpr

public double aucpr()

getFromDKV

public static ModelMetricsMultinomial getFromDKV(Model model,
                                                 Frame frame)

updateHits

public static void updateHits(double w,
                              int iact,
                              double[] ds,
                              double[] hits)

updateHits

public static void updateHits(double w,
                              int iact,
                              double[] ds,
                              double[] hits,
                              double[] priorClassDistribution)

getHitRatioTable

public static TwoDimTable getHitRatioTable(float[] hits)

make

public static ModelMetricsMultinomial make(Frame perClassProbs,
                                           Vec actualLabels,
                                           MultinomialAucType aucType)

Build a Multinomial ModelMetrics object from per-class probabilities (in Frame preds - no labels!), from actual labels, and a given domain for all possible labels (maybe more than what's in labels)

Parameters:

perClassProbs - Frame containing predicted per-class probabilities (and no predicted labels)

actualLabels - A Vec containing the actual labels (can be for fewer labels than what's in domain, since the predictions can be for a small subset of the data)

aucType - Type of multinomial AUC/AUCPR calculation, if NONE is set, multinomila AUC and AUCPR will not be computed

Returns:

ModelMetrics object

make

public static ModelMetricsMultinomial make(Frame perClassProbs,
                                           Vec actualLabels,
                                           java.lang.String[] domain,
                                           MultinomialAucType aucType)

make

public static ModelMetricsMultinomial make(Frame perClassProbs,
                                           Vec actualLabels,
                                           Vec weights,
                                           java.lang.String[] domain,
                                           MultinomialAucType aucType)

Build a Multinomial ModelMetrics object from per-class probabilities (in Frame preds - no labels!), from actual labels, and a given domain for all possible labels (maybe more than what's in labels)

Parameters:

perClassProbs - Frame containing predicted per-class probabilities (and no predicted labels)

actualLabels - A Vec containing the actual labels (can be for fewer labels than what's in domain, since the predictions can be for a small subset of the data)

weights - A Vec containing the observation weights.

domain - Ordered list of factor levels for which the probabilities are given (perClassProbs[i] are the per-observation probabilities for belonging to class domain[i])

aucType - Type of multinomial AUC/AUCPR calculation, if NONE is set, multinomila AUC and AUCPR will not be computed

Returns:

ModelMetrics object