water.rapids

Class PermutationVarImp

java.lang.Object

water.rapids.PermutationVarImp

public class PermutationVarImp
extends java.lang.Object

Permutation Variable (feature) importance measures the increase in the prediction error of the model after permuting the variables' values, which breaks the relationship between the variables and the true outcome. https://christophm.github.io/interpretable-ml-book/feature-importance.html

Calculate permutation variables importance, by shuffling randomly each variable of the training Frame, scoring the model with the newly created frame using One At a Time approach and Morris method; creating TwoDimTable with relative, scaled, and percentage value TwoDimTable with mean of the absolute value, and standard deviation of all features importance

Constructor Summary

Constructors

Constructor and Description
PermutationVarImp(Model model, Frame fr) Constructor that stores the model, frame

Method Summary

Modifier and Type	Method and Description
TwoDimTable	getPermutationVarImp(java.lang.String metric) Get PermutationVarImp
TwoDimTable	getPermutationVarImp(java.lang.String metric, long n_samples, java.lang.String[] features, long seed) Get PermutationVarImp
TwoDimTable	getRepeatedPermutationVarImp(java.lang.String metric, long n_samples, int n_repeats, java.lang.String[] features, long seed) Get Repeated Permutation Variable Importance

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PermutationVarImp

public PermutationVarImp(Model model,
                         Frame fr)

Constructor that stores the model, frame

Parameters:

model - trained model

fr - training frame

Method Detail

getPermutationVarImp

public TwoDimTable getPermutationVarImp(java.lang.String metric,
                                        long n_samples,
                                        java.lang.String[] features,
                                        long seed)

Get PermutationVarImp

Parameters:

metric - Metric to use to calculate the variable (feature) importance

n_samples - Number of samples to use to calculate the variable (feature) importance; Use -1 to use the whole frame

features - Features to evaluate

seed - Seed for random generator

Returns:

TwoDimTable of Permutation Feature Importance scores

getRepeatedPermutationVarImp

public TwoDimTable getRepeatedPermutationVarImp(java.lang.String metric,
                                                long n_samples,
                                                int n_repeats,
                                                java.lang.String[] features,
                                                long seed)

Get Repeated Permutation Variable Importance

Parameters:

metric - Metric to use to calculate the variable (feature) importance

n_samples - Number of samples to use to calculate the variable (feature) importance; Use -1 to use the whole frame

n_repeats - Number of repeats

features - Features to evaluate

seed - Seed for random generator

Returns:

TwoDimTable of Permutation Feature Importance scores

getPermutationVarImp

public TwoDimTable getPermutationVarImp(java.lang.String metric)

Get PermutationVarImp

Parameters:

metric - Metric to use to calculate the feature importance

Returns:

TwoDimTable of Permutation Feature Importance scores