hex.anovaglm

Class ANOVAGLMUtils

java.lang.Object

hex.anovaglm.ANOVAGLMUtils

public class ANOVAGLMUtils
extends java.lang.Object

Constructor Summary

Constructors

Constructor and Description
ANOVAGLMUtils()

Method Summary

Modifier and Type	Method and Description
static void	addIndividualPred(java.lang.String[] predNames, java.util.List<java.lang.String[]> predCombo)
static GLM[]	buildGLMBuilders(GLMModel.GLMParameters[] glmParams)
static GLMModel.GLMParameters[]	buildGLMParameters(water.fvec.Frame[] trainingFrames, ANOVAGLMModel.ANOVAGLMParameters parms)
static water.fvec.Frame	buildSpecificFrame(int[] predNums, water.fvec.Frame allCols, java.lang.String[][] transformedColNames, ANOVAGLMModel.ANOVAGLMParameters parms) This method is used to attach the weight/offset columns if they exist and the response columns, specific transformed columns to a training frames.
static water.fvec.Frame[]	buildTrainingFrames(water.Key<water.fvec.Frame> transformedCols, int numberOfModels, java.lang.String[][] transformedColNames, ANOVAGLMModel.ANOVAGLMParameters parms) This method will take the frame that contains transformed columns of predictor A, predictor B, interaction of predictor A and B and generate new training frames that contains the following columns: - transformed columns of predictor B, interaction of predictor A and B, response - transformed columns of predictor A, interaction of predictor A and B, response - transformed columns of predictor A, predictor B, response - transformed columns of predictor A, predictor B, interaction of predictor A and B, response The same logic applies if there are more than two individual predictors.
static int	calculatePredComboNumber(int numPred, int highestInteractionTerms) Given the number of individual predictors, the highest order of interaction terms allowed, this method will calculate the total number of predictors that will be used to build the full model.
static java.lang.String[]	combineAndFlat(java.lang.String[][] predictComboNames)
static GLMModel[]	extractGLMModels(GLM[] glmResults) Simple method to extract GLM Models from GLM ModelBuilders.
static java.lang.String[]	extractPredNames(DataInfo dinfo, int numOfPredictors) This method will extract the individual predictor names that will be used to build the GLM models.
static void	fillModelMetrics(ANOVAGLMModel aModel, GLMModel glmModel, water.fvec.Frame trainingFrame) I copied this method from Zuzana Olajcova to add model metrics of the full GLM model as the ANOVAModel model metrics
static int	findComboMatch(java.lang.String[][] predComboNames, int currIndex)
static double[]	generateGLMSS(GLMModel[] glmModels, GLMModel.GLMParameters.Family family) This method is used to generate Model SS for all models built except the full model.
static java.lang.String[]	generateModelNames(java.lang.String[][] predictComboNames)
static void	generateOneCombo(java.lang.String[] predNames, int numInteract, java.util.List<java.lang.String[]> predCombo)
static java.lang.String[][]	generatePredictorCombos(java.lang.String[] predNamesIndividual, int maxPredInt) In order to calculate Type III SS, we need the individual predictors and their interactions.
static void	generatePredictorNames(java.lang.String[][] predComboNames, java.lang.String[][] predictorNames, int[] predColumnStart, int[] degreeOfFreedom, DataInfo dinfo) This method aims to generate the column names of the final transformed frames.
static int[]	oneIndexOut(int currIndex, int indexRange)
static java.lang.String[]	predCombo(java.lang.String[] predNames, int[] predInd)
static void	removeFromDKV(water.fvec.Frame[] trainingFrames, int numFrame2Delete)
static java.lang.String[]	transformMultipleCols(water.fvec.Frame vec2Transform, java.lang.String[][] predComboNames, int currIndex, java.lang.String[][] predNames)
static java.lang.String[]	transformOneCol(water.fvec.Frame vec2Transform, java.lang.String vecName) perform data transformation described in AnovaGLMTutorial https://github.com/h2oai/h2o-3/issues/7561 section III.II on one predictor.
static java.lang.String[]	transformTwoCols(water.fvec.Frame vec2Transform, java.lang.String[] vecNames, java.lang.String[] lastComboNames) Generate frame transformation on two interacting columns.
static int	updateDOFColInfo(int predInd, java.lang.String[] predComboNames, int[] dof, int[] predCS, int offset)
static void	updateLaterBits(int[] predInd, int[] bounds, int index, int predNum)
static boolean	updatePredCombo(int[] predInd, int[] bounds)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ANOVAGLMUtils

public ANOVAGLMUtils()

Method Detail

extractPredNames

public static java.lang.String[] extractPredNames(DataInfo dinfo,
                                                  int numOfPredictors)

This method will extract the individual predictor names that will be used to build the GLM models.

Parameters:

dinfo: - DataInfo generated from dataset with all predictors

numOfPredictors: - number of individual predictors

generatePredictorCombos

public static java.lang.String[][] generatePredictorCombos(java.lang.String[] predNamesIndividual,
                                                           int maxPredInt)

In order to calculate Type III SS, we need the individual predictors and their interactions. For details, refer to ANOVAGLMTutorial https://github.com/h2oai/h2o-3/issues/7561 section IV

Parameters:

predNamesIndividual: - string containing individual predictor names

maxPredInt: - maximum number of predictors allowed in interaction term generation

Returns:

String array with double indices. First index refers to the predictor number, second index refers to the names of predictors involved in generating the interaction terms. For terms involving only individual predictors, there is only one predictor name.

addIndividualPred

public static void addIndividualPred(java.lang.String[] predNames,
                                     java.util.List<java.lang.String[]> predCombo)

generateOneCombo

public static void generateOneCombo(java.lang.String[] predNames,
                                    int numInteract,
                                    java.util.List<java.lang.String[]> predCombo)

updatePredCombo

public static boolean updatePredCombo(int[] predInd,
                                      int[] bounds)

updateLaterBits

public static void updateLaterBits(int[] predInd,
                                   int[] bounds,
                                   int index,
                                   int predNum)

predCombo

public static java.lang.String[] predCombo(java.lang.String[] predNames,
                                           int[] predInd)

calculatePredComboNumber

public static int calculatePredComboNumber(int numPred,
                                           int highestInteractionTerms)

Given the number of individual predictors, the highest order of interaction terms allowed, this method will calculate the total number of predictors that will be used to build the full model.

Parameters:

numPred: - number of individual predictors

highestInteractionTerms: - highest number of predictors allowed in generating interactions

Returns:

buildTrainingFrames

public static water.fvec.Frame[] buildTrainingFrames(water.Key<water.fvec.Frame> transformedCols,
                                                     int numberOfModels,
                                                     java.lang.String[][] transformedColNames,
                                                     ANOVAGLMModel.ANOVAGLMParameters parms)

This method will take the frame that contains transformed columns of predictor A, predictor B, interaction of predictor A and B and generate new training frames that contains the following columns: - transformed columns of predictor B, interaction of predictor A and B, response - transformed columns of predictor A, interaction of predictor A and B, response - transformed columns of predictor A, predictor B, response - transformed columns of predictor A, predictor B, interaction of predictor A and B, response The same logic applies if there are more than two individual predictors. You basically generate all the predictor combos. In building the model, you leave one predictor combo out.

Parameters:

transformedCols: - contains frame key of frame containing transformed columns of predictor A, predictor B, interaction of predictor A and B

numberOfModels: - number of models to build. For 2 factors, this should be 4.

Returns:

Array of training frames to build all the GLM models.

oneIndexOut

public static int[] oneIndexOut(int currIndex,
                                int indexRange)

fillModelMetrics

public static void fillModelMetrics(ANOVAGLMModel aModel,
                                    GLMModel glmModel,
                                    water.fvec.Frame trainingFrame)

I copied this method from Zuzana Olajcova to add model metrics of the full GLM model as the ANOVAModel model metrics

Parameters:

aModel -

glmModel -

trainingFrame -

extractGLMModels

public static GLMModel[] extractGLMModels(GLM[] glmResults)

Simple method to extract GLM Models from GLM ModelBuilders.

Parameters:

glmResults: - array of GLM ModelBuilders

removeFromDKV

public static void removeFromDKV(water.fvec.Frame[] trainingFrames,
                                 int numFrame2Delete)

buildSpecificFrame

public static water.fvec.Frame buildSpecificFrame(int[] predNums,
                                                  water.fvec.Frame allCols,
                                                  java.lang.String[][] transformedColNames,
                                                  ANOVAGLMModel.ANOVAGLMParameters parms)

This method is used to attach the weight/offset columns if they exist and the response columns, specific transformed columns to a training frames.

Parameters:

predNums: - number of all predictor combos

allCols: - Frame containing all transformed columns

transformedColNames: - transformed predictor combo arrays containing only predictor combos for a specific training dataset. Recall that models are built with one predictor combo left out. This is to generate that training frame with a specific predictor combo left out.

parms: - AnovaGLMParameters

Returns:

training frame excluding a specific set of predictor combos.

buildGLMParameters

public static GLMModel.GLMParameters[] buildGLMParameters(water.fvec.Frame[] trainingFrames,
                                                          ANOVAGLMModel.ANOVAGLMParameters parms)

generateGLMSS

public static double[] generateGLMSS(GLMModel[] glmModels,
                                     GLMModel.GLMParameters.Family family)

This method is used to generate Model SS for all models built except the full model. Refer to AnovaGLMTutorial https://github.com/h2oai/h2o-3/issues/7561 section V.

Parameters:

glmModels -

family -

Returns:

buildGLMBuilders

public static GLM[] buildGLMBuilders(GLMModel.GLMParameters[] glmParams)

generatePredictorNames

public static void generatePredictorNames(java.lang.String[][] predComboNames,
                                          java.lang.String[][] predictorNames,
                                          int[] predColumnStart,
                                          int[] degreeOfFreedom,
                                          DataInfo dinfo)

This method aims to generate the column names of the final transformed frames. This means that for single enum predictor "ABC" with domains "0" and "1", "2", the new column names will be ABC_0, ABC_1. For single numerical column, the same column name will be used. To generate the names of interaction columns, let's assume there are three predictors, R (2 levels), C (3 levels), S (3 levels). If the highest interaction terms allowed is 3, we will generate the following transformed names for the interaction columns: R0:C0, R0:C1, C0:S0, C0:S1, C1:S0, C1:S1, R0:C0:S0, R0:C0:S1, R0:C1:S0, R0:C1:S1

Parameters:

predComboNames: - string array containing all predictor combos and for each combo, all the predictor names involved in generating the interactions.

predictorNames: - string array containing all predictor names and for each combo, all the predictor names * involved in generating the interactions.

predColumnStart: - column of each predictor combo after the frame transformation.

degreeOfFreedom: - degree of freedom for each predictor combo

dinfo -

updateDOFColInfo

public static int updateDOFColInfo(int predInd,
                                   java.lang.String[] predComboNames,
                                   int[] dof,
                                   int[] predCS,
                                   int offset)

findComboMatch

public static int findComboMatch(java.lang.String[][] predComboNames,
                                 int currIndex)

combineAndFlat

public static java.lang.String[] combineAndFlat(java.lang.String[][] predictComboNames)

transformMultipleCols

public static java.lang.String[] transformMultipleCols(water.fvec.Frame vec2Transform,
                                                       java.lang.String[][] predComboNames,
                                                       int currIndex,
                                                       java.lang.String[][] predNames)

transformTwoCols

public static java.lang.String[] transformTwoCols(water.fvec.Frame vec2Transform,
                                                  java.lang.String[] vecNames,
                                                  java.lang.String[] lastComboNames)

Generate frame transformation on two interacting columns. Refer to AnovaGLMTutorial https://github.com/h2oai/h2o-3/issues/7561 sectinos III.II. and IV.

Parameters:

vec2Transform: - frame containing the two predictors to transform

vecNames: - name of the predictors

lastComboNames: - predictor combo names of the second vector if applicable. This is used to transform more than two predictors

Returns:

String containing the transformed column names.

transformOneCol

public static java.lang.String[] transformOneCol(water.fvec.Frame vec2Transform,
                                                 java.lang.String vecName)

perform data transformation described in AnovaGLMTutorial https://github.com/h2oai/h2o-3/issues/7561 section III.II on one predictor.

Parameters:

vec2Transform: - frame containing that one predictor to transform.

vecName: - name of predictor

generateModelNames

public static java.lang.String[] generateModelNames(java.lang.String[][] predictComboNames)