hex.glm

Class GLM.GLMDriver

java.lang.Object

jsr166y.ForkJoinTask<java.lang.Void>

jsr166y.CountedCompleter

water.H2O.H2OCountedCompleter<hex.ModelBuilder.Driver>

hex.glm.GLM.GLMDriver

All Implemented Interfaces:

L_BFGS.ProgressMonitor, java.io.Serializable, java.lang.Cloneable, java.util.concurrent.Future<java.lang.Void>, water.Freezable<hex.ModelBuilder.Driver>

Enclosing class:

GLM

public final class GLM.GLMDriver
extends water.H2O.H2OCountedCompleter<hex.ModelBuilder.Driver>
implements L_BFGS.ProgressMonitor

Main loop of the glm algo.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
GLMDriver()

Method Summary

Modifier and Type	Method and Description
void	addWdataZiEtaOld2Response() Internal H2O method.
GLMModel	buildGammaGLM(water.fvec.Frame returnFrame, water.fvec.Frame constXYWeight, int[] devHvColIdx, long startRowIndex, long numRows, boolean computePValues) This method will generate a training frame according to HGLM doc, build a gamma GLM model with dispersion parameter set to 1 if enabled and calcluate the p-value if enabled.
double[]	calculate_all_beta(double[] start_delta, water.fvec.Frame augXZ, water.fvec.Frame augZW, int totRandCatLevels, double[][] cholRcopy) This method will estimate beta and ubeta using QR decomposition.
GLMTask.CalculateW4Data	calculateNewWAugXZ(water.fvec.Frame augXZ, int[] randCatLevels)
void	computeImpl()
protected GLMModel.Submodel	computeSubmodel(int i, double lambda, double nullDevTrain, double nullDevValid)
void	copyOver(double[][] cholR, double[][] cholRcopy)
GLMModel	fitDataDispersion(water.fvec.Frame returnFrame, int[] devHvColIdx, double[] VC1) This method estimates the init_sig_e by building a gamma GLM with response
void	genRID() Generate the regression influence diagnostic for gaussian and binomial families.
double[]	lfv_du_dv(GLMModel.GLMParameters.Family[] family, GLMModel.GLMParameters.Link[] link, double[] phi, double[] u)
water.fvec.Frame	makeZeroOrOneFrame(long rowNumber, int colNumber, int val, java.lang.String[] columnNames)
void	onCompletion(jsr166y.CountedCompleter caller)
boolean	onExceptionalCompletion(java.lang.Throwable t, jsr166y.CountedCompleter caller)
boolean	progress(double[] beta, double likelihood)
boolean	progress(double[] beta, OptimizationUtils.GradientInfo ginfo)
boolean	progressHGLMGLMMME(double sumDiff2, double sumeta2, int iteration, boolean atGLMMME, GLMModel fixedModel, GLMModel[] randModels, water.fvec.Frame glmmmeReturns, water.fvec.Frame hvDataOnly, double[] VC1, double[][] VC2, double[][] cholR, water.fvec.Frame augZ)
protected void	updateProgress(boolean canScore)
protected void	updateProgress(GLMModel fixedModel, GLMModel[] randModels, water.fvec.Frame glmmmeReturns, water.fvec.Frame hvDataOnly, double[] VC1, double[][] VC2, double sumDiff2, double convergence, boolean canScore, double[][] cholR, water.fvec.Frame augXZ)

Methods inherited from class water.H2O.H2OCountedCompleter

asBytes, clone, compute, compute1, currThrPriority, frozenType, icer, priority, read, readJSON, reloadFromBytes, write, writeJSON

Methods inherited from class jsr166y.CountedCompleter

__tryComplete, addToPendingCount, compareAndSetPendingCount, complete, exec, getCompleter, getPendingCount, getRawResult, setCompleter, setPendingCount, setRawResult, tryComplete

Methods inherited from class jsr166y.ForkJoinTask

adapt, adapt, adapt, cancel, compareAndSetForkJoinTaskTag, completeExceptionally, fork, get, get, get, getException, getForkJoinTaskTag, getPool, getQueuedTaskCount, getSurplusQueuedTaskCount, helpQuiesce, inForkJoinPool, invoke, invokeAll, invokeAll, invokeAll, isCancelled, isCompletedAbnormally, isCompletedNormally, isDone, join, peekNextLocalTask, pollNextLocalTask, pollTask, quietlyComplete, quietlyInvoke, quietlyJoin, reinitialize, setForkJoinTaskTag, tryUnfork

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GLMDriver

public GLMDriver()

Method Detail

makeZeroOrOneFrame

public water.fvec.Frame makeZeroOrOneFrame(long rowNumber,
                                           int colNumber,
                                           int val,
                                           java.lang.String[] columnNames)

calculateNewWAugXZ

public GLMTask.CalculateW4Data calculateNewWAugXZ(water.fvec.Frame augXZ,
                                                  int[] randCatLevels)

copyOver

public void copyOver(double[][] cholR,
                     double[][] cholRcopy)

calculate_all_beta

public double[] calculate_all_beta(double[] start_delta,
                                   water.fvec.Frame augXZ,
                                   water.fvec.Frame augZW,
                                   int totRandCatLevels,
                                   double[][] cholRcopy)

This method will estimate beta and ubeta using QR decomposition. Refer to HGLM documentation appendix step 2.

Parameters:

start_delta -

augXZ -

augZW -

totRandCatLevels -

cholRcopy -

Returns:

fitDataDispersion

public GLMModel fitDataDispersion(water.fvec.Frame returnFrame,
                                  int[] devHvColIdx,
                                  double[] VC1)

This method estimates the init_sig_e by building a gamma GLM with response

Parameters:

returnFrame -

devHvColIdx -

VC1 -

Returns:

buildGammaGLM

public GLMModel buildGammaGLM(water.fvec.Frame returnFrame,
                              water.fvec.Frame constXYWeight,
                              int[] devHvColIdx,
                              long startRowIndex,
                              long numRows,
                              boolean computePValues)

This method will generate a training frame according to HGLM doc, build a gamma GLM model with dispersion parameter set to 1 if enabled and calcluate the p-value if enabled. It will return the GLM model.

lfv_du_dv

public double[] lfv_du_dv(GLMModel.GLMParameters.Family[] family,
                          GLMModel.GLMParameters.Link[] link,
                          double[] phi,
                          double[] u)

computeSubmodel

protected GLMModel.Submodel computeSubmodel(int i,
                                            double lambda,
                                            double nullDevTrain,
                                            double nullDevValid)

computeImpl

public void computeImpl()

genRID

public void genRID()

Generate the regression influence diagnostic for gaussian and binomial families. It takes the following steps: 1. generate the inverse of gram matrix; 2. generate sum of all columns of gram matrix; 3. task is called to generate the hat matrix equation 2 or equation 6 and the diagnostic in equation 3 or 7. Note that redundant predictors are excluded.

addWdataZiEtaOld2Response

public void addWdataZiEtaOld2Response()

Internal H2O method. Do not use. This method will add three more columns to _dinfo.response for easy access later - column 1: wdata, calculated weight for fixed columns/effects - column 2: zi, intermediate values - column 3: eta = X*beta, intermediate values

onCompletion

public void onCompletion(jsr166y.CountedCompleter caller)

onExceptionalCompletion

public boolean onExceptionalCompletion(java.lang.Throwable t,
                                       jsr166y.CountedCompleter caller)

progress

public boolean progress(double[] beta,
                        OptimizationUtils.GradientInfo ginfo)

Specified by:

progress in interface L_BFGS.ProgressMonitor

progressHGLMGLMMME

public boolean progressHGLMGLMMME(double sumDiff2,
                                  double sumeta2,
                                  int iteration,
                                  boolean atGLMMME,
                                  GLMModel fixedModel,
                                  GLMModel[] randModels,
                                  water.fvec.Frame glmmmeReturns,
                                  water.fvec.Frame hvDataOnly,
                                  double[] VC1,
                                  double[][] VC2,
                                  double[][] cholR,
                                  water.fvec.Frame augZ)

progress

public boolean progress(double[] beta,
                        double likelihood)

updateProgress

protected void updateProgress(GLMModel fixedModel,
                              GLMModel[] randModels,
                              water.fvec.Frame glmmmeReturns,
                              water.fvec.Frame hvDataOnly,
                              double[] VC1,
                              double[][] VC2,
                              double sumDiff2,
                              double convergence,
                              boolean canScore,
                              double[][] cholR,
                              water.fvec.Frame augXZ)

updateProgress

protected void updateProgress(boolean canScore)