hex.glm

Class GLMModel

java.lang.Object

water.Iced<T>

water.Keyed<T>

water.Lockable<M>

hex.Model<GLMModel,GLMModel.GLMParameters,GLMModel.GLMOutput>

hex.glm.GLMModel

All Implemented Interfaces:

hex.Model.Contributions, java.io.Externalizable, java.io.Serializable, java.lang.Cloneable, water.api.StreamWriter, water.Freezable<GLMModel>

public class GLMModel
extends hex.Model<GLMModel,GLMModel.GLMParameters,GLMModel.GLMOutput>
implements hex.Model.Contributions

Created by tomasnykodym on 8/27/14.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	GLMModel.GLMOutput
static class	GLMModel.GLMParameters
static class	GLMModel.GLMWeights
static class	GLMModel.GLMWeightsFun
static class	GLMModel.RegularizationPath
static class	GLMModel.Submodel

Nested classes/interfaces inherited from class hex.Model

hex.Model.AdaptFrameParameters, hex.Model.BigScore, hex.Model.BigScoreChunkPredict, hex.Model.BigScorePredict, hex.Model.Contributions, hex.Model.DeepFeatures, hex.Model.ExemplarMembers, hex.Model.FeatureFrequencies, hex.Model.GetMostImportantFeatures, hex.Model.GetNTrees, hex.Model.GLRMArchetypes, hex.Model.GridSortBy, hex.Model.H2OModelDescriptor, hex.Model.InteractionBuilder, hex.Model.InteractionPair, hex.Model.InteractionSpec, hex.Model.JavaModelStreamWriter, hex.Model.JavaScoringOptions, hex.Model.LeafNodeAssignment, hex.Model.Output, hex.Model.Parameters, hex.Model.PredictScoreResult, hex.Model.RowToTreeAssignment, hex.Model.StagedPredictions, hex.Model.UpdateAuxTreeWeights

Nested classes/interfaces inherited from interface hex.Model.Contributions

hex.Model.Contributions.ContributionsOptions, hex.Model.Contributions.ContributionsOutputFormat

Field Summary

Fields

Modifier and Type	Field and Description
double[]	_betaCndCheckpoint
static double	_EPS
boolean	_finalScoring
double	_lambda_max
long	_nobs
long	_nullDOF
static double	_OneOEPS
double[]	_ymu
double	_ySigma

Fields inherited from class hex.Model

_dist, _input_parms, _output, _parms, _toDelete, _warnings, _warningsP, EVAL_AUTO_PARAMS_ENABLED, evalAutoParamsEnabled, scoringInfo

Fields inherited from class water.Lockable

_lockers

Fields inherited from class water.Keyed

_key

Constructor Summary

Constructors

Constructor and Description
GLMModel(water.Key selfKey, GLMModel.GLMParameters parms, GLM job, double[] ymu, double ySigma, double lambda_max, long nobs)

Method Summary

Modifier and Type	Method and Description
void	addScoringInfo(GLMModel.GLMParameters parms, int nclasses, long currTime, int iter)
GLMModel	addSubmodel(int idx, GLMModel.Submodel sm)
protected double[]	beta_internal()
double[]	beta_std(double lambda)
double[]	beta()
double[]	beta(double lambda)
java.lang.String[]	buildVariableInflationFactors(water.fvec.Frame train, DataInfo dinfo) This method will calculate the variable inflation factor of each numerical predictor using the following procedure: 1.
java.lang.String[]	buildVariableInflationFactors(GLMModel.GLMParameters parms, java.lang.String[] validPredictors, java.lang.String[] predictorNames)
long	checksum_impl()
java.util.HashMap<java.lang.String,java.lang.Double>	coefficients() get beta coefficients in a map indexed by name
java.util.HashMap<java.lang.String,java.lang.Double>	coefficients(boolean standardized)
protected GLMModel	deepClone(water.Key<GLMModel> result)
double	deviance(double w, double y, double f)
DataInfo	dinfo()
water.util.TwoDimTable	generateSummary(water.Key train, int iter) Re-do the TwoDim table generation with updated model.
water.util.TwoDimTable	generateSummaryHGLM(water.Key train, int iter) This one is for HGLM
GLMMojoWriter	getMojo()
GLMModel.RegularizationPath	getRegularizationPath()
water.fvec.Frame	getRIDFrame()
hex.ScoringInfo[]	getScoringInfo()
boolean	haveMojo()
boolean	havePojo()
void	initActualParamValues()
protected boolean	isFeatureUsedInPredict(int featureIdx)
double	likelihood(double w, double y, double[] f)
hex.ModelMetrics.MetricBuilder	makeMetricBuilder(java.lang.String[] domain)
java.lang.String[]	makeScoringNames()
java.lang.String[]	names()
protected boolean	needsPostProcess()
protected hex.Model.PredictScoreResult	predictScoreImpl(water.fvec.Frame fr, water.fvec.Frame adaptFrm, java.lang.String destination_key, water.Job j, boolean computeMetrics, water.udf.CFuncRef customMetricFunc) Score an already adapted frame.
protected water.Keyed	readAll_impl(water.AutoBuffer ab, water.Futures fs)
protected water.Futures	remove_impl(water.Futures fs, boolean cascade)
double	score(double[] data)
protected double[]	score0(double[] data, double[] preds)
protected double[]	score0(double[] data, double[] preds, double o)
water.fvec.Frame	scoreContributions(water.fvec.Frame frame, water.Key<water.fvec.Frame> destination_key, water.Job<water.fvec.Frame> j, hex.Model.Contributions.ContributionsOptions options, water.fvec.Frame backgroundFrame)
hex.ScoreKeeper[]	scoreKeepers()
protected hex.ModelMetrics.MetricBuilder	scoreMetrics(water.fvec.Frame adaptFrm) Score an already adapted frame.
protected java.lang.String[][]	scoringDomains()
void	setDispersion(double dispersion, boolean dispersionEstimated)
void	setVcov(double[][] inv)
void	setZValues(double[] zValues, double dispersion, boolean dispersionEstimated)
protected boolean	toJavaCheckTooBig()
protected water.util.SBPrintStream	toJavaInit(water.util.SBPrintStream sb, water.codegen.CodeGeneratorPipeline fileCtx)
protected void	toJavaPredictBody(water.util.SBPrintStream body, water.codegen.CodeGeneratorPipeline classCtx, water.codegen.CodeGeneratorPipeline fileCtx, boolean verboseCode)
void	update(double[] beta, double[] ubeta, double devianceTrain, double devianceTest, int iter)
void	update(double[] beta, double devianceTrain, double devianceTest, int iter)
GLMModel	updateSubmodel(int idx, GLMModel.Submodel sm)
protected water.AutoBuffer	writeAll_impl(water.AutoBuffer ab)

Methods inherited from class hex.Model

adaptFrameForScore, adaptTestForJavaScoring, adaptTestForTrain, adaptTestForTrain, adaptTestForTrain, addMetrics, addModelMetrics, addWarning, aic, auc, AUCPR, classification_error, compareTo, computeDeviances, containsResponse, data, defaultThreshold, defaultThreshold, deleteCrossValidationFoldAssignment, deleteCrossValidationModels, deleteCrossValidationPreds, deviance, evaluateAutoModelParameters, exportBinaryModel, exportMojo, fetchAll, fillScoringInfo, getDefaultGridSortBy, getGenModelEncoding, getPojoInterfaces, getToEigenVec, importBinaryModel, isDistributionHuber, isFeatureUsedInPredict, isGeneric, isSupervised, last_scored, lift_top_group, logloss, loss, mae, makeAdaptFrameParameters, makeBigScoreTask, makeInteraction, makeInteractions, makeInteractions, makePojoWriter, makeSchema, makeScoringDomains, makeScoringNames, mean_per_class_error, modelDescriptor, mse, postProcessPredictions, r2, resetThreshold, result, rmsle, score, score, score, score, score, score, score, score0, score0, score0PostProcessSupervised, scoring_history, setInputParms, setupBigScorePredict, testJavaScoring, testJavaScoring, testJavaScoring, testJavaScoring, testJavaScoring, toJava, toJava, toJava, toJavaAlgo, toJavaModelClassName, toJavaTransform, toJavaUUID, toMojo, toMojo, toString, transform, uploadBinaryModel, writeTo

Methods inherited from class water.Lockable

delete_and_lock, delete_and_lock, delete_and_lock, delete_and_lock, delete, delete, delete, delete, read_lock, read_lock, read_lock, unlock_all, unlock, unlock, unlock, unlock, update, update, update, write_lock_to_read_lock, write_lock, write_lock, write_lock

Methods inherited from class water.Keyed

checksum_impl, checksum, checksum, getKey, readAll, remove_impl, remove_self_key_impl, remove, remove, remove, remove, remove, remove, removeQuietly, writeAll

Methods inherited from class water.Iced

asBytes, clone, copyOver, frozenType, read, readExternal, readJSON, reloadFromBytes, toJsonBytes, toJsonString, write, writeExternal, writeJSON

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface hex.Model.Contributions

composeScoreContributionTaskMetadata, scoreContributions, scoreContributions, scoreContributions, scoreContributionsWorkEstimate

Field Detail

_EPS

public static final double _EPS

See Also:

Constant Field Values

_OneOEPS

public static final double _OneOEPS

See Also:

Constant Field Values

_lambda_max

public final double _lambda_max

_ymu

public final double[] _ymu

_nullDOF

public final long _nullDOF

_ySigma

public final double _ySigma

_nobs

public final long _nobs

_betaCndCheckpoint

public double[] _betaCndCheckpoint

_finalScoring

public boolean _finalScoring

Constructor Detail

GLMModel

public GLMModel(water.Key selfKey,
                GLMModel.GLMParameters parms,
                GLM job,
                double[] ymu,
                double ySigma,
                double lambda_max,
                long nobs)

Method Detail

getRIDFrame

public water.fvec.Frame getRIDFrame()

initActualParamValues

public void initActualParamValues()

Overrides:

initActualParamValues in class hex.Model<GLMModel,GLMModel.GLMParameters,GLMModel.GLMOutput>

scoreKeepers

public hex.ScoreKeeper[] scoreKeepers()

getScoringInfo

public hex.ScoringInfo[] getScoringInfo()

addScoringInfo

public void addScoringInfo(GLMModel.GLMParameters parms,
                           int nclasses,
                           long currTime,
                           int iter)

setVcov

public void setVcov(double[][] inv)

buildVariableInflationFactors

public java.lang.String[] buildVariableInflationFactors(water.fvec.Frame train,
                                                        DataInfo dinfo)

This method will calculate the variable inflation factor of each numerical predictor using the following procedure: 1. For each numerical predictor, choose it as the response variable and leave all the other predictors as the predictors; 2. Build a GLM model and obtained the R2 of the model; 3. the variable inflation factor for that predictor is calculated as 1.0/(1.0-R2). All the variable inflation factors for all valid predictors are saved in a double array. No variable inflation factors are generated for non-numerical predictors.

buildVariableInflationFactors

public java.lang.String[] buildVariableInflationFactors(GLMModel.GLMParameters parms,
                                                        java.lang.String[] validPredictors,
                                                        java.lang.String[] predictorNames)

scoreContributions

public water.fvec.Frame scoreContributions(water.fvec.Frame frame,
                                           water.Key<water.fvec.Frame> destination_key,
                                           water.Job<water.fvec.Frame> j,
                                           hex.Model.Contributions.ContributionsOptions options,
                                           water.fvec.Frame backgroundFrame)

Specified by:

scoreContributions in interface hex.Model.Contributions

getRegularizationPath

public GLMModel.RegularizationPath getRegularizationPath()

toJavaCheckTooBig

protected boolean toJavaCheckTooBig()

Overrides:

toJavaCheckTooBig in class hex.Model<GLMModel,GLMModel.GLMParameters,GLMModel.GLMOutput>

dinfo

public DataInfo dinfo()

makeMetricBuilder

public hex.ModelMetrics.MetricBuilder makeMetricBuilder(java.lang.String[] domain)

Specified by:

makeMetricBuilder in class hex.Model<GLMModel,GLMModel.GLMParameters,GLMModel.GLMOutput>

beta_internal

protected double[] beta_internal()

beta

public double[] beta()

beta

public double[] beta(double lambda)

beta_std

public double[] beta_std(double lambda)

names

public java.lang.String[] names()

deviance

public double deviance(double w,
                       double y,
                       double f)

Overrides:

deviance in class hex.Model<GLMModel,GLMModel.GLMParameters,GLMModel.GLMOutput>

likelihood

public double likelihood(double w,
                         double y,
                         double[] f)

Overrides:

likelihood in class hex.Model<GLMModel,GLMModel.GLMParameters,GLMModel.GLMOutput>

addSubmodel

public GLMModel addSubmodel(int idx,
                            GLMModel.Submodel sm)

updateSubmodel

public GLMModel updateSubmodel(int idx,
                               GLMModel.Submodel sm)

update

public void update(double[] beta,
                   double devianceTrain,
                   double devianceTest,
                   int iter)

update

public void update(double[] beta,
                   double[] ubeta,
                   double devianceTrain,
                   double devianceTest,
                   int iter)

deepClone

protected GLMModel deepClone(water.Key<GLMModel> result)

scoringDomains

protected java.lang.String[][] scoringDomains()

Overrides:

scoringDomains in class hex.Model<GLMModel,GLMModel.GLMParameters,GLMModel.GLMOutput>

setZValues

public void setZValues(double[] zValues,
                       double dispersion,
                       boolean dispersionEstimated)

setDispersion

public void setDispersion(double dispersion,
                          boolean dispersionEstimated)

coefficients

public java.util.HashMap<java.lang.String,java.lang.Double> coefficients()

get beta coefficients in a map indexed by name

Returns:

the estimated coefficients

coefficients

public java.util.HashMap<java.lang.String,java.lang.Double> coefficients(boolean standardized)

generateSummary

public water.util.TwoDimTable generateSummary(water.Key train,
                                              int iter)

Re-do the TwoDim table generation with updated model.

generateSummaryHGLM

public water.util.TwoDimTable generateSummaryHGLM(water.Key train,
                                                  int iter)

This one is for HGLM

Parameters:

train -

iter -

Returns:

checksum_impl

public long checksum_impl()

Overrides:

checksum_impl in class hex.Model<GLMModel,GLMModel.GLMParameters,GLMModel.GLMOutput>

score0

protected double[] score0(double[] data,
                          double[] preds)

Specified by:

score0 in class hex.Model<GLMModel,GLMModel.GLMParameters,GLMModel.GLMOutput>

score0

protected double[] score0(double[] data,
                          double[] preds,
                          double o)

Overrides:

score0 in class hex.Model<GLMModel,GLMModel.GLMParameters,GLMModel.GLMOutput>

needsPostProcess

protected boolean needsPostProcess()

Overrides:

needsPostProcess in class hex.Model<GLMModel,GLMModel.GLMParameters,GLMModel.GLMOutput>

score

public double score(double[] data)

Overrides:

score in class hex.Model<GLMModel,GLMModel.GLMParameters,GLMModel.GLMOutput>

toJavaPredictBody

protected void toJavaPredictBody(water.util.SBPrintStream body,
                                 water.codegen.CodeGeneratorPipeline classCtx,
                                 water.codegen.CodeGeneratorPipeline fileCtx,
                                 boolean verboseCode)

Overrides:

toJavaPredictBody in class hex.Model<GLMModel,GLMModel.GLMParameters,GLMModel.GLMOutput>

toJavaInit

protected water.util.SBPrintStream toJavaInit(water.util.SBPrintStream sb,
                                              water.codegen.CodeGeneratorPipeline fileCtx)

Overrides:

toJavaInit in class hex.Model<GLMModel,GLMModel.GLMParameters,GLMModel.GLMOutput>

predictScoreImpl

protected hex.Model.PredictScoreResult predictScoreImpl(water.fvec.Frame fr,
                                                        water.fvec.Frame adaptFrm,
                                                        java.lang.String destination_key,
                                                        water.Job j,
                                                        boolean computeMetrics,
                                                        water.udf.CFuncRef customMetricFunc)

Score an already adapted frame. Returns a new Frame with new result vectors, all in the DKV. Caller responsible for deleting. Input is already adapted to the Model's domain, so the output is also. Also computes the metrics for this frame.

Overrides:

predictScoreImpl in class hex.Model<GLMModel,GLMModel.GLMParameters,GLMModel.GLMOutput>

Parameters:

adaptFrm - Already adapted frame

computeMetrics -

Returns:

A Frame containing the prediction column, and class distribution

makeScoringNames

public java.lang.String[] makeScoringNames()

Overrides:

makeScoringNames in class hex.Model<GLMModel,GLMModel.GLMParameters,GLMModel.GLMOutput>

scoreMetrics

protected hex.ModelMetrics.MetricBuilder scoreMetrics(water.fvec.Frame adaptFrm)

Score an already adapted frame. Returns a MetricBuilder that can be used to make a model metrics.

Overrides:

scoreMetrics in class hex.Model<GLMModel,GLMModel.GLMParameters,GLMModel.GLMOutput>

Parameters:

adaptFrm - Already adapted frame

Returns:

MetricBuilder

haveMojo

public boolean haveMojo()

Overrides:

haveMojo in class hex.Model<GLMModel,GLMModel.GLMParameters,GLMModel.GLMOutput>

havePojo

public boolean havePojo()

Overrides:

havePojo in class hex.Model<GLMModel,GLMModel.GLMParameters,GLMModel.GLMOutput>

getMojo

public GLMMojoWriter getMojo()

Overrides:

getMojo in class hex.Model<GLMModel,GLMModel.GLMParameters,GLMModel.GLMOutput>

isFeatureUsedInPredict

protected boolean isFeatureUsedInPredict(int featureIdx)

Overrides:

isFeatureUsedInPredict in class hex.Model<GLMModel,GLMModel.GLMParameters,GLMModel.GLMOutput>

remove_impl

protected water.Futures remove_impl(water.Futures fs,
                                    boolean cascade)

Overrides:

remove_impl in class hex.Model<GLMModel,GLMModel.GLMParameters,GLMModel.GLMOutput>

writeAll_impl

protected water.AutoBuffer writeAll_impl(water.AutoBuffer ab)

Overrides:

writeAll_impl in class hex.Model<GLMModel,GLMModel.GLMParameters,GLMModel.GLMOutput>

readAll_impl

protected water.Keyed readAll_impl(water.AutoBuffer ab,
                                   water.Futures fs)

Overrides:

readAll_impl in class hex.Model<GLMModel,GLMModel.GLMParameters,GLMModel.GLMOutput>