hex.grid

Class Grid<MP extends Model.Parameters>

java.lang.Object

water.Iced<T>

water.Keyed<T>

water.Lockable<Grid<MP>>

hex.grid.Grid<MP>

Type Parameters:

MP - type of model build parameters

All Implemented Interfaces:

Recoverable<Grid<MP>>, ModelContainer<Model>, java.io.Externalizable, java.io.Serializable, java.lang.Cloneable, Freezable<Grid<MP>>

public class Grid<MP extends Model.Parameters>
extends Lockable<Grid<MP>>
implements ModelContainer<Model>, Recoverable<Grid<MP>>

A Grid of Models representing result of hyper-parameter space exploration. Lazily filled in, this object represents the potentially infinite variety of hyperparameters of a given model & dataset.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Grid.SearchFailure<MP extends Model.Parameters> Failure that occurred during hyperspace exploration.

Field Summary

Fields

Modifier and Type	Field and Description
static Grid	GRID_PROTO Publicly available Grid prototype - used by REST API.

Fields inherited from class water.Lockable

_lockers

Fields inherited from class water.Keyed

_key

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	Grid(Key key, HyperSpaceWalker<MP,?> walker, int parallelism)
protected	Grid(Key key, MP params, java.lang.String[] hyperNames, java.util.Map<java.lang.String,java.lang.Object[]> hyperParams, HyperSpaceSearchCriteria searchCriteria, PojoUtils.FieldNaming fieldNaming, int parallelism) Construct a new grid object to store results of grid search.

Method Summary

Modifier and Type	Method and Description
protected long	checksum_impl() High-quality 64-bit checksum of the content of the object.
protected void	clearNonRelatedFailures() Removes failures found while walking the hyperspace related to models not present in Grid.
int	countTotalFailures()
TwoDimTable	createScoringHistoryTable()
TwoDimTable	createSummaryTable(Key<Model>[] model_ids, java.lang.String sort_by, boolean decreasing)
java.util.List<java.lang.String>	exportBinary(java.lang.String gridExportDir, boolean exportModels, ModelExportOption... options) Exports this Grid in a binary format using AutoBuffer.
java.util.Set<Key<?>>	getDependentKeys()
Grid.SearchFailure	getFailures() Returns all failures currently listed in this Grid instance, including failures related to models not present in the grid that failed during the last run.
java.lang.String[]	getHyperNames() Returns an array of used hyper parameters names.
java.util.Map<java.lang.String,java.lang.Object[]>	getHyperParams()
java.lang.Object[]	getHyperValues(MP parms) Return value of hyper parameters used for this grid search.
Model	getModel(MP params) Returns model for given combination of model parameters or null if the model does not exist.
int	getModelCount() Returns number of models in this grid.
Key<Model>	getModelKey(MP params)
Key<Model>[]	getModelKeys() Returns keys of all models included in this object.
java.lang.String	getModelName() Returns name of model included in this object.
Model[]	getModels() Return all models included in this grid object.
int	getParallelism()
MP	getParams()
ScoringInfo[]	getScoringInfos()
HyperSpaceSearchCriteria	getSearchCriteria()
Frame	getTrainingFrame() Returns the data frame used to train all these models.
static Grid	importBinary(java.lang.String gridPath, boolean loadReferences)
java.lang.Class<KeyV3.GridKeyV3>	makeSchema()
protected Keyed	readAll_impl(AutoBuffer ab, Futures fs)
protected Futures	remove_impl(Futures fs, boolean cascade) Override to remove subparts, but not self, of composite Keyed objects.
void	setScoringInfos(ScoringInfo[] scoring_infos)
void	update(java.util.Map<java.lang.String,java.lang.Object[]> hyperParams, HyperSpaceSearchCriteria searchCriteria, int parallelism)
protected AutoBuffer	writeAll_impl(AutoBuffer ab) Write out K/V pairs

Methods inherited from class water.Lockable

delete_and_lock, delete_and_lock, delete_and_lock, delete_and_lock, delete, delete, delete, delete, read_lock, read_lock, read_lock, unlock_all, unlock, unlock, unlock, unlock, update, update, update, write_lock_to_read_lock, write_lock, write_lock, write_lock

Methods inherited from class water.Keyed

checksum_impl, checksum, checksum, getKey, readAll, remove_impl, remove_self_key_impl, remove, remove, remove, remove, remove, remove, removeQuietly, writeAll

Methods inherited from class water.Iced

asBytes, clone, copyOver, frozenType, read, readExternal, readJSON, reloadFromBytes, toJsonBytes, toJsonString, write, writeExternal, writeJSON

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface hex.faulttolerance.Recoverable

getKey

Field Detail

GRID_PROTO

public static final Grid GRID_PROTO

Publicly available Grid prototype - used by REST API.

See Also:

GridSchemaV99

Constructor Detail

Grid

protected Grid(Key key,
               MP params,
               java.lang.String[] hyperNames,
               java.util.Map<java.lang.String,java.lang.Object[]> hyperParams,
               HyperSpaceSearchCriteria searchCriteria,
               PojoUtils.FieldNaming fieldNaming,
               int parallelism)

Construct a new grid object to store results of grid search.

Parameters:

key - reference to this object

params - initial parameters used by grid search

hyperNames - names of used hyper parameters

Grid

protected Grid(Key key,
               HyperSpaceWalker<MP,?> walker,
               int parallelism)

Method Detail

update

public void update(java.util.Map<java.lang.String,java.lang.Object[]> hyperParams,
                   HyperSpaceSearchCriteria searchCriteria,
                   int parallelism)

getHyperParams

public java.util.Map<java.lang.String,java.lang.Object[]> getHyperParams()

getSearchCriteria

public HyperSpaceSearchCriteria getSearchCriteria()

getParallelism

public int getParallelism()

getModelName

public java.lang.String getModelName()

Returns name of model included in this object. Note: only sensible for Grids which search over a single class of Models.

Returns:

name of model (for example, "DRF", "GBM")

getScoringInfos

public ScoringInfo[] getScoringInfos()

setScoringInfos

public void setScoringInfos(ScoringInfo[] scoring_infos)

getTrainingFrame

public Frame getTrainingFrame()

Returns the data frame used to train all these models.

All models are trained on the same data frame, but might be validated on multiple different frames.

Returns:

training frame shared among all models

getModel

public Model getModel(MP params)

Returns model for given combination of model parameters or null if the model does not exist.

Parameters:

params - parameters of the model

Returns:

A model run with these parameters, or null if the model does not exist.

getModelKey

public Key<Model> getModelKey(MP params)

getModelKeys

public Key<Model>[] getModelKeys()

Returns keys of all models included in this object.

Specified by:

getModelKeys in interface ModelContainer<Model>

Returns:

list of model keys sorted lexically

getModels

public Model[] getModels()

Return all models included in this grid object.

Specified by:

getModels in interface ModelContainer<Model>

Returns:

all models in this grid

getModelCount

public int getModelCount()

Returns number of models in this grid.

Specified by:

getModelCount in interface ModelContainer<Model>

getFailures

public Grid.SearchFailure getFailures()

Returns all failures currently listed in this Grid instance, including failures related to models not present in the grid that failed during the last run.

Returns:

An instance of Grid.SearchFailure with all failures currently linked to this Grid. An empty Grid.SearchFailure instance is returned if there are no failures listed.

countTotalFailures

public int countTotalFailures()

clearNonRelatedFailures

protected void clearNonRelatedFailures()

Removes failures found while walking the hyperspace related to models not present in Grid.

getHyperValues

public java.lang.Object[] getHyperValues(MP parms)

Return value of hyper parameters used for this grid search.

Parameters:

parms - model parameters

Returns:

values of hyper parameters used by grid search producing this grid object.

getHyperNames

public java.lang.String[] getHyperNames()

Returns an array of used hyper parameters names.

Returns:

names of hyper parameters used in this hyper search

remove_impl

protected Futures remove_impl(Futures fs,
                              boolean cascade)

Description copied from class: Keyed

Override to remove subparts, but not self, of composite Keyed objects. Examples include Vec (removing associated Chunk keys) and Frame (removing associated Vec keys.)

Overrides:

remove_impl in class Keyed<Grid<MP extends Model.Parameters>>

writeAll_impl

protected AutoBuffer writeAll_impl(AutoBuffer ab)

Write out K/V pairs

Overrides:

writeAll_impl in class Keyed<Grid<MP extends Model.Parameters>>

readAll_impl

protected Keyed readAll_impl(AutoBuffer ab,
                             Futures fs)

Overrides:

readAll_impl in class Keyed<Grid<MP extends Model.Parameters>>

checksum_impl

protected long checksum_impl()

Description copied from class: Keyed

High-quality 64-bit checksum of the content of the object. Similar to hashcode(), but a long to reduce the chance of hash clashes. For composite objects this should be defined using the subcomponents' checksums (or hashcodes if not available). If two Keyed objects have the same checksum() there should be a 1 - 1/2^64 chance that they are the same object by value.

Overrides:

checksum_impl in class Keyed<Grid<MP extends Model.Parameters>>

makeSchema

public java.lang.Class<KeyV3.GridKeyV3> makeSchema()

Overrides:

makeSchema in class Keyed<Grid<MP extends Model.Parameters>>

createSummaryTable

public TwoDimTable createSummaryTable(Key<Model>[] model_ids,
                                      java.lang.String sort_by,
                                      boolean decreasing)

createScoringHistoryTable

public TwoDimTable createScoringHistoryTable()

exportBinary

public java.util.List<java.lang.String> exportBinary(java.lang.String gridExportDir,
                                                     boolean exportModels,
                                                     ModelExportOption... options)

Exports this Grid in a binary format using AutoBuffer. Related models are not saved.

Specified by:

exportBinary in interface Recoverable<Grid<MP extends Model.Parameters>>

Parameters:

gridExportDir - Full path to the folder this Grid should be saved to

Returns:

Path of the file written

importBinary

public static Grid importBinary(java.lang.String gridPath,
                                boolean loadReferences)

getDependentKeys

public java.util.Set<Key<?>> getDependentKeys()

Specified by:

getDependentKeys in interface Recoverable<Grid<MP extends Model.Parameters>>

Returns:

list of all keys of objects this recoverable needs to resume operation after recovery

getParams

public MP getParams()