hex.faulttolerance

Class Recovery<T extends Keyed>

java.lang.Object

hex.faulttolerance.Recovery<T>

Type Parameters:

T - Type of object to be recovered

public class Recovery<T extends Keyed>
extends java.lang.Object

H2O Auto-Recovery Support

This class encapsulates what is the core of H2O's (auto)recovery support. It manages storing of data needed to recover a Recoverable process as well as performing the actual recovery and re-start of such process.

Preparing for recovery

A Recoverable will instantiate a Recovery instance at start and call the onStart(Recoverable, Job) method first. This should provide sufficient data to re-instantiate the Recoverable later on. Every time a Model is successfully built the Recoverable should call onModel(Recoverable, Key) method so that this Model can be later recovered and does not need to be trained again. If a Recoverable process finishes successfully it will call onDone(Recoverable), this will lead to all stored data being cleaned up.

Recovering manually

Recoverable objects may use this class to restore their state in case user has sent such a request. This is useful since this class implements mechanisms not implemented by other components in the system (such as storing parameter references).

Auto-Recovery

Calling autoRecover(Optional) will trigger an auto-recovery. The method will check if there is any recovery data present in the supplied directory and if there is it will load all the stored data, references and models and resume the Recoverable process. It is the responsibility of the Recoverable to check in what state was it stored (models already trained) and continue on a best-effort basis with its job such that no unnecessary repetition of work is done.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Recovery.ReferenceType Frame object referenced by params are stored in a distributed manner, hence we need to distinguish them from other types of Keyed references.

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	INFO_CLASS
static java.lang.String	INFO_JOB_KEY
static java.lang.String	INFO_RESULT_KEY
static java.lang.String	RECOVERY_META_FILE
static java.lang.String	REFERENCES_META_FILE_SUFFIX

Constructor Summary

Constructors

Constructor and Description
Recovery(java.lang.String storagePath)

Method Summary

Modifier and Type	Method and Description
static void	autoRecover(java.util.Optional<java.lang.String> autoRecoveryDirOpt) Will check the supplied directory for presence of recovery metadata and if found, trigger a recovery of interrupted Recoverable process.
void	exportReferences(Recoverable<T> r) Saves all of the keyed objects used by this Grid's params.
void	loadReferences(Recoverable<T> r) Will locate a references metadata file and load all objects mentioned in this file.
void	onDone(Recoverable<T> r) Called by the recoverable that the training was finished successfully.
void	onModel(Recoverable<T> r, Key<Model> modelKey) Called by the Recoverable to notify of new model was trained and needs to persisted
void	onStart(Recoverable<T> r, Job job) Called when the training begins, so that initial state can be persisted
java.lang.String	recoveryMetaFile()
java.lang.String	referencesMetaFile(Recoverable<T> r)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

REFERENCES_META_FILE_SUFFIX

public static final java.lang.String REFERENCES_META_FILE_SUFFIX

See Also:

Constant Field Values

RECOVERY_META_FILE

public static final java.lang.String RECOVERY_META_FILE

See Also:

Constant Field Values

INFO_CLASS

public static final java.lang.String INFO_CLASS

See Also:

Constant Field Values

INFO_RESULT_KEY

public static final java.lang.String INFO_RESULT_KEY

See Also:

Constant Field Values

INFO_JOB_KEY

public static final java.lang.String INFO_JOB_KEY

See Also:

Constant Field Values

Constructor Detail

Recovery

public Recovery(java.lang.String storagePath)

Parameters:

storagePath - directory to use as base for recovery snapshots

Method Detail

autoRecover

public static void autoRecover(java.util.Optional<java.lang.String> autoRecoveryDirOpt)

Will check the supplied directory for presence of recovery metadata and if found, trigger a recovery of interrupted Recoverable process.

Parameters:

autoRecoveryDirOpt - directory from which to load recovery data

referencesMetaFile

public java.lang.String referencesMetaFile(Recoverable<T> r)

recoveryMetaFile

public java.lang.String recoveryMetaFile()

onStart

public void onStart(Recoverable<T> r,
                    Job job)

Called when the training begins, so that initial state can be persisted

Parameters:

r - a Recoverable to persist

onModel

public void onModel(Recoverable<T> r,
                    Key<Model> modelKey)

Called by the Recoverable to notify of new model was trained and needs to persisted

Parameters:

r - a Recoverable to update

modelKey - key of the newly trained model

onDone

public void onDone(Recoverable<T> r)

Called by the recoverable that the training was finished successfully. This means that recovery snapshots (persisted data) is no longer needed and can be deleted.

exportReferences

public void exportReferences(Recoverable<T> r)

Saves all of the keyed objects used by this Grid's params. Files are named by objects' keys.

loadReferences

public void loadReferences(Recoverable<T> r)

Will locate a references metadata file and load all objects mentioned in this file.

Parameters:

r - a Recoverable whose references are to be loaded