hex

Class FrameTask<T extends FrameTask<T>>

java.lang.Object

jsr166y.ForkJoinTask<java.lang.Void>

jsr166y.CountedCompleter

water.H2O.H2OCountedCompleter<T>

water.DTask<T>

water.MRTask<T>

hex.FrameTask<T>

All Implemented Interfaces:

java.io.Serializable, java.lang.Cloneable, java.util.concurrent.Future<java.lang.Void>, jsr166y.ForkJoinPool.ManagedBlocker, water.Freezable<T>

Direct Known Subclasses:

DeepLearningTask, LinearAlgebraUtils.BMulTask

public abstract class FrameTask<T extends FrameTask<T>>
extends water.MRTask<T>

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	FrameTask.ExtractDenseRow

Nested classes/interfaces inherited from class water.MRTask

water.MRTask.PostMapAction<T extends water.MRTask.PostMapAction<T>>

Nested classes/interfaces inherited from class water.DTask

water.DTask.DKeyTask<T extends water.DTask.DKeyTask,V extends water.Keyed>, water.DTask.RemoveCall

Field Summary

Fields

Modifier and Type	Field and Description
protected DataInfo	_dinfo
protected water.Key<water.Job>	_jobKey
protected boolean	_shuffle
protected boolean	_sparse
protected float	_useFraction

Fields inherited from class water.MRTask

_appendables, _fr, _fs, _hi, _keys, _left, _lo, _nhi, _nleft, _nlo, _nrite, _rite, _run_local, _topLocal

Fields inherited from class water.DTask

_ex

Constructor Summary

Constructors

Constructor and Description
FrameTask(water.Key<water.Job> jobKey, DataInfo dinfo)
FrameTask(water.Key<water.Job> jobKey, DataInfo dinfo, long seed, int iteration, boolean sparse)
FrameTask(water.Key<water.Job> jobKey, DataInfo dinfo, long seed, int iteration, boolean sparse, water.H2O.H2OCountedCompleter cmp)

Method Summary

Modifier and Type	Method and Description
protected void	chunkDone(long n) Override this to do post-chunk processing work.
protected boolean	chunkInit() Override this to initialize at the beginning of chunk processing.
protected void	closeLocal()
DataInfo	dinfo()
protected int	getMiniBatchSize() Note: If this is overridden, then applyMiniBatch must be overridden as well to perform the model/weight mini-batch update
void	map(water.fvec.Chunk[] chunks, water.fvec.NewChunk[] outputs) Extracts the values, applies regularization to numerics, adds appropriate offsets to categoricals, and adapts response according to the CaseMode/CaseValue if set.
protected void	processMiniBatch(long seed, double[] responses, double[] offsets, int n) Mini-Batch update of model parameters
protected void	processRow(long gid, DataInfo.Row r) Method to process one row of the data.
protected void	processRow(long gid, DataInfo.Row r, int mb)
protected void	processRow(long gid, DataInfo.Row r, water.fvec.NewChunk[] outputs)
protected void	setupLocal()
protected boolean	skipRow(long gid)

Methods inherited from class water.MRTask

appendables, asyncExecOnAllNodes, block, compute2, dfork, dfork, dfork, dfork, dfork, dinvoke, doAll, doAll, doAll, doAll, doAll, doAll, doAll, doAll, doAll, doAll, doAll, doAll, doAllNodes, getResult, getResult, isReleasable, map, map, map, map, map, map, map, map, map, map, map, modifiesVolatileVecs, onCompletion, onExceptionalCompletion, outputFrame, outputFrame, outputFrame, postGlobal, profile, profString, reduce, self, withPostMapAction

Methods inherited from class water.DTask

copyOver, getDException, hasException, logVerbose, onAck, onAckAck, setException

Methods inherited from class water.H2O.H2OCountedCompleter

asBytes, clone, compute, compute1, currThrPriority, frozenType, icer, priority, read, readJSON, reloadFromBytes, write, writeJSON

Methods inherited from class jsr166y.CountedCompleter

__tryComplete, addToPendingCount, compareAndSetPendingCount, complete, exec, getCompleter, getPendingCount, getRawResult, setCompleter, setPendingCount, setRawResult, tryComplete

Methods inherited from class jsr166y.ForkJoinTask

adapt, adapt, adapt, cancel, compareAndSetForkJoinTaskTag, completeExceptionally, fork, get, get, get, getException, getForkJoinTaskTag, getPool, getQueuedTaskCount, getSurplusQueuedTaskCount, helpQuiesce, inForkJoinPool, invoke, invokeAll, invokeAll, invokeAll, isCancelled, isCompletedAbnormally, isCompletedNormally, isDone, join, peekNextLocalTask, pollNextLocalTask, pollTask, quietlyComplete, quietlyInvoke, quietlyJoin, reinitialize, setForkJoinTaskTag, tryUnfork

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

_sparse

protected boolean _sparse

_dinfo

protected transient DataInfo _dinfo

_jobKey

protected final water.Key<water.Job> _jobKey

_useFraction

protected float _useFraction

_shuffle

protected boolean _shuffle

Constructor Detail

FrameTask

public FrameTask(water.Key<water.Job> jobKey,
                 DataInfo dinfo)

FrameTask

public FrameTask(water.Key<water.Job> jobKey,
                 DataInfo dinfo,
                 long seed,
                 int iteration,
                 boolean sparse)

FrameTask

public FrameTask(water.Key<water.Job> jobKey,
                 DataInfo dinfo,
                 long seed,
                 int iteration,
                 boolean sparse,
                 water.H2O.H2OCountedCompleter cmp)

Method Detail

dinfo

public DataInfo dinfo()

setupLocal

protected void setupLocal()

Overrides:

setupLocal in class water.MRTask<T extends FrameTask<T>>

closeLocal

protected void closeLocal()

Overrides:

closeLocal in class water.MRTask<T extends FrameTask<T>>

processRow

protected void processRow(long gid,
                          DataInfo.Row r)

Method to process one row of the data. See for separate mini-batch logic below. Numeric and categorical values are passed separately, as is response. Categoricals are passed as absolute indexes into the expanded beta vector, 0-levels are skipped (so the number of passed categoricals will not be the same for every row). Categorical expansion/indexing: Categoricals are placed in the beginning of the beta vector. Each cat variable with n levels is expanded into n-1 independent binary variables. Indexes in cats[] will point to the appropriate coefficient in the beta vector, so e.g. assume we have 2 categorical columns both with values A,B,C, then the following rows will have following indexes: A,A - ncats = 0, we do not pass any categorical here A,B - ncats = 1, indexes = [2] B,B - ncats = 2, indexes = [0,2] and so on

Parameters:

gid - - global id of this row, in [0,_adaptedFrame.numRows())

processRow

protected void processRow(long gid,
                          DataInfo.Row r,
                          water.fvec.NewChunk[] outputs)

processRow

protected void processRow(long gid,
                          DataInfo.Row r,
                          int mb)

skipRow

protected boolean skipRow(long gid)

processMiniBatch

protected void processMiniBatch(long seed,
                                double[] responses,
                                double[] offsets,
                                int n)

Mini-Batch update of model parameters

Parameters:

seed -

responses -

offsets -

n - actual number of rows in this minibatch

getMiniBatchSize

protected int getMiniBatchSize()

Note: If this is overridden, then applyMiniBatch must be overridden as well to perform the model/weight mini-batch update

Returns:

Return the mini-batch size

chunkInit

protected boolean chunkInit()

Override this to initialize at the beginning of chunk processing.

Returns:

whether or not to process this chunk

chunkDone

protected void chunkDone(long n)

Override this to do post-chunk processing work.

Parameters:

n - Number of processed rows

map

public void map(water.fvec.Chunk[] chunks,
                water.fvec.NewChunk[] outputs)

Extracts the values, applies regularization to numerics, adds appropriate offsets to categoricals, and adapts response according to the CaseMode/CaseValue if set.

Overrides:

map in class water.MRTask<T extends FrameTask<T>>