water.fvec

Class RebalanceDataSet

java.lang.Object

jsr166y.ForkJoinTask<java.lang.Void>

jsr166y.CountedCompleter

water.H2O.H2OCountedCompleter

water.fvec.RebalanceDataSet

All Implemented Interfaces:

java.io.Serializable, java.lang.Cloneable, java.util.concurrent.Future<java.lang.Void>, Freezable

public class RebalanceDataSet
extends H2O.H2OCountedCompleter

Created by tomasnykodym on 3/28/14. Utility to rebalance dataset so that it has requested number of chunks and each chunk has the same number of rows +/-1. It *does not* guarantee even chunk-node placement. (This can not currently be done in H2O, since the placement of chunks is governed only by key-hash /vector group/ for Vecs)

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	RebalanceDataSet.RebalanceTask

Constructor Summary

Constructors

Constructor and Description
RebalanceDataSet(Frame modelFrame, Frame srcFrame, Key<?> dstKey) Constructor for make-compatible task.
RebalanceDataSet(Frame srcFrame, Key dstKey, int nchunks)
RebalanceDataSet(Frame srcFrame, Key dstKey, int nchunks, H2O.H2OCountedCompleter cmp, Key jobKey)
RebalanceDataSet(long[] espc, Vec.VectorGroup vg, Frame srcFrame, Key<?> dstKey)

Method Summary

Modifier and Type	Method and Description
void	compute2() Override compute3() with actual work without having to worry about tryComplete()
Frame	getResult()
void	onCompletion(CountedCompleter caller) Performs an action when method CountedCompleter.tryComplete() is invoked and there are no pending counts, or when the unconditional method CountedCompleter.complete(java.lang.Void) is invoked.
boolean	onExceptionalCompletion(java.lang.Throwable t, CountedCompleter caller) Performs an action when method ForkJoinTask.completeExceptionally(java.lang.Throwable) is invoked or method CountedCompleter.compute() throws an exception, and this task has not otherwise already completed normally.
static Frame	toSingleChunk(Frame fr) Rebalance a (small) frame into a single chunk.
static Frame	toSingleChunk(Frame fr, Key<Frame> destinationKey) Rebalance a (small) frame into a single chunk.

Methods inherited from class water.H2O.H2OCountedCompleter

asBytes, clone, compute, compute1, currThrPriority, frozenType, icer, priority, read, readJSON, reloadFromBytes, write, writeJSON

Methods inherited from class jsr166y.CountedCompleter

__tryComplete, addToPendingCount, compareAndSetPendingCount, complete, exec, getCompleter, getPendingCount, getRawResult, setCompleter, setPendingCount, setRawResult, tryComplete

Methods inherited from class jsr166y.ForkJoinTask

adapt, adapt, adapt, cancel, compareAndSetForkJoinTaskTag, completeExceptionally, fork, get, get, get, getException, getForkJoinTaskTag, getPool, getQueuedTaskCount, getSurplusQueuedTaskCount, helpQuiesce, inForkJoinPool, invoke, invokeAll, invokeAll, invokeAll, isCancelled, isCompletedAbnormally, isCompletedNormally, isDone, join, peekNextLocalTask, pollNextLocalTask, pollTask, quietlyComplete, quietlyInvoke, quietlyJoin, reinitialize, setForkJoinTaskTag, tryUnfork

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RebalanceDataSet

public RebalanceDataSet(Frame modelFrame,
                        Frame srcFrame,
                        Key<?> dstKey)

Constructor for make-compatible task. To be used to make frame compatible with other frame (i.e. make all vecs compatible with other vector group and rows-per-chunk).

RebalanceDataSet

public RebalanceDataSet(long[] espc,
                        Vec.VectorGroup vg,
                        Frame srcFrame,
                        Key<?> dstKey)

RebalanceDataSet

public RebalanceDataSet(Frame srcFrame,
                        Key dstKey,
                        int nchunks)

RebalanceDataSet

public RebalanceDataSet(Frame srcFrame,
                        Key dstKey,
                        int nchunks,
                        H2O.H2OCountedCompleter cmp,
                        Key jobKey)

Method Detail

getResult

public Frame getResult()

compute2

public void compute2()

Description copied from class: H2O.H2OCountedCompleter

Override compute3() with actual work without having to worry about tryComplete()

Overrides:

compute2 in class H2O.H2OCountedCompleter

onCompletion

public void onCompletion(CountedCompleter caller)

Description copied from class: CountedCompleter

Performs an action when method CountedCompleter.tryComplete() is invoked and there are no pending counts, or when the unconditional method CountedCompleter.complete(java.lang.Void) is invoked. By default, this method does nothing.

Overrides:

onCompletion in class CountedCompleter

Parameters:

caller - the task invoking this method (which may be this task itself).

onExceptionalCompletion

public boolean onExceptionalCompletion(java.lang.Throwable t,
                                       CountedCompleter caller)

Description copied from class: CountedCompleter

Performs an action when method ForkJoinTask.completeExceptionally(java.lang.Throwable) is invoked or method CountedCompleter.compute() throws an exception, and this task has not otherwise already completed normally. On entry to this method, this task ForkJoinTask.isCompletedAbnormally(). The return value of this method controls further propagation: If true and this task has a completer, then this completer is also completed exceptionally. The default implementation of this method does nothing except return true.

Overrides:

onExceptionalCompletion in class CountedCompleter

Parameters:

t - the exception

caller - the task invoking this method (which may be this task itself).

Returns:

true if this exception should be propagated to this tasks completer, if one exists.

toSingleChunk

public static Frame toSingleChunk(Frame fr)

Rebalance a (small) frame into a single chunk. This function is useful after filtering/aggregating the frame.

Parameters:

fr - frame to rebalance

Returns:

rebalanced frame, installed into DKV with a random key

toSingleChunk

public static Frame toSingleChunk(Frame fr,
                                  Key<Frame> destinationKey)

Rebalance a (small) frame into a single chunk. This function is useful after filtering/aggregating the frame.

Parameters:

fr - frame to rebalance

destinationKey - key for the new single-chunk Frame

Returns:

rebalanced frame, keyed using destinationKey