water.util

Class VecUtils

java.lang.Object

water.util.VecUtils

public class VecUtils
extends java.lang.Object

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	VecUtils.CollectDomainFast (Optimized for positive ints) Collect numeric domain of given Vec A map-reduce task to collect up the unique values of an integer Vec and returned as the domain for the Vec.
static class	VecUtils.CollectDomainWeights
static class	VecUtils.CollectDoubleDomain
static class	VecUtils.CollectIntegerDomain Collect numeric domain of given Vec A map-reduce task to collect up the unique values of an integer Vec and returned as the domain for the Vec.
static class	VecUtils.DomainDedupe Create a new categorical Vec with deduplicated domains from a categorical Vec.
static class	VecUtils.DotProduct DotProduct of two Vecs of the same length
static class	VecUtils.MeanResponsePerLevelTask Compute the mean (weighted) response per categorical level Skip NA values (those are already a separate bucket in the tree building histograms, for which this is designed)
static class	VecUtils.MinMaxTask
static class	VecUtils.ReorderTask Reorder an integer (such as Enum storage) Vec using an int -> int mapping
static class	VecUtils.SequenceProduct
static class	VecUtils.ShuffleVecTask Randomly shuffle a Vec using Fisher Yates shuffle https://en.wikipedia.org/wiki/Fisher%E2%80%93Yates_shuffle

Constructor Summary

Constructors

Constructor and Description
VecUtils()

Method Summary

Modifier and Type	Method and Description
static Vec	categoricalDomainsToNumeric(Vec src) Create a new Vec of numeric values from a categorical Vec.
static Vec	categoricalToInt(Vec src) Create a new Vec of numeric values from a categorical Vec.
static Vec	categoricalToStringVec(Vec src) Create a new Vec of string values from a categorical Vec.
static java.lang.String[]	collectDomainFast(Vec vec) Collects current domain of a categorical vector in an optimized way.
static double[]	collectDomainWeights(Vec vec, Vec weights) Collect the frequencies of each level in a categorical Vec.
static void	deleteVecs(Vec[] vs, int cnt)
static VecUtils.MinMaxTask	findMinMax(Vec numVec, Vec weightVec)
static int[]	getLocalChunkIds(Vec v)
static Vec	numericToCategorical(Vec src) Create a new Vec of categorical values from a numeric Vec.
static Vec	numericToStringVec(Vec src) Create a new Vec of string values from a numeric Vec.
static Vec	numToInteger(Vec src)
static Vec	remapDomain(java.lang.String[] newDomainValues, Vec originalVec) Remaps vec's current domain levels to a new set of values.
static Vec	shuffleVec(Vec origVec, long seed) Randomly shuffle a Vec.
static Vec	stringToCategorical(Vec vec) Create a new Vec of categorical values from string Vec.
static Vec	stringToInteger(Vec src)
static Vec	stringToNumeric(Vec src) Create a new Vec of numeric values from a string Vec.
static Vec	toCategoricalVec(Vec src) Create a new Vec of categorical values from an existing Vec.
static Vec	toIntegerVec(Vec src)
static Vec	toNumericVec(Vec src) Create a new Vec of numeric values from an existing Vec.
static Vec	toStringVec(Vec src) Create a new Vec of string values from an existing Vec.
static Vec	UUIDToStringVec(Vec src) Create a new Vec of string values from a UUID Vec.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

VecUtils

public VecUtils()

Method Detail

toCategoricalVec

public static Vec toCategoricalVec(Vec src)

Create a new Vec of categorical values from an existing Vec. This method accepts all Vec types as input. The original Vec is not mutated. If src is a categorical Vec, a copy is returned. If src is a numeric Vec, the values are converted to strings used as domain values. For all other types, an exception is currently thrown. These need to be replaced with appropriate conversions. Throws H2OIllegalArgumentException() if the resulting domain exceeds Categorical.MAX_CATEGORICAL_COUNT.

Parameters:

src - A Vec whose values will be used as the basis for a new categorical Vec

Returns:

the resulting categorical Vec

stringToCategorical

public static Vec stringToCategorical(Vec vec)

Create a new Vec of categorical values from string Vec. FIXME: implement in more efficient way with Brandon's primitives for BufferedString manipulation

Parameters:

vec - a string Vec

Returns:

a categorical Vec

numericToCategorical

public static Vec numericToCategorical(Vec src)

Create a new Vec of categorical values from a numeric Vec. This currently only ingests a Vec of integers. Handling reals is PUBDEV-2207

Parameters:

src - a numeric Vec

Returns:

a categorical Vec

toNumericVec

public static Vec toNumericVec(Vec src)

Create a new Vec of numeric values from an existing Vec. This method accepts all Vec types as input. The original Vec is not mutated. If src is a categorical Vec, a copy is returned. If src is a string Vec, all values that can be are parsed into reals or integers, and all others become NA. See stringToNumeric for parsing details. If src is a numeric Vec, a copy is made. If src is a time Vec, the milliseconds since the epoch are used to populate the new Vec. If src is a UUID Vec, the existing numeric storage is used to populate the new Vec. Throws H2OIllegalArgumentException() if the resulting domain exceeds Categorical.MAX_CATEGORICAL_COUNT.

Parameters:

src - A Vec whose values will be used as the basis for a new numeric Vec

Returns:

the resulting numeric Vec

toIntegerVec

public static Vec toIntegerVec(Vec src)

numToInteger

public static Vec numToInteger(Vec src)

stringToNumeric

public static Vec stringToNumeric(Vec src)

Create a new Vec of numeric values from a string Vec. Any rows that cannot be converted to a number are set to NA. Currently only does basic numeric formats. No exponents, or hex values. Doesn't even like commas or spaces. :( Needs love. Handling more numeric representations is PUBDEV-2209

Parameters:

src - a string Vec

Returns:

a numeric Vec

stringToInteger

public static Vec stringToInteger(Vec src)

categoricalToInt

public static Vec categoricalToInt(Vec src)

Create a new Vec of numeric values from a categorical Vec. If the first value in the domain of the src Vec is a stringified ints, then it will use those ints. Otherwise, it will use the raw enumeration level mapping. If the domain is stringified ints, then all of the domain must be able to be parsed as an int. If it cannot be parsed as such, a NumberFormatException will be caught and rethrown as an H2OIllegalArgumentException that declares the illegal domain value. Otherwise, the this pointer is copied to a new Vec whose domain is null. The magic of this method should be eliminated. It should just use enumeration level maps. If the user wants domains to be used, call categoricalDomainsToNumeric(). PUBDEV-2209

Parameters:

src - a categorical Vec

Returns:

a numeric Vec

toStringVec

public static Vec toStringVec(Vec src)

Create a new Vec of string values from an existing Vec. This method accepts all Vec types as input. The original Vec is not mutated. If src is a string Vec, a copy of the Vec is made. If src is a categorical Vec, levels are dropped, and the Vec only records the string. For all numeric Vecs, the number is converted to a string. For all UUID Vecs, the hex representation is stored as a string.

Parameters:

src - A Vec whose values will be used as the basis for a new string Vec

Returns:

the resulting string Vec

categoricalToStringVec

public static Vec categoricalToStringVec(Vec src)

Create a new Vec of string values from a categorical Vec. Transformation is done by a Categorical2StrChkTask which provides a mapping between values - without copying the underlying data.

Parameters:

src - a categorical Vec

Returns:

a string Vec

numericToStringVec

public static Vec numericToStringVec(Vec src)

Create a new Vec of string values from a numeric Vec. Currently only uses a default pretty printer. Would be better if it accepted a format string PUBDEV-2211

Parameters:

src - a numeric Vec

Returns:

a string Vec

UUIDToStringVec

public static Vec UUIDToStringVec(Vec src)

Create a new Vec of string values from a UUID Vec. String Vec is the standard hexadecimal representations of a UUID.

Parameters:

src - a UUID Vec

Returns:

a string Vec

categoricalDomainsToNumeric

public static Vec categoricalDomainsToNumeric(Vec src)

Create a new Vec of numeric values from a categorical Vec. Numeric values are generated explicitly from the domain values, and not the enumeration levels. If a domain value cannot be translated as a number, that domain and all values for that domain will be NA.

Parameters:

src - a categorical Vec

Returns:

a numeric Vec

collectDomainFast

public static java.lang.String[] collectDomainFast(Vec vec)
                                            throws java.lang.IllegalArgumentException

Collects current domain of a categorical vector in an optimized way. Original vector's domain is not modified.

Parameters:

vec - A categorical vector to collect domain of.

Returns:

An array of String with the domain of given vector - possibly empty if the domain is empty. Never null.

Throws:

java.lang.IllegalArgumentException - If the given vector is not categorical

collectDomainWeights

public static double[] collectDomainWeights(Vec vec,
                                            Vec weights)

Collect the frequencies of each level in a categorical Vec.

Parameters:

vec - categorical Vec

weights - optional weight Vec

Returns:

(weighted) frequencies of each level of the input Vec

deleteVecs

public static void deleteVecs(Vec[] vs,
                              int cnt)

getLocalChunkIds

public static int[] getLocalChunkIds(Vec v)

remapDomain

public static Vec remapDomain(java.lang.String[] newDomainValues,
                              Vec originalVec)
                       throws java.lang.UnsupportedOperationException,
                              java.lang.IllegalArgumentException

Remaps vec's current domain levels to a new set of values. The cardinality of the new set of domain values might be less than or equal to the cardinality of current domain values. The new domain set is automatically extracted from the given mapping array.

Changes are made to this very vector, no copying is done. If you need the original vector to remain unmodified, please make sure to copy it first.

Parameters:

newDomainValues - An array of new domain values. For each old domain value, there must be a new value in this array. The value at each index of newDomainValues array represents the new mapping for this very index. May not be null.

originalVec - Vector with values corresponding to the original domain to be remapped. Remains unmodified.

Returns:

A new instance of categorical Vec with exactly the same length as the original vector supplied. Its domain values are re-mapped.

Throws:

java.lang.UnsupportedOperationException - When invoked on non-categorical vector

java.lang.IllegalArgumentException - Length of newDomainValues must be equal to length of current domain values of this vector

shuffleVec

public static Vec shuffleVec(Vec origVec,
                             long seed)

Randomly shuffle a Vec.

Parameters:

origVec - original Vec

seed - seed for random generator

Returns:

shuffled Vec

findMinMax

public static VecUtils.MinMaxTask findMinMax(Vec numVec,
                                             Vec weightVec)