Skip to main content
Version: v3.0.0

Create a derived schema

A derived feature set is defined in terms of one or more parent feature sets and a transformation. To register one you first need a derived schema. To see all supported transformations, see Supported derived transformations.

Create a derived schema from a parent feature set with applied transformation​

A derived schema can be created from one or more existing feature sets using a selected transformation.

from h2o_featurestore import SparkPipeline
spark_pipeline_transformation = SparkPipeline("...")

workspace = client.workspaces.list(name="my_workspace")[0]
schema = workspace.extract_derived_schema([parent_feature_set], spark_pipeline_transformation)

# Keyword form, deriving from more than one parent
schema = workspace.extract_derived_schema(
    feature_sets=[fs_a, fs_b],
    transformation=SparkPipeline(...),
)

# Async version — returns a Job immediately
job = workspace.extract_derived_schema_async(
    feature_sets=[fs_a, fs_b],
    transformation=SparkPipeline(...),
)
schema = job.wait_for_result()
note

Each parent feature set must already have been ingested, and you need editor permission on it. An optional spark_size parameter selects a Spark size preset; when left blank, the workspace default is used.

An asynchronous variant is also available — see Asynchronous schema extraction.

Create a derived schema from a string​

from h2o_featurestore import Schema, SparkPipeline
spark_pipeline_transformation = SparkPipeline("...")
schema_str = "id INT, text STRING, label DOUBLE, state STRING, date STRING, words ARRAY<STRING>"
schema = Schema.create_derived_from(schema_str, [parent_feature_set], spark_pipeline_transformation)

Parameters​

Parameters

ParameterTypeRequiredDescription
feature_setslistYesParent feature sets to derive from. Each one must already have been ingested.
transformationtransformationYesThe transformation to apply, for example SparkPipeline. See Supported derived transformation.
spark_sizestrNoSpark size preset name. When left blank, the workspace default is used.

The parent feature sets may live in other workspaces, as long as you have editor permission on each of them. The workspace you call the method on runs the job and owns the resulting feature set. See Joining feature sets from different workspaces for a worked example.

Feedback