Skip to main content
Version: v3.0.0

Register a feature set

To register a feature set, you first need to obtain the schema. See Create a schema for information on how to create the schema.

workspace = client.workspaces.list(name="my_workspace")[0]
workspace.feature_sets.register(schema, "feature_set_name", description="", primary_key=None, time_travel_column=None, time_travel_column_format="yyyy-MM-dd HH:mm:ss", partition_by=None, time_travel_column_as_partition=False)

If the partition_by argument is not set, the time travel column will be used by Feature Store to partition the layout by each ingestion. If it is defined, time_travel_column_as_partition can be set to True to use time travel based partitioning additionally.

note

In case primary key or partition by arguments contain same feature multiple times, only distinct values are used.

note

If value in primary key or partition by or time travel column corresponds to two or more features, most nested is selected by default. In other cases, specific feature can be selected by enclosing the feature name in ``

For example, feature set contains feature named "test.data" and second feature "test" with nested feature "data". But default for value "test.data", nested feature "data" will be selected. If feature with name "test.data" should be selected, value should be changed to "`.test.data`"

note

Feature Store is using time format used by Spark. Specification is available here.

note

If users wants to create feature sets which are accessible only by the owner and users the owner gave permission to, that feature set should be created in a private workspace.

To see naming conventions for feature set names, please visit Default naming rules.

To register a derived feature set, you first need to obtain the derived schema. See Create a schema for information on how to create the schema.

from h2o_featurestore import SparkPipeline
spark_pipeline_transformation = SparkPipeline("...")

workspace = client.workspaces.list(name="my_workspace")[0]
derived_schema = workspace.extract_derived_schema([parent_feature_set], spark_pipeline_transformation)

workspace.feature_sets.register(derived_schema, "derived_feature_set", description="", primary_key=None, time_travel_column=None, time_travel_column_format="yyyy-MM-dd HH:mm:ss", partition_by=None, time_travel_column_as_partition=False)

Features can be masked by setting Special Data fields in the schema. For further information, please visit Modify special data on a schema.

Setting any of the following attributes to true marks the feature for masking:

  • spi - Sensitive Personal Information
  • pci - Payment Card Industry
  • rpi - Real Property Inventory
  • demographic
  • sensitive

Any of the special data tags would allow for the masking functionality to work and separate sensitive consumer output (e.g. unmasked data) from the masked view that the consumer role sees. Which tag is selected is more bookkeeping than leading to different functionality.

note

Feature Store does not support registering feature sets with the following characters in column names:

  • ,
  • ;
  • { or }
  • ( or )
  • new line character
  • tab character
  • =

Time travel column selection​

You can specify a time travel column during the registration call. If the column is specified, Feature Store will use that column to obtain time travel data and will use it for incremental ingest purposes. The explicitly passed time travel column must be present in the schema passed to the registration call.

If the time travel column is not specified, a virtual one is created, so you can still do time travel on static feature sets. Each ingestion to this feature set is treated as a new batch of data with a new timestamp.

Use the following register method argument to specify the name of the time travel column explicitly:

time_travel_column

Inferring the data type of date-time columns during feature set registration​

File types without schema information: For file types that have no metadata about column types (e.g., CSV), Feature Store detects date-time columns as regular string.

File types containing schema information: For file types that keep information about the data types (e.g., Parquet), Feature Store respects those types. If a date-time column is stored with a type of Timestamp or Date, Feature Store will respect that during the registration.

End-to-end examples​

For complete, runnable registration and ingestion examples for each supported data source — CSV, Parquet, JSON, Delta, JDBC, Snowflake, MongoDB, Spark pipelines and more — see Examples.

Feedback