Skip to main content
Version: Next

Settings: Size dependency

Overview​

H2O Model Validation offers an array of settings for a size dependency test. Below, each setting is described in turn.

Settings​

Test name​

This setting defines the name for the validation test. By default, H2O Model Validation assigns a name to the validation test that you can rewrite.

Model​

This setting defines the model H2O Model Validation utilizes to run the size dependency test.

Model train dataset​

note

Model train dataset refers to one of the model's informational points, not a setting. This informational point refers to the model's training dataset that H2O Model Validation utilizes during the test to analyze the effects different training data sizes have on the model's accuracy. The model's training dataset is one of the two datasets H2O Model Validation uses during the test. To learn more, see Size dependency.

Primary dataset​

This setting defines the dataset the test utilizes to analyze the effects different training data sizes have on the model's accuracy.

note

The defined dataset (primary dataset) must follow the model's training dataset format.

Time column​

This setting defines the time column of the primary dataset, which H2O Model Validation utilizes to perform time-based splits.

note

If the selected model already used a time column during training, H2O Model Validation automatically utilizes the used time column during training as the time column.

Number of splits​

This setting defines the number of splits that H2O Model Validation performs on the model training dataset to assess dataset size dependency.

note

The more splits, the better; it maximizes the test's explanatory power. Each split leads to an extra experiment run. This setting affects the runtime.

Delete test models and datasets from the DAI after finish​

This setting determines if H2O Model Validation should delete the artifacts created on the Connection. In this case, artifacts refer to experiments and datasets generated during the backtesting validation test. By default, H2O Model Validation checks this setting (enables it), and accordingly, H2O Model Validation deletes all artifacts because they are no longer needed after the validation test is complete.

Feedback