Skip to main content
Version: v0.15.0

Settings: Size dependency

H2O Model Validation offers an array of settings for a size dependency test. Below, each setting is described in turn.

Test name​

Defines the name for the validation test; by default, H2O Model Validation will assign a name to the validation test that you can rewrite.

Model​

Defines the model H2O Model Validation utilizes to run the size dependency test.

Primary Dataset​

Defines one of the two datasets H2O Model Validation uses during the validation test to analyze the effects different training data sizes will have on the selected model's accuracy. The defined Primary Dataset needs to follow the model's training dataset format.

Secondary Dataset​

Defines one of the two datasets H2O Model Validation uses during the validation test to analyze the effects different training data sizes will have on the selected model's accuracy. Within the context of validating a model, the defined Secondary Dataset needs to follow the structure of the Primary Dataset.

note

By default, you cannot modify this setting; a size dependency validation test can only run models that use a test dataset. In that case, if you did not utilize a test dataset, H2O Model Validation will create a Secondary Dataset during the validation test.

Time column​

Defines the time column of the Primary and Secondary Dataset, which H2O Model Validation utilizes to perform time-based splits.

Number of splits​

Defines the number of splits that H2O Model Validation performs on the Primary Dataset to assess dataset size dependency.

note

The more splits, the better in general; it maximizes the test's explanatory power. Each split leads to an extra experiment run; this setting affects the runtime.

Delete test models and datasets from the Worker after finish​

Determines if H2O Model Validation should delete the artifacts created in the Platform connected to the Worker Connection. In this case, artifacts refer to experiments and datasets generated during the backtesting validation test. By default, H2O Model Validation checks this setting (enables it), and accordingly, H2O Model Validation deletes all artifacts because they are no longer needed after the validation test is complete.

Feedback