Skip to main content
Version: v0.15.0

Settings: Backtesting

H2O Model Validation offers an array of settings for a backtesting test. Below, each setting is described in turn.

Test name​

Defines the name for the validation test; by default, H2O Model Validation will assign a name to the validation test that you can rewrite.

Model​

Defines the model H2O Model Validation utilizes to run the size dependency test.

Primary Dataset​

Defines the dataset H2O Model Validation uses during the validation test to assess model performance over time. H2O Model Validation splits the dataset by time to simulate a model refitting process while observing and recording the model's accuracy over time. The defined Primary Dataset needs to follow the model's training dataset format. Training data (Primary Dataset) before the specified date is utilized as the training dataset during each split. Training data (Primary Dataset) after the specified date is utilized as the test dataset.

Time column​

Defines the time column in the Primary Dataset that H2O Model Validation utilizes to split the primary dataset (training dataset) during the backtesting test.

Split dates​

Determines if you or H2O Model Validation define the split dates, which creates the backtesting experiments.

Auto​

H2O Model Validation determines the split dates, which creates the backtesting experiments.

Custom​

You determine the split dates, which creates the backtesting experiments.

Custom split dates​

Defines the split dates H2O Model Validation utilizes to create the backtesting experiments.

Number of splits​

Defines the number of dates (splits) H2O Model Validation utilizes for the dataset splitting and model refitting process during the validation test. H2O Model Validation sets the specified number of dates in the past while being equally apart from one another to generate appropriate dataset splits.

Period lengths​

Test period length​

Defines the length of the test dataset for each backtesting split, starting from a split date. By default (auto), and if a model test dataset is available, H2O Model Validation utilizes the length of the model test dataset. H2O Model Validation would implement a model forecast horizon approach if you did not utilize a test dataset when building the model.

Units​

Defines the data time unit H2O Model Validation uses for each test dataset split.

Training period length​

Defines the length of the training dataset for each backtesting split, ending at a split date. By default, H2O Model Validation specifies this setting as auto, which indicates that H2O Model Validation uses all the data up to a split date as the training dataset.

Units​

Defines the data time unit H2O Model Validation uses for each training dataset split.

Delete test models and datasets from the Worker after finish​

Determines if H2O Model Validation should delete the artifacts created in the Platform connected to the Worker Connection. In this case, artifacts refer to experiments and datasets generated during the backtesting validation test. By default, H2O Model Validation checks this setting (enables it), and accordingly, H2O Model Validation deletes all artifacts because they are no longer needed after the validation test is complete.

Feedback