Skip to main content

Task 2: Training settings and expert settings

Overview​

  • Accuracy: Higher accuracy indicates better model performance in predicting outcomes.
  • Time: This refers to the relative amount of time needed to complete the experiment. Higher values indicate longer execution times.
  • Interpretability: This reflects how easy it is to understand the model's reasoning and the features it uses for prediction. Higher interpretability often means simpler features, making the model's decisions easier to explain.
  1. The Training Settings describe the Accuracy, Time, and Interpretability of your specific experiment. The knobs on the training settings are adjustable, as the values change the meaning of the settings on the left-bottom page. For more information on training settings, see training settings in the H2O Driverless AI documentation.

Accuracy​

By increasing the ACCURACY setting, Driverless AI gradually adjusts the method for performing evolution and ensemble creation. A machine learning ensemble consists of multiple learning algorithms to obtain better predictive performance than any single algorithm could achieve. With a low accuracy setting, Driverless AI explores and uses a wider variety of features engineered from the data and models, but they all compete evenly against each other. At higher accuracy, each independent main model will evolve independently and be part of the final ensemble as an ensemble over different main models. Driverless AI will evolve and ensemble feature types like Target Encoding (on and off) that evolve independently at higher accuracy. Finally, at the highest accuracies, Driverless AI performs both model and feature tracking and ensembles all those variations.

Time​

TIME specifies the relative time for completing the experiment. Higher settings result in longer execution times. Early stopping will occur if the experiment's score doesn't improve for a specified number of iterations. A higher time value allocates more time for further iterations, allowing the recipe to explore more feature engineering transformations and hyperparameter tunings.

Interpretability​

The INTERPRETABILITY knob is adjustable. Higher interpretability leads to simpler features extracted by the main modeling routine from the dataset. At sufficiently high interpretability, a monotonically constrained model will be generated.

  1. For this tutorial, update the training settings to match the following:
  • ACCURACY : 4
  • TIME: 3
  • INTERPRETABILITY: 4
  • SCORER: LOGLOSS
note

This configuration was selected to generate a model quickly with a sufficient level of accuracy in the H2O Driverless Test Drive environment.

Expert settings​

Expert settings are options that are available for those who wish to set their settings manually.

  1. Click on EXPERT SETTINGS: expert-settings-0
  1. For this experiment, set RuleFit models to ON under the MODELS selection and click SAVE. expert-settings-rulefit-on

The RuleFit algorithm creates an optimal set of decision rules. It first fits a tree model and then fits a Lasso (L1-regularized) GLM model. This creates a linear model consisting of the most important tree leaves (rules). The RuleFit model can achieve higher accuracy than Random Forests while still retaining the interpretability of decision trees.

Turning the RuleFit model ON will be added to the list of algorithms that Driverless AI will consider for the experiment. The selection of the algorithm depends on the data and the configuration selected.

note
  • You can also search for specific settings by searching for it on the SEARCH bar located on the menu bar.
  • You can view more details about each setting by clicking on the i icon located on the right side of each option.
  1. Before launching the experiment, ensure your Experiment page looks identical to the one below. Once ready, click on LAUNCH EXPERIMENT. review-experiment For more information about what each setting means and how it can be updated from its default values, see Expert Settings in the H2O Driverless AI documentation.

Congratulations, you have successfully configured the training settings and expert settings for your experiment. In Task 3, you will learn more about experiment scoring and analysis concepts.

Feedback