Model Settings¶
enable_constant_model¶
Constant Models
Specify whether to enable constant models. This is set to Auto (enabled) by default.
enable_decision_tree¶
Decision Tree Models
Specify whether to build Decision Tree models as part of the experiment. This is set to Auto by default. In this case, Driverless AI will build Decision Tree models if interpretability is greater than or equal to the value of decision_tree_interpretability_switch (which defaults to 7) and accuracy is less than or equal to decision_tree_accuracy_switch (which defaults to 7).
enable_glm¶
GLM Models
Specify whether to build GLM models (generalized linear models) as part of the experiment (usually only for the final model unless it’s used exclusively). GLMs are very interpretable models with one coefficient per feature, an intercept term and a link function. This is set to Auto by default (enabled if accuracy <= 5 and interpretability >= 6).
enable_xgboost_gbm¶
XGBoost GBM Models
Specify whether to build XGBoost models as part of the experiment (for both the feature engineering part and the final model). XGBoost is a type of gradient boosting method that has been widely successful in recent years due to its good regularization techniques and high accuracy. This is set to Auto by default. In this case, Driverless AI will use XGBoost unless the number of rows * columns is greater than a threshold. This threshold is a config setting that is 100M by default for CPU and 30M by default for GPU.
enable_lightgbm¶
LightGBM Models
Specify whether to build LightGBM models as part of the experiment. LightGBM Models are the default models. This is set to Auto (enabled) by default.
enable_xgboost_dart¶
XGBoost Dart Models
Specify whether to use XGBoost’s Dart method when building models for experiment (for both the feature engineering part and the final model). This is set to Auto (disabled) by default.
enable_xgboost_rapids¶
Enable RAPIDS-cuDF extensions to XGBoost GBM/Dart
Specify whether to enable RAPIDS extensions to XGBoost GBM/Dart. If selected, python scoring package can only be used on GPU system. The equivalent config.toml parameter is enable_xgboost_rapids and the default value is False.
Disabled for dask multinode models due to bug in dask_cudf and xgboost.
enable_xgboost_rf¶
Enable XGBoost RF model
Specify whether to enable XGBoost RF mode without early stopping. This setting is disabled unless switched on.
enable_xgboost_gbm_dask¶
Enable Dask_cuDF (multi-GPU) XGBoost GBM
Specify whether to enable Dask_cudf (multi-GPU) version of XGBoost GBM. Disabled unless switched on. Only applicable for single final model without early stopping. No Shapley possible. The equivalent config.toml parameter is enable_xgboost_gbm_dask and the default value is “auto”.
enable_xgboost_dart_dask¶
Enable Dask_cuDF (multi-GPU) XGBoost Dart
Specify whether to enable Dask_cudf (multi-GPU) version of XGBoost GBM/Dart. This option is disabled unless switched on. Only applicable for single final model without early stopping. No Shapley is possible. The equivalent config.toml parameter is enable_xgboost_dart_dask and the default value is “auto”.
It is recommended to run Dask_cudf on multi gpus; if for say debugging purposes, user would like to enable them on 1 GPU, then set use_dask_for_1_gpu to True via config.toml setting.
enable_lightgbm_dask¶
Enable Dask (multi-node) LightGBM
Specify whether to enable multi-node LightGBM. It is disabled by default unless switched on. The equivalent config.toml parameter is enable_lightgbm_dask and default value is “auto”.
To enable multinode Dask see Dask Multinode Training.
enable_hyperopt_dask¶
Enable Dask (multi-node/multi-GPU) hyperparameter search
Specify whether to enable Dask (multi-node/multi-GPU) version of hyperparameter search. “auto” and “on” are same currently. Dask mode for hyperparameter search is enabled if:
Have a Dask multinode cluster or multi-GPU node and model uses 1 GPU for each model( see num_gpus_per_model).
Not already using a Dask model.
The equivalent config.toml parameter is enable_hyperopt_dask and the default value is “auto”.
num_inner_hyperopt_trials_prefinal¶
Number of trials for hyperparameter optimization during model tuning only
Specify the number of trials for Optuna hyperparameter optimization for tuning and evolution of models. If using RAPIDS or DASK, this parameter specifies the number of trials for hyperparameter optimization within XGBoost GBM/Dart and LightGBM and hyperparameter optimization keeps data on GPU entire time.
0 means no trials. For small data, 100 is fine, while for larger data smaller values are reasonable if need results quickly. If using RAPIDS or DASK, hyperparameter optimization stays on GPU the entire time. The equivalent config.toml parameter is num_inner_hyperopt_trials_prefinal and the default value is 0.
Note that, this is useful when there is high overhead of DAI outside inner model fit/predict (i.e the various file, process, and other DAI management processes), so this tunes without that overhead. However, this can overfit on a single fold when doing tuning or evolution, and if using Cross Validation then, averaging the fold hyperparameters can lead to unexpected results.
num_inner_hyperopt_trials_final¶
Number of trials for hyperparameter optimization for final model only
Number of trials for Optuna hyperparameter optimization for final models. If using RAPIDS or DASK, this is number of trials for rapids-cudf hyperparameter optimization within XGBoost GBM/Dart and LightGBM, and hyperparameter optimization keeps data on GPU entire time.
0 means no trials.For small data, 100 is ok choice, while for larger data smaller values are reasonable if need results quickly. This setting applies to final model only, even if num_inner_hyperopt_trials=0. The equivalent config.toml parameter is num_inner_hyperopt_trials_final and the default value is 0.
num_hyperopt_individuals_final¶
Number of individuals in final ensemble to use Optuna on
Number of individuals in final model (all folds/repeats for given base model) to optimize with Optuna hyperparameter tuning. The default value is -1, means all. 0 is same as choosing no Optuna trials. Might be only beneficial to optimize hyperparameters of best individual (i.e. value of 1) in ensemble.
The default value is -1, means all. The equivalent config.toml parameter is num_hyperopt_individuals_final
optuna_pruner¶
Optuna Pruners
Optuna Pruner algorithm to use for early stopping of unpromising trials (applicable to XGBoost and LightGBM that support Optuna callbacks). The default is MedianPruner. To disable choose None.
The equivalent config.toml parameter is optuna_pruner
optuna_sampler¶
Optuna Samplers
Optuna Sampler algorithm to use for narrowing down and optimizing the search space (applicable to XGBoost and LightGBM that support Optuna callbacks). The default is TPESampler. To disable choose None.
The equivalent config.toml parameter is optuna_sampler
enable_xgboost_hyperopt_callback¶
Enable Optuna XGBoost Pruning callback
Specify whether to enable Optuna’s XGBoost Pruning callback to abort unpromising runs. This is True by default. This not is enabled when tuning learning rate.
The equivalent config.toml parameter is enable_xgboost_hyperopt_callback
enable_lightgbm_hyperopt_callback¶
Enable Optuna LightGBM Pruning callback
Specify whether to enable Optuna’s LightGBM Pruning callback to abort unpromising runs. This is True by default. This not is enabled when tuning learning rate.
The equivalent config.toml parameter is enable_lightgbm_hyperopt_callback
enable_tensorflow¶
TensorFlow Models
Specify whether to build TensorFlow models as part of the experiment (usually only for text features engineering and for the final model unless it’s used exclusively). Enable this option for NLP experiments. This is set to Auto by default (not used unless the number of classes is greater than 10).
TensorFlow models are not yet supported by Java MOJOs (only Python scoring pipelines and C++ MOJOs are supported).
enable_grownet¶
PyTorch GrowNet Models
Specify whether to enable PyTorch-based GrowNet models. By default, this parameter is set to auto i.e Driverless decides internally whether to use the algorithm for the experiment. Set it to on to force the experiment to build a GrowNet model.
enable_ftrl¶
FTRL Models
Specify whether to build Follow the Regularized Leader (FTRL) models as part of the experiment. Note that MOJOs are not yet supported (only Python scoring pipelines). FTRL supports binomial and multinomial classification for categorical targets, as well as regression for continuous targets. This is set to Auto (disabled) by default.
enable_rulefit¶
RuleFit Models
Specify whether to build RuleFit models as part of the experiment. Note that MOJOs are not yet supported (only Python scoring pipelines). Note that multiclass classification is not yet supported for RuleFit models. Rules are stored to text files in the experiment directory for now. This is set to Auto (disabled) by default.
enable_zero_inflated_models¶
Zero-Inflated Models
Specify whether to enable the automatic addition of zero-inflated models for regression problems with zero-inflated target values that meet certain conditions:
y >= 0, y.std() > y.mean()")
This is set to Auto by default.
enable_lightgbm_boosting_types¶
LightGBM Boosting Types
Specify which boosting types to enable for LightGBM. Select one or more of the following:
gbdt: Boosted trees
rf_early_stopping: Random Forest with early stopping
rf: Random Forest
dart: Dropout boosted trees with no early stopping
gbdt and rf are both enabled by default.
enable_lightgbm_cat_support¶
LightGBM Categorical Support
Specify whether to enable LightGBM categorical feature support. This is disabled by default.
Notes:
Only supported for CPU.
A MOJO is not built when this is enabled.
enable_lightgbm_cuda_support¶
LightGBM CUDA Support
Specify whether to enable LightGBM CUDA implementation instead of OpenCL. LightGBM CUDA is supported on Linux x86-64 environments.
show_constant_model¶
Whether to Show Constant Models in Iteration Panel
Specify whether to show constant models in the iteration panel. This is disabled by default.
params_tensorflow¶
Parameters for TensorFlow
Specify specific parameters for TensorFlow to override Driverless AI parameters. The following is an example of how the parameters can be configured:
params_tensorflow = '{'lr': 0.01, 'add_wide': False, 'add_attention': True, 'epochs': 30,
'layers': [100, 100], 'activation': 'selu', 'batch_size': 64, 'chunk_size': 1000, 'dropout': 0.3,
'strategy': 'one_shot', 'l1': 0.0, 'l2': 0.0, 'ort_loss': 0.5, 'ort_loss_tau': 0.01, 'normalize_type': 'streaming'}'
The following is an example of how layers can be configured:
[500, 500, 500], [100, 100, 100], [100, 100], [50, 50]
More information about TensorFlow parameters can be found in the Keras documentation. Different strategies for using TensorFlow parameters can be viewed here.
max_nestimators¶
Max Number of Trees/Iterations
Specify the upper limit on the number of trees (GBM) or iterations (GLM). This defaults to 3000. Depending on accuracy settings, a fraction of this limit will be used.
n_estimators_list_no_early_stopping¶
n_estimators List to Sample From for Model Mutations for Models That Do Not Use Early Stopping
For LightGBM, the dart and normal random forest modes do not use early stopping. This setting lets you specify the n_estimators (number of trees in the forest) list to sample from for model mutations for these types of models.
min_learning_rate_final¶
Minimum Learning Rate for Final Ensemble GBM Models
This value defaults to 0.01. This is the lower limit on learning rate for final ensemble GBM models.In some cases, the maximum number of trees/iterations is insufficient for the final learning rate, which can lead to no early stopping getting triggered and poor final model performance. Then, one can try increasing the learning rate by raising this minimum, or one can try increasing the maximum number of trees/iterations.
max_learning_rate_final¶
Maximum Learning Rate for Final Ensemble GBM Models
Specify the maximum (upper limit) learning rate for final ensemble GBM models. This value defaults to 0.05.
max_nestimators_feature_evolution_factor¶
Reduction Factor for Max Number of Trees/Iterations During Feature Evolution
Specify the factor by which the value specified by the max_nestimators setting is reduced for tuning and feature evolution. This option defaults to 0.2. So by default, Driverless AI will produce no more than 0.2 * 3000 trees/iterations during feature evolution.
max_abs_score_delta_train_valid¶
Max. absolute delta between training and validation scores for tree models
Modify early stopping behavior for tree-based models (LightGBM, XGBoostGBM, CatBoost) such that training score (on training data, not holdout) and validation score differ no more than this absolute value (i.e., stop adding trees once abs(train_score - valid_score) > max_abs_score_delta_train_valid). Keep in mind that the meaning of this value depends on the chosen scorer and the dataset (i.e., 0.01 for LogLoss is different than 0.01 for MSE). This option is Experimental, and only for expert use to keep model complexity low. To disable, set to 0.0. By default this option is disabled.
max_rel_score_delta_train_valid¶
Max. relative delta between training and validation scores for tree models
Modify early stopping behavior for tree-based models (LightGBM, XGBoostGBM, CatBoost) such that training score (on training data, not holdout) and validation score differ no more than this relative value (i.e., stop adding trees once abs(train_score - valid_score) > max_rel_score_delta_train_valid * abs(train_score)). Keep in mind that the meaning of this value depends on the chosen scorer and the dataset (i.e., 0.01 for LogLoss is different than 0.01 for MSE etc). This option is Experimental, and only for expert use to keep model complexity low. To disable, set to 0.0. By default this option is disabled.
min_learning_rate¶
Minimum Learning Rate for Feature Engineering GBM Models
Specify the minimum learning rate for feature engineering GBM models. This value defaults to 0.05.
max_learning_rate¶
Max Learning Rate for Tree Models
Specify the maximum learning rate for tree models during feature engineering. Higher values can speed up feature engineering but can hurt accuracy. This value defaults to 0.5.
max_epochs¶
Max Number of Epochs for TensorFlow/FTRL
When building TensorFlow or FTRL models, specify the maximum number of epochs to train models with (it might stop earlier). This value defaults to 10. This option is ignored if TensorFlow models and/or FTRL models is disabled.
max_max_depth¶
Max Tree Depth
Specify the maximum tree depth. The corresponding maximum value for max_leaves is double the specified value. This value defaults to 12.
max_max_bin¶
Max max_bin for Tree Features
Specify the maximum max_bin for tree features. This value defaults to 256.
rulefit_max_num_rules¶
Max Number of Rules for RuleFit
Specify the maximum number of rules to be used for RuleFit models. This defaults to -1, which specifies to use all rules.
ensemble_meta_learner¶
Ensemble Level for Final Modeling Pipeline
Model to combine base model predictions, for experiments that create a final pipeline consisting of multiple base models:
blender: Creates a linear blend with non-negative weights that add to 1 (blending) - recommended
extra_trees: Creates a tree model to non-linearly combine the base models (stacking) - experimental, and recommended to also set enable cross_validate_meta_learner.
fixed_ensemble_level¶
Ensemble Level for Final Modeling Pipeline
Specify one of the following ensemble levels:
-1 = auto, based upon ensemble_accuracy_switch, accuracy, size of data, etc. (Default)
0 = No ensemble, only final single model on validated iteration/tree count. Note that holdout predicted probabilities will not be available. (For more information, refer to this FAQ.)
1 = 1 model, multiple ensemble folds (cross-validation)
2 = 2 models, multiple ensemble folds (cross-validation)
3 = 3 models, multiple ensemble folds (cross-validation)
4 = 4 models, multiple ensemble folds (cross-validation)
The equivalent config.toml parameter is fixed_ensemble_level.
cross_validate_meta_learner¶
Ensemble Level for Final Modeling Pipeline
If enabled, use cross-validation to create an ensemble for the meta learner itself. Especially recommended for ensemble_meta_learner=’extra_trees’, to make unbiased training holdout predictions. No MOJO will be created if this setting is enabled. Not needed for ensemble_meta_learner=’blender’.
cross_validate_single_final_model¶
Cross-Validate Single Final Model
Driverless AI normally produces a single final model for low accuracy settings (typically, less than 5). When the Cross-validate single final model option is enabled (default for regular experiments), Driverless AI will perform cross-validation to determine optimal parameters and early stopping before training the final single modeling pipeline on the entire training data. The final pipeline will build \(N+1\) models, with N-fold cross validation for the single final model. This also creates holdout predictions for all non-time-series experiments with a single final model.
Note that the setting for this option is ignored for time-series experiments or when a validation dataset is provided.
parameter_tuning_num_models¶
Number of Models During Tuning Phase
Specify the number of models to tune during pre-evolution phase. Specify a lower value to avoid excessive tuning, or specify a higher to perform enhanced tuning. This option defaults to -1 (auto).
imbalance_sampling_method¶
Sampling Method for Imbalanced Binary Classification Problems
Specify the sampling method for imbalanced binary classification problems. This is set to off by default. Choose from the following options:
auto: sample both classes as needed, depending on data
over_under_sampling: over-sample the minority class and under-sample the majority class, depending on data
under_sampling: under-sample the majority class to reach class balance
off: do not perform any sampling
This option is closely tied with the Imbalanced Light GBM and Imbalanced XGBoost GBM models, which can be enabled/disabled on the Recipes tab under included_models. Specifically:
If this option is ENABLED (set to a value other than off) and the ImbalancedLightGBM and/or ImbalancedXGBoostGBM models are ENABLED, then Driverless AI will check your target imbalance fraction. If the target fraction proves to be above the allowed imbalance threshold, then sampling will be triggered.
If this option is ENABLED and the ImbalancedLightGBM and/or ImbalancedXGBoostGBM models are DISABLED, then no special sampling technique will be performed. The setting here will be ignored.
imbalance_sampling_threshold_min_rows_original¶
Threshold for Minimum Number of Rows in Original Training Data to Allow Imbalanced Sampling
Specify a threshold for the minimum number of rows in the original training data that allow imbalanced sampling. This value defaults to 100,000.
imbalance_ratio_sampling_threshold¶
Ratio of Majority to Minority Class for Imbalanced Binary Classification to Trigger Special Sampling Techniques (if Enabled)
For imbalanced binary classification problems, specify the ratio of majority to minority class. Special imbalanced models with sampling techniques are enabled when the ratio is equal to or greater than the specified ratio. This value defaults to 5.
heavy_imbalance_ratio_sampling_threshold¶
Ratio of Majority to Minority Class for Heavily Imbalanced Binary Classification to Only Enable Special Sampling Techniques (if Enabled)
For heavily imbalanced binary classification, specify the ratio of the majority to minority class equal and above which to enable only special imbalanced models on the full original data without upfront sampling. This value defaults to 25.
imbalance_sampling_number_of_bags¶
Number of Bags for Sampling Methods for Imbalanced Binary Classification (if Enabled)
Specify the number of bags for sampling methods for imbalanced binary classification. This value defaults to -1.
imbalance_sampling_max_number_of_bags¶
Hard Limit on Number of Bags for Sampling Methods for Imbalanced Binary Classification
Specify the limit on the number of bags for sampling methods for imbalanced binary classification. This value defaults to 10.
imbalance_sampling_max_number_of_bags_feature_evolution¶
Hard Limit on Number of Bags for Sampling Methods for Imbalanced Binary Classification During Feature Evolution Phase
Specify the limit on the number of bags for sampling methods for imbalanced binary classification. This value defaults to 3. Note that this setting only applies to shift, leakage, tuning, and feature evolution models. To limit final models, use the Hard Limit on Number of Bags for Sampling Methods for Imbalanced Binary Classification setting.
imbalance_sampling_max_multiple_data_size¶
Max Size of Data Sampled During Imbalanced Sampling
Specify the maximum size of the data sampled during imbalanced sampling in terms of the dataset’s size. This setting controls the approximate number of bags and is only active when the “Hard limit on number of bags for sampling methods for imbalanced binary classification during feature evolution phase” option is set to -1. This value defaults to 1.
imbalance_sampling_target_minority_fraction¶
Target Fraction of Minority Class After Applying Under/Over-Sampling Techniques
Specify the target fraction of a minority class after applying under/over-sampling techniques. A value of 0.5 means that models/algorithms will be given a balanced target class distribution. When starting from an extremely imbalanced original target, it can be advantageous to specify a smaller value such as 0.1 or 0.01. This value defaults to -1.
ftrl_max_interaction_terms_per_degree¶
Max Number of Automatic FTRL Interactions Terms for 2nd, 3rd, 4th order interactions terms (Each)
Samples the number of automatic FTRL interactions terms to no more than this value (for each of 2nd, 3rd, 4th order terms). This value defaults to 10000
enable_bootstrap¶
Whether to Enable Bootstrap Sampling for Validation and Test Scores
Specify whether to enable bootstrap sampling. When enabled, this setting provides error bars to validation and test scores based on the standard error of the bootstrap mean. This is enabled by default.
tensorflow_num_classes_switch¶
For Classification Problems with This Many Classes, Default to TensorFlow
Specify the number of classes above which to use TensorFlow when it is enabled. Others model that are set to Auto will not be used above this number. (Models set to On, however, are still used.) This value defaults to 10.
prediction_intervals¶
Compute Prediction Intervals
Specify whether to compute empirical prediction intervals based on holdout predictions. This is enabled by default.
prediction_intervals_alpha¶
Confidence Level for Prediction Intervals
Specify a confidence level for prediction intervals. This value defaults to 0.9.
dump_modelparams_every_scored_indiv¶
Enable detailed scored model info
Whether to dump every scored individual’s model parameters to csv/tabulated/json file produces files. For example: individual_scored.params.[txt, csv, json]