Nlp configuration

enable_tensorflow_textcnn

Enable word-based CNN TensorFlow transformers for NLP (String) (Expert Setting)

Default value 'auto'

Whether to use out-of-fold predictions of Word-based CNN TensorFlow models as transformers for NLP if TensorFlow enabled

enable_tensorflow_textbigru

Enable word-based BiGRU TensorFlow transformers for NLP (String) (Expert Setting)

Default value 'auto'

Whether to use out-of-fold predictions of Word-based Bi-GRU TensorFlow models as transformers for NLP if TensorFlow enabled

enable_tensorflow_charcnn

Enable character-based CNN TensorFlow transformers for NLP (String) (Expert Setting)

Default value 'auto'

Whether to use out-of-fold predictions of Character-level CNN TensorFlow models as transformers for NLP if TensorFlow enabled

enable_pytorch_nlp_transformer

Enable PyTorch transformers for NLP (String) (Expert Setting)

Default value 'auto'

Whether to use pretrained PyTorch models as transformers for NLP tasks. Fits a linear model on top of pretrained embeddings. Requires internet connection. Default of ‘auto’ means disabled. To enable, set to ‘on’. GPU(s) are highly recommended.Reduce string_col_as_text_min_relative_cardinality closer to 0.0 and string_col_as_text_threshold closer to 0.0 to force string column to be treated as text despite low number of uniques.

pytorch_nlp_transformer_max_rows_linear_model

Max number of rows to use for fitting the linear model on top of the pretrained embeddings. (Number) (Expert Setting)

Default value 50000

More rows can slow down the fitting process. Recommended values are less than 100000.

enable_pytorch_nlp_model

Enable PyTorch models for NLP (String) (Expert Setting)

Default value 'auto'

Whether to use pretrained PyTorch models and fine-tune them for NLP tasks. Requires internet connection. Default of ‘auto’ means disabled. To enable, set to ‘on’. These models are only using the first text column, and can be slow to train. GPU(s) are highly recommended.Set string_col_as_text_min_relative_cardinality=0.0 to force string column to be treated as text despite low number of uniques.

pytorch_nlp_pretrained_models

Select which pretrained PyTorch NLP model(s) to use. (List) (Expert Setting)

Default value ['bert-base-uncased', 'distilbert-base-uncased', 'bert-base-multilingual-cased']

Select which pretrained PyTorch NLP model(s) to use. Non-default ones might have no MOJO support. Requires internet connection. Only if PyTorch models or transformers for NLP are set to ‘on’.

tensorflow_max_epochs_nlp

Max. TensorFlow epochs for NLP (Number) (Expert Setting)

Default value 2

Max. number of epochs for TensorFlow models for making NLP features

enable_tensorflow_nlp_accuracy_switch

Accuracy above enable TensorFlow NLP by default for all models (Number) (Expert Setting)

Default value 5

Accuracy setting equal and above which will add all enabled TensorFlow NLP models below at start of experiment for text dominated problems when TensorFlow NLP transformers are set to auto. If set to on, this parameter is ignored. Otherwise, at lower accuracy, TensorFlow NLP transformations will only be created as a mutation.

tensorflow_nlp_pretrained_embeddings_file_path

Path to pretrained embeddings for TensorFlow NLP models. If empty, will train from scratch. (String) (Expert Setting)

Default value ''

Path to pretrained embeddings for TensorFlow NLP models, can be a path in local file system or an S3 location (s3://). For example, download and unzip https://nlp.stanford.edu/data/glove.6B.zip tensorflow_nlp_pretrained_embeddings_file_path = /path/on/server/to/glove.6B.300d.txt

tensorflow_nlp_pretrained_s3_access_key_id

S3 access key Id to use when tensorflow_nlp_pretrained_embeddings_file_path is set to an S3 location. (String) (Expert Setting)

Default value ''

tensorflow_nlp_pretrained_s3_secret_access_key

S3 secret access key to use when tensorflow_nlp_pretrained_embeddings_file_path is set to an S3 location. (String) (Expert Setting)

Default value ''

tensorflow_nlp_pretrained_embeddings_trainable

For TensorFlow NLP, allow training of unfrozen pretrained embeddings (in addition to fine-tuning of the rest of the graph) (Boolean) (Expert Setting)

Default value False

Allow training of all weights of the neural network graph, including the pretrained embedding layer weights. If disabled, then the embedding layer is frozen, but all other weights are still fine-tuned.

pytorch_tokenizer_parallel

pytorch_tokenizer_parallel (Boolean)

Default value True

Whether to parallelize tokenization for BERT Models/Transformers.

pytorch_nlp_fine_tuning_num_epochs

Number of epochs for fine-tuning of PyTorch NLP models. (Number) (Expert Setting)

Default value -1

Number of epochs for fine-tuning of PyTorch NLP models. Larger values can increase accuracy but take longer to train.

pytorch_nlp_fine_tuning_batch_size

Batch size for PyTorch NLP models. -1 for automatic. (Number) (Expert Setting)

Default value -1

Batch size for PyTorch NLP models. Larger models and larger batch sizes will use more memory.

pytorch_nlp_fine_tuning_padding_length

Maximum sequence length (padding length) for PyTorch NLP models. -1 for automatic. (Number) (Expert Setting)

Default value -1

Maximum sequence length (padding length) for PyTorch NLP models. Larger models and larger padding lengths will use more memory.

pytorch_nlp_pretrained_models_dir

Path to pretrained PyTorch NLP models. If empty, will get models from S3 (String) (Expert Setting)

Default value ''

Path to pretrained PyTorch NLP models. Note that this can be either a path in the local file system (/path/on/server/to/bert_models_folder), an URL or a S3 location (s3://). To get all models, download http://s3.amazonaws.com/artifacts.h2o.ai/releases/ai/h2o/pretrained/bert_models.zip and unzip and store it in a directory on the instance where DAI is installed. pytorch_nlp_pretrained_models_dir=/path/on/server/to/bert_models_folder

pytorch_nlp_pretrained_s3_access_key_id

S3 access key Id to use when pytorch_nlp_pretrained_models_dir is set to an S3 location. (String) (Expert Setting)

Default value ''

pytorch_nlp_pretrained_s3_secret_access_key

S3 secret access key to use when pytorch_nlp_pretrained_models_dir is set to an S3 location. (String) (Expert Setting)

Default value ''

text_fraction_for_text_dominated_problem

Fraction of text columns out of all features to be considered a text-dominated problem (Float) (Expert Setting)

Default value 0.3

Fraction of text columns out of all features to be considered a text-dominated problem

text_transformer_fraction_for_text_dominated_problem

Fraction of text per all transformers to trigger that text dominated (Float) (Expert Setting)

Default value 0.3

Fraction of text transformers to all transformers above which to trigger that text dominated problem

string_col_as_text_threshold

Threshold for string columns to be treated as text (0.0 - text, 1.0 - string) (Float) (Expert Setting)

Default value 0.3

Threshold for average string-is-text score as determined by internal heuristics It decides when a string column will be treated as text (for an NLP problem) or just as a standard categorical variable. Higher values will favor string columns as categoricals, lower values will favor string columns as text. Set string_col_as_text_min_relative_cardinality=0.0 to force string column to be treated as text despite low number of uniques.

string_col_as_text_threshold_preview

string_col_as_text_threshold_preview (Float)

Default value 0.1

Threshold for string columns to be treated as text during preview - should be less than string_col_as_text_threshold to allow data with first 20 rows that don’t look like text to still work for Text-only transformers (0.0 - text, 1.0 - string)

string_col_as_text_min_relative_cardinality

string_col_as_text_min_relative_cardinality (Float) (Expert Setting)

Default value 0.1

Mininum fraction of unique values for string columns to be considered as possible text (otherwise categorical)

string_col_as_text_min_absolute_cardinality

string_col_as_text_min_absolute_cardinality (Number) (Expert Setting)

Default value 10000

Mininum number of uniques for string columns to be considered as possible text (if not already)

tokenize_single_chars

Tokenize single characters. (Boolean) (Expert Setting)

Default value True

If disabled, require 2 or more alphanumeric characters for a token in Text (Count and TF/IDF) transformers, otherwise create tokens out of single alphanumeric characters. True means that ‘Street 3’ is tokenized into ‘Street’ and ‘3’, while False means that it’s tokenized into ‘Street’.

text_transformers_max_vocabulary_size

Max size of the vocabulary for text transformers. (List) (Expert Setting)

Default value [1000, 5000]

Max size (in tokens) of the vocabulary created during fitting of Tfidf/Count based text

transformers (not CNN/BERT). If multiple values are provided, will use the first one for initial models, and use remaining values during parameter tuning and feature evolution. Values smaller than 10000 are recommended for speed, and a reasonable set of choices include: 100, 1000, 5000, 10000, 50000, 100000, 500000.