Skip to main content
Version: v0.23.x

Key concepts

This section introduces foundational terms and ideas used throughout H2O Cascade. These concepts help you understand how the platform works and why it’s designed this way.

Large language model (LLM)​

A type of neural network trained on large text datasets to perform a variety of natural language tasks. LLMs power many of the features in H2O Cascade. In most enterprise workflows, LLMs are not directly fine-tuned due to their large size and computational requirements. Instead, LLMs are often used in the Data Generation section to generate labeled training data or annotations.

Small language model (SLM)​

A compact neural network model derived from a larger LLM, typically through techniques like distillation, quantization, or pruning. In the H2O Cascade workflow, you generate or annotate data using an LLM, then use it to fine-tune an SLM for your specific domain or application. SLMs are efficient enough to be fine-tuned and deployed in production environments using your organization’s own data.

Fine-tuning​

The process of adapting a pre-trained model to a new dataset or task. The platform supports supervised fine-tuning using your own labeled data or data generated via prompt-based workflows.

LLM backbone​

The base model you’re fine-tuning. H2O Cascade supports a wide range of popular open-source models including Meta Llama, Qwen, Google Gemma, Mistral, DeepSeek, and H2O Danube. This choice impacts performance and resource requirements.

Parameters vs. hyperparameters​

  • Parameters are learned during training (like weights in the model).
  • Hyperparameters are settings you choose before training (like learning rate, batch size, or number of epochs).

You can set these manually or let AutoML or Ask KGM tune them automatically.

LoRA and quantization​

  • LoRA (Low-Rank Adaptation) is a memory-efficient way to fine-tune large models by updating only a small set of weights.
  • Quantization reduces model precision (for example, from 16-bit to 8-bit) to lower memory usage and speed up training and inference.

To use them, choose LoRA or QLoRA in the Training Mode field when you create an experiment. To set LoRA parameters, see Advanced configuration.

Evaluation metrics​

After fine-tuning, your models are evaluated using standard metrics. The available metrics depend on the problem type:

  • Perplexity: Lower is better; measures how confidently a model predicts text. Used in language modeling and multimodal tasks.
  • BLEU: Used in text generation; higher is better and measures output quality against a reference.
  • LLM-as-a-Judge: Uses a large language model to evaluate the quality of generated responses. Higher is better.
  • QA_Accuracy: Measures accuracy of question-answering outputs. Higher is better.
  • Accuracy: Measures classification correctness. Used for text and image classification.
  • AUC: Area Under the Curve; measures a classifier’s ability to distinguish between classes. Used for text and image classification.
  • Log Loss: Measures the uncertainty of classification predictions. Lower is better. Used for text and image classification.
  • mAP (Mean Average Precision): Overall detection accuracy across all IoU thresholds. Used for object detection.
  • mAP@50 / mAP@75: Precision at specific IoU thresholds (0.50 and 0.75). Used for object detection.
  • mAR (Mean Average Recall): Overall detection recall. Used for object detection.

Prompt-based data generation​

In cases where you don’t have labeled data, you can use a large model (like GPT-4) to generate rows or annotate existing data. This is managed through the Data Generation section and is useful for bootstrapping training sets.

AutoML and Ask KGM​

H2O Cascade offers two AI-driven ways to improve your experiments automatically:

  • AutoML runs a series of experiments, automatically tuning hyperparameters and backbone selection to find the best-performing configuration.
  • Ask KGM is an AI assistant that recommends a single next experiment, such as a new backbone or set of hyperparameters, based on your training results. KGM stands for Kaggle Grandmaster, a reference to H2O.ai's expert data scientists.

You can start AutoML from the AutoML tab, or use Ask KGM from the action menu of a completed experiment.

Feedback

Related content

Experiments
Data generation
AutoML