Skip to main content

Key concepts

This section introduces foundational terms and ideas used throughout H2O Enterprise LLM Studio. These concepts help you understand how the platform works and why it’s designed this way.

Large Language Model (LLM)​

A type of neural network trained on large text datasets to perform a variety of natural language tasks. LLMs power many of the features in Enterprise LLM Studio. In most enterprise workflows, LLMs are not directly fine-tuned due to their large size and computational requirements. Instead, LLMs are often used in the Data Generation section to generate labeled training data or annotations.

Small Language Model (SLM)​

A compact neural network model derived from a larger LLM, typically through techniques like distillation, quantization, or pruning. In the Enterprise LLM Studio workflow, you generate or annotate data using an LLM, then use it to fine-tune an SLM for your specific domain or application. SLMs are efficient enough to be fine-tuned and deployed in production environments using your organization’s own data.

Fine-tuning​

The process of adapting a pre-trained model to a new dataset or task. The platform supports supervised fine-tuning using your own labeled data or with data generated via prompt-based workflows.

LLM Backbone​

The base model you're fine-tuning. You can choose from various backbones (e.g., Mistral, Falcon, Danube) depending on what your deployment supports. This choice impacts performance and resource requirements.

Parameters vs. Hyperparameters​

  • Parameters are learned during training (like weights in the model).
  • Hyperparameters are settings you choose before training (like learning rate, batch size, or number of epochs).

Enterprise users can set these manually or let AutoML or Ask KGM tune them automatically.

LoRA and Quantization​

  • LoRA (Low-Rank Adaptation) is a memory-efficient way to fine-tune large models by updating only a small set of weights.
  • Quantization reduces model precision (e.g., from 16-bit to 8-bit) to lower memory usage and speed up training and inference.

These techniques are used behind the scenes in the “Advanced Configuration” section of the Experiments UI.

Evaluation Metrics​

After fine-tuning, your models are evaluated using standard metrics. For example:

  • Perplexity: Lower is better; measures how confidently a model predicts text.
  • BLEU: Used in text generation; higher is better and measures output quality against a reference.

Prompt-Based Data Generation​

In cases where you don’t have labeled data, you can use a large model (like GPT-4) to generate rows or annotate existing data. This is managed through the Data Generation section and is useful for bootstrapping training sets.

AutoML and Ask KGM​

Enterprise LLM Studio includes intelligent agents that can:

  • Automatically tune hyperparameters
  • Choose the best backbone model
  • Iterate on experiments to improve performance

You can start these from the AutoML tab or invoke Ask KGM during experiment setup.

Feedback