Skip to main content
Version: v0.24.x

Token classification

Token classification assigns a label to every token in a sequence, for example named entity recognition (NER) that tags person, organization, and location spans. Token classification is an optional feature. If Token Classification isn't listed as a problem type when you upload a dataset, ask your administrator to enable it.

Dataset format​

Your dataset must contain matching input and label columns of equal length, one label per token:

  • Parquet files may store the input and label columns as native list cells (a list of strings per row).
  • CSV and JSON files store the same data as a JSON-encoded list string in each cell — for example ["Alice", "works", "at", "Acme"] and ["B-PER", "O", "O", "B-ORG"].
  • Labels must be strings, not integers.
  • Every label that appears in the validation set must also appear somewhere in the training set.

A Named Entity Recognition demo dataset (synthetic sentences with IOB2 PER/ORG/LOC tags) is available on the dataset creation page's demo list.

Experiment settings​

Select a Train Dataset whose problem type is Token Classification. You set the problem type when you upload the dataset. The experiment form asks for:

  • Input column / Output column — the token and label list columns described above.
  • Model — the encoder backbone to fine-tune, for example google-bert/bert-base-cased.
  • Metric — The single metric to optimize. Other problem types let you select several.
    • Micro F1 (MICRO_F1_SCORE, default)
    • Macro F1 (MACRO_F1_SCORE)
    • CoNLL Micro F1 (CONLL_MICRO_F1_SCORE)
    • CoNLL Macro F1 (CONLL_MACRO_F1_SCORE)

Advanced configuration​

The Advanced Configuration editor accepts YAML for the dataset, tokenizer, architecture, training, prediction, logging, augmentation, and environment sections, merged over the defaults. Most experiments only change training, tokenizer, architecture, and prediction. For example:

training:
epochs: 2
learning_rate: 0.00001
batch_size: 16

tokenizer:
max_length: 256

architecture:
dropout: 0

Settings that Cascade manages can't be overridden: the problem type, experiment name, model backbone and pretrained weights, dataset paths, columns and validation strategy, training fold, optimization metric, GPU settings, and output location. Keys that start with an underscore are also rejected. If you set any of these keys, Cascade rejects the experiment when you start it, and the error names the key.

Cascade doesn't check for unknown keys or values of the wrong type until training starts. If your YAML contains one, the experiment fails right after it starts, and the System Logs name the key.

During and after training​

While the experiment trains, the experiment page shows live training logs. Metrics and training and validation curves appear only after training finishes.

Download the MLOps package​

Token classification experiments cannot be deployed via Model Zoo. Instead, click Download MLOps package on the experiment page to download a zip archive with an MLflow-compatible model package. You can serve this package outside H2O Cascade, for example in H2O MLOps. See When to use Model Zoo or H2O MLOps.

Known limitations​

  • Not deployable in Model Zoo. Download the MLOps package and serve it separately.
  • No AutoML. You can't use a token classification dataset in AutoML or start AutoML from a token classification experiment.
  • Single fold. Training always uses fold 0; there is no cross-validation.
  • Backbone checkpoint format. Backbones whose checkpoints ship only .bin weights (no safetensors) may fail to load.

Feedback