Skip to main content
Version: v0.4.0

Specify an annotation task rubric

Overview

After creating a new annotation task, specify an annotation task rubric in the Rubric tab. An annotation task rubric refers to the labels (for example, object classes) to use when annotating a dataset. For example, after creating a new annotation task for an object detection dataset, you have to specify the object classes to use when labeling the dataset in the annotation task rubric.

Instructions

An annotation task rubric differs based on the specified task type of the dataset used to create the annotation task.

Text annotation tasks

  • Instructions: Specify one or more categorical target labels for a text classification task rubric.

  • Example: To specify happy and unhappy as labels, one can consider the following instructions in the Rubric tab of the annotation task:

    note

    To learn how to access the Rubric tab of an annotation task (or other tabs), see Access an annotation task's tabs.

    1. In the New class name box, enter happy.
    2. Click Add.
    3. Click Add class.
    4. In the New class name box, enter unhappy.
    5. Click Add.

Text classification annotation task

Image annotation tasks

For an image classification task rubric, you need specify one or more categorical target labels in the annotation task rubric for an image classification annotation task. To learn more, see Tutorial 1B: Annotation task: Image classification

  • Instructions: Specify one or more categorical target labels.

  • Example: To create a car and coffee label, one can consider the following instructions in the Rubric tab of the annotation task:

    note

    To learn how to access the Rubric tab of an annotation task (or other tabs), see Access an annotation task's tabs.

    1. In the New class name box, enter car.
    2. Click Add.
    3. Click Add class.
    4. In the New class name box, enter coffee.
    5. Click Add.

Image classification rubric

Audio annotation tasks

  • Instructions: Specify one or more categorical target labels.

  • Example: To specify chainsaw and clock_tick as labels, one can consider the following instructions in the Rubric tab of the annotation task:

    note

    To learn how to access the Rubric tab of an annotation task (or other tabs), see Access an annotation task's tabs.

    1. In the New class name box, enter chainsaw.
    2. Click Add.
    3. Click Add class.
    4. In the New class name box, enter clock_tick.
    5. Click Add.

Audio classification annotation task rubric

Large language model (LLM) parameters

Select model family

Defines the zero-shot learning model family (large language model (LLM)) to utilize for the text-generative AI annotation task.

Options

  • h2oGPT
    • This option enables available h2oGPT large language models (LLMs) to utilize for a text-generative AI annotation task
  • h2oGPT Custom
    • This option enables available h2oGPT large language models (LLMs) for a text-generative AI annotation task
  • OpenAI
    • This option enables available OpenAI large language models (LLMs) in your OpenAI account for a text-generative AI annotation task. To connect to your OpenAI LLMs, see OpenAI API settings

LLM model name

Defines the zero-shot learning model name (large language model (LLM)) to utilize for a text-generative AI annotation task.

Max response tokens

Defines the maximum number of tokens for a response; a low number can result in short responses, which might limit the responses.

Temperature

Defines the randomness of predictions by scaling the logits. Higher temperature values increase creativity on the part of the model while producing more diverse outputs. In other words, the temperature makes the distribution more random.

Repetition penalty

Defines the penalty value of tokens frequently reappearing in the text (response). For example, a token that has already appeared ten times can be penalized more than a token that has appeared only two times. A 1.0 value means no penalty.

tip

This setting can be helpful when attempting to reduce the model's tendency to generate verbatim/identical text.

Prompt template

Select example prompt

This setting defines the input format for the selected model. There are several options, including the option to create your own custom input format (custom).


Feedback