Skip to main content
Version: v0.4.0

Tutorial 3A: Text-entity recognition annotation task

Overview​

This tutorial describes the process of creating a text-entity recognition annotation task, including specifying an annotation task rubric for it. To highlight the process, we will annotate a dataset containing user reviews (in text format) and ratings (from 0 to 5) of Amazon products.

Step 1: Explore dataset​

We are going to use the preloaded Amazon reviews demo dataset for this tutorial. The dataset contains 180 samples (text), each containing a review of an Amazon product. Let's quickly explore the dataset.

  1. On the H2O Label Genie navigation menu, click Datasets.
  2. In the Datasets table, click amazon-reviews-demo.

Step 2: Create an annotation task​

Now that we understand the dataset, let's create an annotation task that enables you to annotate the dataset. For this tutorial, the annotation task refers to a text-entity recognition annotation task locating and classifying named entities in unstructured text into pre-defined categories.

  1. Click New annotation task.
  2. In the Task name box, enter tutorial-3a.
  3. In the Task description box, enter Annotate a dataset containing reviews from Amazon products.
  4. In the Select task list, select Entity recognition.
  5. In the Select text column box, select comment.
  6. Click Create task.

Step 3: Specify an annotation task rubric​

Before we can start annotating our dataset, we need to specify an annotation task rubric. An annotation task rubric refers to the labels (for example, object classes) you want to use when annotating your dataset. For our dataset, let's define the following two entities: Product and Emotion.

Product refers to the Amazon product reviewed, while Emotion refers to one or several expressed (written) feelings during the review.

  1. In the New object name box, enter product.
  2. Click Add.
  3. Click Add entity.
  4. In the New object name box, enter emotion.
  5. Click Continue to annotate.

Specifying objects for the annotation task

Step 4: Annotate dataset​

In the Annotate tab, you can individually annotate each review (text) in the dataset. Let's annotate the first review.

  1. Let's start by annotating the product entities in the review. Highlight Charge 5.
  2. Now, let's annotate the review's emotional (emotion) entities. Click Emotion.
  3. Highlight I'm disappointed.
    Note

    You can attribute a particular entity (Product or Emotion) to a word by clicking it.

  4. Highlight I'm not impressed. annotated-review-3-a.png
  5. Click Save and next.
    Note
    • Save and next saves the annotated review
    • To skip a review to annotate later: Click Skip.
      • Skipped reviews (samples) reappear after all non-skipped reviews are annotated
  6. Annotate all dataset samples.
    note

    At any point in an annotation task, you can download the already annotated (approved) samples. You do not need to fully annotate a dataset to download already annotated samples. To learn more, see Download an annotated dataset.

Summary​

In this tutorial, we learned how to annotate and specify an annotation task rubric for a text-entity recognition annotation task.

Next​

To learn the process of annotating and specifying an annotation task rubric for other various annotation tasks in computer vision (CV), natural language processing (NLP), and audio, see Tutorials.

Feedback