Skip to main content
Version: v1.4.0

Dataset format: Image instance segmentation

H2O Hydrogen Torch supports several dataset (data) formats for an image instance segmentation experiment. Supported formats are as follows:

The data following the Hydrogen Torch format for an image instance segmentation experiment is structured as follows: A zip file (1) containing a Parquet file (2) and an image folder (3):

folder_name.zip (1)
│ └───pq_name.pq (2)
│ │
│ └───image_folder_name (3)
│ └───name_of_image.image_extension
│ └───name_of_image.image_extension
│ └───name_of_image.image_extension
│ ...
Note

You can have multiple Parquet files in the zip file that you can use as train, validation, and test dataframes:

  • A train CSV file needs to follow the format described above
  • A validation CSV file needs to follow the same format as a train CSV file
  • A test CSV file needs to follow the same format as a train CSV, but does not require a class_id and rle_mask column
  1. The available dataset connectors require the data for an image instance segmentation experiment to be in a zip file.
    Note

    To learn how to upload your zip file as your dataset in H2O Hydrogen Torch, see Dataset connectors.

  2. A Parquet file containing the following columns:
    • An image column containing the names of the images for the experiment, where each image has an image extension specified
      Note
      • Images can contain a mix of supported image extensions. To learn about supported image extensions, see Supported image extensions for image processing.
      • The names of the image files do not specify the data directory (location of the images in the zip file). You can specify the data directory (data folder) when uploading the dataset or before the dataset is used for an experiment. For more information, see Import dataset settings.
    • A class_id column containing the class names of each instance mask. Each row of the dataset should contain a list of class names, where each element in the list refers to a single mask instance.
    • A rle_mask column containing run-length-encoded (RLE) masks for each instance from the class_id column. Each row of the dataset should contain a list of RLE-encoded masks, where each element in the list refers to a single instance.
      Note

      The length of each class_id and rle_mask list must be equal while referring to the total number of instances in each respective image. If an instance is not present for a given image, all lists need to be empty.

    • An optional fold column containing cross-validation fold indexes
      Note

      The fold column can include integers (0, 1, 2, … , N-1 values or 1, 2, 3… , N values) or categorical values.

  3. An image folder that contains all the images specified in the image column; H2O Hydrogen Torch uses the images in this folder to run the image instance segmentation experiment.
    Note

    All image file names need to specify image extension. Images can contain a mix of supported image extensions. To learn about supported image extensions, see Supported image extensions for image processing.


Feedback