Skip to main content
Version: Next

Deployment options

Overview​

After building a model in H2O Hydrogen Torch, you have several options to deploy your trained model for scoring new data.

H2O Hydrogen Torch UI​

In the H2O Hydrogen Torch UI, you can deploy a built model to score new data. Also, through the H2O Hydrogen Torch UI, you can deploy a built model to H2O MLOps.

tutorial

Explore the following tutorial to learn how to deploy a model in the H2O Hydrogen Torch UI: Tutorial 1B: Model deployment in the H2O Hydrogen Torch UI.

Deploy in the H2O Hydrogen Torch UI​

You can score new data on built models that generate downloadable predictions through the H2O Hydrogen Torch UI. To score new data through the H2O Hydrogen Torch UI, follow these instructions:

  1. In the H2O Hydrogen Torch navigation menu, click Predict data.
  2. In the Experiment box, select the built experiment (model) you want to use to score new data.
  3. In the Prediction Name box, enter a name for your prediction.
  4. In the General, Dataset, Prediction, and Environment settings section, define the displayed settings.
    Note

    Displayed settings depend on the problem type of the selected built model (experiment). See Prediction settings to learn about the settings.

  5. Click Run predictions.
    Note
    • After running your predictions, H2O Hydrogen Torch takes you to the View predictions card, where you can view running and completed predictions. To learn more, see View a prediction.
    • To download the generated predictions in the H2O Hydrogen Torch UI, see Download a prediction.

Deploy from within the H2O Hydrogen Torch UI to H2O MLOps​

After building an H2O Hydrogen Torch model, you can deploy it to H2O MLOps utilizing the H2O Hydrogen Torch UI. To learn more, see Deploy a model to H2O MLOps through the H2O Hydrogen Torch UI.

H2O MLOps pipeline​

H2O Hydrogen Torch allows you to download an H2O MLOps scoring pipeline of a built model that you can use to score new data using the H2O MLOps REST API.

tutorial

Explore the following tutorial to learn how to deploy a model using the model's H2O MLOps pipeline: Tutorial 2B: Model deployment with a model's H2O MLOps pipeline.

Download a model's H2O MLOps Pipeline​

To download the H2O MLOps pipeline of a built model, follow these instructions:

  1. In the H2O Hydrogen Torch navigation menu, click View experiments.
  2. In the View experiments table, select the name of the experiment (model) you want to download its H2O MLOps Pipeline.
    Note
    • An H2O MLOps pipeline is only available for an experiment with a finished status. List experiments
  3. Click Download MLOps.
    Note

    The downloaded H2O MLOps pipeline, contains the following files:

    • api_pipeline.py: an example Python script demonstrating how to score new data using an MLOps API endpoint
    • model.mlflow.zip: a ZIP file container (model) ready to be uploaded to H2O MLOps for deployment
    • README.txt: a README file that contains information about the other files in the folder

Deploy a model utilizing its H2O MLOps pipeline​

Consider the following high level sequential steps to deploy a model using the model's H2O MLOps pipeline:

  1. Select a built model.
  2. Download the built model's H2O MLOps Pipeline.
  3. Deploy the MLFlow model to H2O MLOps.
    Note

    The MLFlow model comes inside the downloaded H2O MLOps pipeline (model.mlflow.zip).

  4. You can score new data by using the endpoint URL of your deploy model in H2O MLOps. The downloaded H2O MLOps pipeline includes sample code in the api_pipeline.py file.
    Note

    The received JSON response from an H2O MLOps REST API call follows the same format as the Pickle files discussed on the follwing page: Download a prediction.

  5. Monitor requests and predictions in H2O MLOps.

Python scoring pipeline​

H2O Hydrogen Torch allows you to download a Python scoring pipeline of a built model that you can use to score new data in any external Python environment.

tutorial

Explore the following tutorial to learn how to deploy a model using the model's Python scoring pipeline: Tutorial 3B: Model deployment with a model's Python scoring pipeline.

Download a model's Python scoring pipeline​

To download a model's Python scoring pipeline, follow these instructions:

  1. In the H2O Hydrogen Torch navigation menu, click View experiments.
  2. In the View experiments table, select the name of the experiment (model) you want to download its Python scoring pipeline.
    Note
    • A Python scoring pipeline is only available for experiments with a finished status. List experiments
  3. Click Download scoring.
    Note

    The downloaded Python scoring pipeline, contains the following files:

    • Dockerfile.scoring: a Dockerfile that contains all the commands to assemble an H2O Hydrogen Torch scoring image.
    • hydrogen_torch-*.whl: a wheel package containing the necessary H2O Hydrogen Torch framework functionality to generate predictions
    • scoring_pipeline.py: an example Python script demonstrating how to load the model and score new data
    • README.txt: a README file that contains information about the other files in the folder
    • checkpoint.pth: checkpoint of trained model
    • model.onnx: optional file of the model weights converted to ONNX format
    • cfg.p: internal hydrogen_torch config file
    • images: a folder containing sample images from the validation dataset
    • audios: a folder containing sample audios from the validation dataset
    • texts: a folder containing sample texts from the validation dataset

Deploy a model utilizing its Python scoring pipeline​

Consider the following high level sequential steps to deploy a model using the model's Python scoring pipeline:

  1. Select a built model.
  2. Download the model's Python scoring pipeline.
  3. Build a Docker image and start a Docker container with the H2O Hydrogen Torch environment.
    Note
    • To build it, run the following command: docker build -t ht_scoring -f Dockerfile.scoring ..
    • To run it on CPU, run the following command: docker run -it --rm --shm-size 2G ht_scoring bash.
  4. Inside a container, run the scoring_pipeline.py file (which contains sample code to score new data using your trained model weights).

Python scoring pipeline with ONNX model​

To use this pipeline, follow the instructions for the standard Python scoring pipeline described above. However, instead of running the scoring_pipeline.py file for scoring new data, you should run the onnx_scoring.py file.

Feedback