Skip to main content
Version: v1.7.0

Deployment options

Overview​

After building a model in H2O Hydrogen Torch, you have four options to deploy your trained model for scoring new data.

H2O Hydrogen Torch UI​

In the H2O Hydrogen Torch UI, you can deploy a built model to score new data. You can also upload a built model to H2O MLOps from the H2O Hydrogen Torch UI.

tutorial

Explore the following tutorial to learn how to deploy a model in the H2O Hydrogen Torch UI: Tutorial 1B: Model deployment in the H2O Hydrogen Torch UI.

Deploy in the H2O Hydrogen Torch UI​

You can score new data on built models that generate downloadable predictions through the H2O Hydrogen Torch UI. To score new data through the H2O Hydrogen Torch UI, follow these instructions:

  1. Open the predict data card

    In the H2O Hydrogen Torch navigation menu, click Predict data.

  2. Select the built model

    In the Experiment box, select the built experiment (model) you want to use to score new data.

  3. Name the prediction

    In the Prediction name box, enter a name for your prediction.

  4. Define the displayed settings

    In the General, Dataset, Prediction, and Environment settings section, define the displayed settings.

    Note

    Displayed settings depend on the problem type of the selected built model (experiment). See Prediction settings to learn about the settings.

  5. Run the predictions

    Click Run predictions.

    Note
    • After running your predictions, H2O Hydrogen Torch takes you to the View predictions card, where you can view running and completed predictions. To learn more, see View a prediction.
    • To download the generated predictions in the H2O Hydrogen Torch UI, see Download a prediction.

Upload a model to H2O MLOps from the H2O Hydrogen Torch UI​

After building an H2O Hydrogen Torch model, you can upload it to H2O MLOps from the H2O Hydrogen Torch UI. Deployments are created and managed in the H2O MLOps UI. To learn more, see Deploy a model to H2O MLOps through the H2O Hydrogen Torch UI.

H2O MLOps pipeline​

H2O Hydrogen Torch lets you download an H2O MLOps scoring pipeline of a built model that you can use to score new data using the H2O MLOps REST API.

tutorial

Explore the following tutorial to learn how to deploy a model using the model's H2O MLOps pipeline: Tutorial 2B: Model deployment with a model's H2O MLOps pipeline.

Download a model's H2O MLOps pipeline​

To download the H2O MLOps pipeline of a built model, follow these instructions:

  1. Open the experiments list

    In the H2O Hydrogen Torch navigation menu, click View experiments.

  2. Select the built model

    In the View experiments table, click the name of the experiment (model) whose H2O MLOps pipeline you want to download.

    Note
    • An H2O MLOps pipeline is available only for an experiment with a Finished or Uploaded status. List experiments
  3. Download the H2O MLOps pipeline

    Click Download MLOps.

    Note

    The downloaded H2O MLOps pipeline contains the following files:

    • api_pipeline.py: an example Python script demonstrating how to score new data using an MLOps API endpoint
    • model.mlflow.zip: a ZIP file container (model) ready to be uploaded to H2O MLOps for deployment
    • README.txt: a README file that contains information about the other files in the folder

Deploy a model using its H2O MLOps pipeline​

Consider the following high-level sequential steps to deploy a model using the model's H2O MLOps pipeline:

  1. Select a built model

    Select a built model.

  2. Download the H2O MLOps pipeline

    Download the built model's H2O MLOps pipeline.

  3. Deploy the MLflow model

    Deploy the MLflow model to H2O MLOps.

    Note

    The MLflow model comes inside the downloaded H2O MLOps pipeline (model.mlflow.zip).

  4. Score new data

    You can score new data using the endpoint URL of your deployed model in H2O MLOps. The downloaded H2O MLOps pipeline includes sample code in the api_pipeline.py file.

    Note

    The received JSON response from an H2O MLOps REST API call follows the same format as the Pickle files discussed on the follwing page: Download a prediction.

    The sample code in api_pipeline.py calls json.loads() on the scored value, which assumes the JSON-encoded string output format. If the experiment was built with the Native MLOps output training setting turned on, remove that call: the scored value is already a JSON object.

  5. Monitor the deployment

    Monitor requests and predictions in H2O MLOps.

Python scoring pipeline​

H2O Hydrogen Torch lets you download a Python scoring pipeline of a built model that you can use to score new data in any external Python environment.

tutorial

Explore the following tutorial to learn how to deploy a model using the model's Python scoring pipeline: Tutorial 3B: Model deployment with a model's Python scoring pipeline.

Download a model's Python scoring pipeline​

To download a model's Python scoring pipeline, follow these instructions:

  1. Open the experiments list

    In the H2O Hydrogen Torch navigation menu, click View experiments.

  2. Select the built model

    In the View experiments table, click the name of the experiment (model) whose Python scoring pipeline you want to download.

    Note
    • A Python scoring pipeline is available only for an experiment with a Finished or Uploaded status. List experiments
  3. Download the Python scoring pipeline

    Click Download scoring.

    Note

    The downloaded Python scoring pipeline contains the following files:

    • Dockerfile.scoring: a Dockerfile that contains all the commands to assemble an H2O Hydrogen Torch scoring image.
    • hydrogen_torch-*.whl: a wheel package containing the necessary H2O Hydrogen Torch framework functionality to generate predictions
    • scoring_pipeline.py: an example Python script demonstrating how to load the model and score new data
    • README.txt: a README file that contains information about the other files in the folder
    • checkpoint.pth: checkpoint of trained model
    • model.onnx: optional file of the model weights converted to ONNX format
    • cfg.p: internal hydrogen_torch config file
    • images: a folder containing sample images from the validation dataset
    • audios: a folder containing sample audios from the validation dataset
    • texts: a folder containing sample texts from the validation dataset

Deploy a model using its Python scoring pipeline​

Consider the following high-level sequential steps to deploy a model using the model's Python scoring pipeline:

  1. Select a built model

    Select a built model.

  2. Download the Python scoring pipeline

    Download the model's Python scoring pipeline.

  3. Build and run the Docker container

    Build a Docker image and start a Docker container with the H2O Hydrogen Torch environment.

    Note
    • To build it, run the following command: docker build -t ht_scoring -f Dockerfile.scoring ..
    • To run it on CPU, run the following command: docker run -it --rm --shm-size 2G ht_scoring bash.
  4. Run the scoring pipeline

    Inside a container, run the scoring_pipeline.py file (which contains sample code to score new data using your trained model weights).

Python scoring pipeline with ONNX model​

To use this pipeline, follow the instructions in Python scoring pipeline. However, instead of running the scoring_pipeline.py file to score new data, run the onnx_scoring.py file.

Feedback

Related content

Deploy a model to H2O MLOps through the H2O Hydrogen Torch UI
Download an experiment's H2O MLOps pipeline
Download an experiment's Python scoring pipeline