Skip to main content
Version: v1.1.0

Runtime management workflows

This page provides end-to-end workflows for common runtime management tasks in H2O MLOps. Each workflow describes a complete process from start to finish.

To learn more about the runtime hierarchy, see Understand runtime management.

Roll out a new scorer version using the Python SDK​

Use this workflow to add a new version of a scoring runtime image and upgrade existing deployments through the Python SDK.

Prerequisites​

Steps​

  1. Connect to H2O MLOps and get the workspace:

    import h2o_mlops

    client = h2o_mlops.Client(
       gateway_url="https://mlops.example.com",
       token_provider=lambda: "your-token"
    )
    workspace = client.workspaces.get("production")

  2. Add a new version to the existing runtime image:

    image = workspace.runtimes.images.get("restscorer-uid")
    image.versions.create(docker_image="h2oai/rest-scorer:2.2.0")

  3. Deprecate the old version:

    old_version = image.versions.get("v2.1.0")
    old_version.deprecate(reason="Superseded by v2.2.0")

  4. Upgrade all deployments to the latest version:

    for dep in workspace.deployments.list():
       deployment = workspace.deployments.get(dep["uid"])  # list() returns summaries; get() returns the full object with upgrade methods
       deployment.upgrade_runtime_image()  # Upgrades to v2.2.0 (latest)
note

The upgrade_runtime_image() call upgrades the deployment with zero downtime. If a deployment already uses the latest version, it is skipped.

Roll out a new scorer version using Helm​

Use this workflow to add a new runtime image version through the Helm chart and then upgrade existing deployments through the Python SDK.

Prerequisites​

  • Access to the Helm values.yaml file for the H2O MLOps deployment.
  • Cluster administrator permissions to run helm upgrade.
  • A connection to H2O MLOps for upgrading deployments. For instructions, see Connect to H2O MLOps.

Steps​

  1. Edit values.yaml to add the new version and deprecate the old version:

    runtimeImages:
     h2oai-h2o3mojo-scorer:
       displayName: "H2O-3 MOJO Scorer"
       state: "ACTIVE"
       versions:
         - image:
             repository: "h2oai-h2o3mojo-scorer"
             tag: "v1.5.4"
           version: "v1.5.4"
           state: "DEPRECATED"
           stateReason: "Superseded by v1.6.0"
         - image:
             repository: "h2oai-h2o3mojo-scorer"
             tag: "v1.6.0"
           version: "v1.6.0"
           state: "ACTIVE"

  2. Deploy the updated Helm chart:

    helm upgrade h2oai-mlops ./charts/h2oai-mlops -f values.yaml

    The new version is automatically applied when the deployment server restarts.

  3. Upgrade existing deployments to the new version using the Python SDK:

    import h2o_mlops

    client = h2o_mlops.Client(
       gateway_url="https://mlops.example.com",
       token_provider=lambda: "your-token"
    )
    workspace = client.workspaces.get("production")

    for dep in workspace.deployments.list():
       deployment = workspace.deployments.get(dep["uid"])  # list() returns summaries; get() returns the full object with upgrade methods
       deployment.upgrade_runtime_image()
note

Adding a new version through Helm does not automatically upgrade existing deployments. You must explicitly upgrade them using the Python SDK.

Roll back to a previous runtime image version​

H2O MLOps does not provide a single rollback command. However, you can manually revert deployments to a previous runtime image version by disabling the problematic version, re-activating the previous version, and upgrading each deployment individually.

Prerequisites​

  • A connection to H2O MLOps. For instructions, see Connect to H2O MLOps.
  • The UID of the runtime image that contains the problematic version.

Steps​

  1. Connect to H2O MLOps and retrieve the runtime image:

    import h2o_mlops

    client = h2o_mlops.Client(
       gateway_url="https://mlops.example.com",
       token_provider=lambda: "your-token"
    )
    workspace = client.workspaces.get("production")
    image = workspace.runtimes.images.get("image-uid")

  2. Disable the problematic version:

    bad_version = image.versions.get("v2.2.0")
    bad_version.disable(reason="Regression in scoring accuracy")

  3. Re-activate the previous version if it was deprecated:

    good_version = image.versions.get("v2.1.0")
    good_version.activate()

  4. Upgrade each deployment to the now-current version:

    for dep in workspace.deployments.list():
       deployment = workspace.deployments.get(dep["uid"])  # list() returns summaries; get() returns the full object with upgrade methods
       deployment.upgrade_runtime_image()
caution

Disabling a runtime image version does not automatically stop deployments that use it. You must explicitly upgrade each deployment in step 4. Plan runtime version changes carefully and test new versions in a staging environment before rolling them out to production.

Create a custom runtime for a new model type​

Use this workflow to create a custom scoring runtime for a model type that is not covered by the built-in runtimes.

Prerequisites​

  • A connection to H2O MLOps. For instructions, see Connect to H2O MLOps.
  • A custom scorer container image available in a container registry accessible from your Kubernetes cluster.

Steps​

  1. Connect to H2O MLOps and get the workspace:

    import h2o_mlops

    client = h2o_mlops.Client(
       gateway_url="https://mlops.example.com",
       token_provider=lambda: "your-token"
    )
    workspace = client.workspaces.get("production")

  2. Create a new runtime image:

    image = workspace.runtimes.images.create(
       name="Custom TensorFlow Scorer",
       docker_image="myregistry.io/tf-scorer:1.0.0"
    )

  3. Create a runtime that uses this image:

    runtime = workspace.runtimes.create(
       name="TensorFlow Runtime",
       runtime_image=image,
       model_type="tensorflow_savedmodel",
       environment_variables={"TF_NUM_THREADS": "4"}
    )

The runtime is now available for new deployments of the tensorflow_savedmodel type.

For detailed information on each SDK method used in these workflows, see:

Feedback