Manage runtimes

This page describes how to create, view, update, and delete runtimes, as well as how to upgrade runtime images on existing deployments using the H2O MLOps Python client.

To learn more about the runtime hierarchy, see Understand runtime management.

Prerequisites

Before you begin:

1. Import the necessary Python packages. For instructions, see Step 1: Import the required packages.
2. Connect to H2O MLOps. For instructions, see Connect to H2O MLOps.
3. Retrieve a workspace. For instructions, see Retrieve a workspace. If you do not have a workspace, see Create a workspace.
4. Create or retrieve a runtime image. For instructions, see Manage runtime images.

Create a runtime

Create a runtime within the workspace using the create() method. A runtime binds a runtime image to a model type and adds deployment configuration.

runtime = workspace.runtimes.create(
   name="My H2O-3 Runtime",
   runtime_image=image,    # MLOpsRuntimeImage object
   model_type="h2o3",
   description="Runtime for H2O-3 models",
   environment_variables={"GPU_COUNT": "0", "LOG_LEVEL": "INFO"},
   read_only_data_volume=True
)

caution

The model_type and runtime_image fields are immutable after creation. To change the model type or runtime image for a runtime, delete the existing runtime and create a new one.

View runtimes

Count runtimes

Get the total number of runtimes in a workspace:

Input:

workspace.runtimes.count()

Output:

5

List runtimes

List all runtimes in a workspace:

Input:

runtimes = workspace.runtimes.list()
runtimes

Output:

   | name               | uid
---+--------------------+--------------------------------------
 0 | My H2O-3 Runtime   | b2c3d4e5-f6a7-8901-bcde-f12345678901
 1 | DAI MOJO Runtime   | c3d4e5f6-a7b8-9012-cdef-123456789012

note

• The list() method returns a formatted table. By default, the table shows the first 50 rows.

• Calling len(runtimes) returns the total number of rows, not just the number currently displayed.

• To customize the number of rows displayed, call the show() method with the n argument. For example:

  runtimes.show(n=100)

  This displays up to 100 runtimes.

• The runtimes object supports iteration.

Filter runtimes

Use the list() method with key-value arguments to filter runtimes.

Filter by model type:

Input:

workspace.runtimes.list(model_type="h2o3")

Output:

   | name               | uid
---+--------------------+--------------------------------------
 0 | My H2O-3 Runtime   | b2c3d4e5-f6a7-8901-bcde-f12345678901

Retrieve a runtime

Retrieve a runtime by UID:

runtime = workspace.runtimes.get("runtime-uid")

List runtimes across all workspaces

Administrators can list runtimes across all workspaces:

client.runtimes.list()

Runtime properties

A runtime has the following main properties:

• uid: The unique identifier for the runtime.
• name: The display name.
• model_type: The model type this runtime handles.
• state: The current state (ACTIVE, DEPRECATED, or DISABLED).
• runtime_image: The associated runtime image.
• environment_variables: A dictionary of environment variables.
• managed_by_bootstrap: Whether this runtime is managed through Helm bootstrap configuration.
• scoring_endpoint: The scoring API endpoint version.

Update a runtime

Make sure to retrieve the runtime before updating it. See Retrieve a runtime.

runtime.update(
   name="Updated Runtime Name",
   environment_variables={"GPU_COUNT": "1", "LOG_LEVEL": "DEBUG"}
)

caution

Bootstrap-managed runtimes cannot be updated through the SDK. If runtime.managed_by_bootstrap is True, the update call returns an error. To modify a bootstrap-managed runtime, edit the Helm values.yaml file and redeploy the Helm chart. For details, see Helm configuration for runtimes.

note

The update() method accepts name, description, environment_variables, read_only_data_volume, volumes, security_context, and selector. To change model_type or runtime_image, delete the runtime and create a new one.

Runtime state management

Change the state of a runtime:

runtime = workspace.runtimes.get("runtime-uid")

# Deprecate
runtime.deprecate(reason="Migrating to new runtime")

# Disable
runtime.disable(reason="End of life")

# Re-enable
runtime.enable()

caution

For bootstrap-managed runtimes, state changes follow a ceiling rule: the SDK cannot re-enable a runtime beyond the state set by the platform, and vice versa. For example, if the platform disabled a runtime, calling runtime.enable() returns an error. For details, see State constraints between management modes.

note

Runtimes and runtime images use enable() to re-enable, while runtime image versions use activate(). For details, see Version state management.

Delete a runtime

warning

Only API-managed runtimes can be deleted. Bootstrap-managed runtimes cannot be deleted through the SDK.

runtime.delete()

Upgrade runtime images on existing deployments

When a new version of a runtime image is available, existing deployments are not automatically updated. You must explicitly upgrade them. The upgrade is applied with zero downtime.

Upgrade all compositions to the latest version

Upgrade all compositions in a deployment to their latest available runtime image version:

deployment = workspace.deployments.get("deployment-uid")
deployment.upgrade_runtime_image()

Upgrade to a specific runtime image

For a single-composition deployment, upgrade to a specific runtime image (uses its current/latest version):

deployment = workspace.deployments.get("deployment-uid")
image = workspace.runtimes.images.get("image-uid")

deployment.upgrade_runtime_image(image)

Or upgrade to a specific version:

version = image.versions.get("v2.1.0")
deployment.upgrade_runtime_image(version)

Upgrade multi-composition deployments

For deployments with multiple experiments or compositions, provide a dictionary that maps experiment IDs to target images:

deployment = workspace.deployments.get("deployment-uid")

# Upgrade specific experiments to specific images
deployment.upgrade_runtime_image({
   "experiment-id-1": workspace.runtimes.images.get("image-a"),
   "experiment-id-2": workspace.runtimes.images.get("image-b"),
})

Use None to upgrade a specific experiment to the latest available version:

deployment.upgrade_runtime_image({
   "experiment-id-1": workspace.runtimes.images.get("image-a"),
   "experiment-id-2": None  # Upgrade to latest
})

Use the wildcard key * to set a default for all experiments not explicitly listed:

# Apply a specific image to one experiment, upgrade all others to latest
deployment.upgrade_runtime_image({
   "experiment-id-1": workspace.runtimes.images.get("special-image"),
   "*": None  # All other experiments upgrade to latest
})

# Apply the same image to all experiments
deployment.upgrade_runtime_image({
   "*": workspace.runtimes.images.get("standard-image")
})

What happens during an upgrade

When you call upgrade_runtime_image():

1. H2O MLOps determines the target runtime image version for each experiment.
2. H2O MLOps updates the container image for each composition. Compositions that already use the target version are skipped.
3. The deployment continues serving requests during the upgrade (zero downtime).

---

Feedback

• Submit and view feedback for this page
• Send feedback about H2O MLOps to cloud-feedback@h2o.ai