Skip to main content
Version: v1.1.0

Understand runtime management

H2O MLOps uses a runtime management system to control which container images serve your ML models. This system lets you manage runtime images, track versions, and perform controlled rollouts across deployments.

Runtime hierarchy​

The runtime management system uses a three-tier hierarchy:

TierDescriptionExample
Runtime ImageA logical grouping of container images that represent a scorer family."H2O-3 MOJO Scorer"
Runtime Image VersionA specific versioned container image within a Runtime Image.h2oai-h2o3mojo-scorer:v1.5.4
RuntimeA deployable configuration that ties a Runtime Image to a model type, adding environment variables, volumes, and security settings."DAI MOJO Scorer" runtime for dai_mojo models

When you create a deployment, H2O MLOps selects a Runtime based on the model type. The Runtime references a Runtime Image, and the system resolves the appropriate Runtime Image Version (the actual container image) to run.

State lifecycle​

Each entity in the hierarchy (Runtime Image, Runtime Image Version, and Runtime) has a state that controls its availability:

StateMeaning
ACTIVEAvailable for use in new deployments and upgrades.
DEPRECATEDStill usable, but discouraged. Serves as a warning before disabling.
DISABLEDCannot be used for new deployments or upgrades.

States can transition in both directions:

ACTIVE <--> DEPRECATED <--> DISABLED

You can restrict a resource at any time (for example, move from ACTIVE to DEPRECATED, or from DEPRECATED to DISABLED). You can also re-enable a resource (for example, move from DISABLED to DEPRECATED).

caution

Disabling a runtime image version does not automatically stop existing deployments that use it. However, disabled versions cannot be selected for new deployments or upgrades.

State constraints between management modes​

When a runtime resource is managed by both bootstrap and the API (for example, a bootstrap-managed image whose state was changed via the SDK), a ceiling rule prevents either source from silently overriding the other:

  • Either source can always restrict a resource (move toward DISABLED).
  • Neither source can re-enable a resource beyond the most permissive state set by the other source.
ScenarioResult
Platform bootstraps a resource as ACTIVE; user deprecates it via SDKAllowed. On next bootstrap, the platform cannot re-enable it — the user's change is preserved.
Platform disables a resource (removed from Helm config); user calls enable()Blocked. The user cannot override the platform restriction.
User disables a resource; user later calls enable()Allowed only if the platform's ceiling also permits ACTIVE.

This rule applies uniformly to runtime images, runtime image versions, and runtimes. It ensures that manual state changes made through the SDK are not lost during Helm upgrades, and platform restrictions cannot be bypassed through the API.

Management modes​

Runtime resources can be managed in two ways:

Bootstrap-managed runtimes​

Bootstrap-managed runtimes are defined in the Helm chart values.yaml and automatically applied when the platform starts. These runtimes:

  • Are automatically created or updated during Helm install or upgrade.
  • Cannot be deleted through the Python SDK or API.
  • Are the standard way to manage the built-in set of runtimes that ship with the platform.

For configuration details, see Helm configuration for runtimes.

API-managed runtimes​

API-managed runtimes are created and managed entirely through the Python SDK. These runtimes:

  • Support full create, read, update, and delete operations.
  • Are independent of Helm configuration.
  • Are best suited for custom runtime images specific to a workspace or team.

For SDK usage details, see Manage runtime images and Manage runtimes.

note

Bootstrap-managed runtimes cannot be deleted through the Python SDK or API. You can change their state (for example, deprecate or disable) using the SDK, but you cannot delete them. To fully remove a bootstrap-managed runtime, update the Helm values.yaml and redeploy.

To learn more about runtime management, see the following pages:

Feedback