Helm configuration for runtimes

note

This page is for platform administrators who manage H2O MLOps through Helm. If you manage runtimes through the Python SDK, see Manage runtime images and Manage runtimes.

Platform administrators define and manage scoring runtimes through the Helm chart's values.yaml file. The chart renders these values into an internal configuration file at deploy time, but you only need to edit values.yaml.

Runtimes configured through Helm are bootstrap-managed: H2O MLOps applies them automatically when the deployment server starts.

To learn more about the runtime hierarchy, see Understand runtime management.

note

Bootstrap-managed runtimes cannot be deleted through the Python SDK or API. Removing a runtime from values.yaml and redeploying the Helm chart disables the runtime rather than deleting it.

Runtime images

Define runtime images under config.deployer.composition.runtimeImages in values.yaml. Each runtime image is a logical grouping that can contain multiple versioned container images.

config:
  deployer:
    composition:
      runtimeImages:
        h2oai-h2o3mojo-scorer:                       # Unique runtime image ID
          displayName: "H2O-3 MOJO Scorer"            # Display name in UI
          description: "Scoring using H2O-3 MOJO model"
          state: "ACTIVE"                             # ACTIVE, DEPRECATED, or DISABLED
          stateReason: ""                             # Optional reason for the state
          versions:
            - image:
                repository: "h2oai-h2o3mojo-scorer"   # Docker image repository
                tag: "v1.5.4"                         # Docker image tag
                registry: ""                          # Optional: override global registry
              version: "v1.5.4"                       # Optional: explicit semantic version
              state: "ACTIVE"                         # Per-version state
              stateReason: ""                         # Optional reason

note

If the version field is omitted, the system extracts the version from the Docker image tag automatically. You only need to specify version explicitly when the tag is not a valid semantic version (for example, latest or nightly).

Multiple versions

A runtime image can contain multiple versions, representing successive releases of the same scorer. Use separate runtime images for different configurations such as Python versions (for example, mlflow-scorer-py310 and mlflow-scorer-py311).

h2oai-mlflow-scorer-py310:
  displayName: "MLflow Scorer (Python 3.10)"
  description: "MLflow model scoring runtime for Python 3.10"
  state: "ACTIVE"
  versions:
    - image:
        repository: "h2oai-modelscoring-mlflowscorer-py310"
        tag: "v1.5.3"
      state: "DEPRECATED"
    - image:
        repository: "h2oai-modelscoring-mlflowscorer-py310"
        tag: "v1.5.4"
      state: "ACTIVE"

Runtimes

Define runtimes under config.deployer.composition.runtimes in values.yaml. Each runtime ties a runtime image to a model type and adds deployment configuration such as environment variables, volumes, and security settings.

config:
  deployer:
    composition:
      runtimes:
        dai_mojo_scorer:                                # Unique runtime ID
          enabled: true
          displayName: "DAI MOJO Scorer"
          description: "Runtime for DAI MOJO models"
          modelType: "dai_mojo"                         # Model type this runtime handles
          runtimeImage: "h2oai-modelscoring-daimojoscorer"  # References runtimeImages key
          writableVolume: true                           # Whether /data volume is writable
          scoringEndpoint: "v1"                          # Scoring API endpoint version
          state: "ACTIVE"                                # ACTIVE, DEPRECATED, or DISABLED
          stateReason: ""
          envs:                                          # Environment variables
            H2O_SCORER_PORT: "8080"
            H2O_SCORER_LOADER_PATH: "dai_mojo_scorer.mojo2:Loader"
            H2O_SCORER_GPU_COUNT: "{GPU_COUNT}"          # Dynamic placeholder
          volumes:                                       # Optional runtime-specific volumes
            ephemeral-storage:
              type: "ephemeral"
              writable: true
              properties:
                size: "1Gi"
              paths: ["/tmp"]
          containerSecurityContext: {}                    # Optional security context

Global settings

Configure cluster-wide defaults that apply to all runtimes. Global settings control image pull secrets, default volumes, and default resource limits.

Image pull secrets and registry allowlist

Runtime images can only be pulled from registries explicitly configured in the Helm chart. Images from unlisted registries are rejected when creating deployments or runtime image versions.

The global registry (global.imageRegistry) is always included with its pull secrets and always-attach=true. To allow images from additional registries, add them under config.deployer.config.runtime.registries:

global:
  imageRegistry: "registry.example.com"
  imagePullSecrets:
    - "registry-secret"

config:
  deployer:
    config:
      runtime:
        registries:
          - name: "ghcr.io"
            secrets: ["ghcr-secret"]
          - name: "123456789.dkr.ecr.us-east-1.amazonaws.com"
            secrets: ["ecr-secret"]

Field	Description
name	Exact registry hostname to allow.
secrets	List of Kubernetes secret names for authenticating with this registry.

The global registry secrets are attached to all pods (always-attach), while additional registry secrets are attached only to pods using images from that specific registry.

caution

Every registry you reference in runtime images — including public ones — must be listed either as the global registry or in registries. Images from unlisted registries are rejected.

Default volumes

Define default volumes that are mounted to all runtimes under global.runtimes.defaultVolumes:

global:
  runtimes:
    defaultVolumes:
      tmp-dir:
        type: "emptyDir"
        writable: true
        paths: ["/tmp"]

Individual runtimes can override these defaults by specifying their own volumes configuration.

Default resource limits

Set default resource requests and limits for all runtime containers under config.deployer.config.runtime:

config:
  deployer:
    config:
      runtime:
        logLevel: "INFO"
        pullPolicy: "IfNotPresent"
        dataVolume:
          type: "emptyDir"
          properties: {}
        resources:
          requests:
            cpu: "500m"
            memory: "256Mi"
          limits:
            cpu: ""
            memory: ""

Field	Description
logLevel	Default log level for runtime containers.
pullPolicy	Kubernetes image pull policy (Always, IfNotPresent, or Never).
dataVolume	Default data volume configuration.
resources.requests	Minimum CPU and memory Kubernetes reserves for each runtime container.
resources.limits	Maximum CPU and memory each runtime container can use. An empty string ("") means no limit.

Add a new runtime image version

To add a new version of a runtime image:

1. Edit values.yaml and add the new version to the appropriate runtime image. Optionally, deprecate the old version:

   runtimeImages:
     h2oai-h2o3mojo-scorer:
       displayName: "H2O-3 MOJO Scorer"
       state: "ACTIVE"
       versions:
         - image:
             repository: "h2oai-h2o3mojo-scorer"
             tag: "v1.5.4"
           version: "v1.5.4"
           state: "DEPRECATED"
           stateReason: "Superseded by v1.6.0"
         - image:
             repository: "h2oai-h2o3mojo-scorer"
             tag: "v1.6.0"
           version: "v1.6.0"
           state: "ACTIVE"

2. Deploy the Helm chart:

   # Adjust the chart path for your installation method
   helm upgrade h2oai-mlops ./charts/h2oai-mlops -f values.yaml

   The new version is automatically applied when the deployment server restarts.

note

Existing deployments are not affected by adding a new version. They continue using their current version. Use the Python SDK to upgrade deployments to the new version. For instructions, see Manage runtimes: Upgrade runtime images on existing deployments.

Deprecate or disable versions

To deprecate or disable a version, update the state and stateReason fields in values.yaml and redeploy:

versions:
  - image:
      repository: "h2oai-h2o3mojo-scorer"
      tag: "v1.5.4"
    version: "v1.5.4"
    state: "DISABLED"
    stateReason: "Security vulnerability CVE-2024-12345"
  - image:
      repository: "h2oai-h2o3mojo-scorer"
      tag: "v1.6.0"
    version: "v1.6.0"
    state: "ACTIVE"

For a complete end-to-end workflow, see Runtime management workflows.

---

Feedback

• Submit and view feedback for this page
• Send feedback about H2O MLOps to cloud-feedback@h2o.ai