Configure scheduling and on-demand mode

note

This page is for platform administrators who install and operate H2O MLOps. To put a deployment or an endpoint on a schedule, see Scheduled scaling for deployments and Scheduled target switching for endpoints.

As a platform administrator you control four things: which engine applies deployment scaling windows, whether on-demand mode is available at all, which schedule presets users can start from, and what new deployments inherit in each workspace.

You configure them through the H2O MLOps Helm chart, under config.deploymentServer in values.yaml. The on-demand timeout bounds are the exception; they stay at fixed server defaults the chart does not expose.

For what these features do, see Scheduling overview.

Enable the scaling engine

config.deploymentServer.keda.enabled chooses which component applies scaling windows to deployments. It defaults to false and takes effect only when the platform-level KEDA component is also on:

global:
  components:
    keda:
      enabled: true

config:
  deploymentServer:
    keda:
      enabled: true

• With KEDA off, the deployment server evaluates windows itself on a fixed 60-second cycle and sets replicas directly.
• With KEDA on, KEDA owns deployment replicas. The server compiles each deployment's windows into the cron triggers of a KEDA ScaledObject and stops running its own evaluation cycle.

Either engine produces the same effective replica count from the same windows. Enable KEDA if you intend to offer on-demand mode, which depends on it.

This setting does not affect endpoint schedules. The deployment server always evaluates them on its own 60-second cycle.

caution

Turning KEDA off on a server that has run with it enabled is an offline operation. KEDA does not remove the ScaledObject resources the server created, so you have to delete them by hand as part of the same maintenance window.

To remove them:

kubectl delete scaledobjects -n <mlops-namespace> -l app.kubernetes.io/managed-by=mlops-deployment-server

Set -n to the namespace H2O MLOps runs in. Without it, kubectl uses the namespace your current context points at, where the command matches nothing.

Enable on-demand mode

On-demand mode rests a deployment at zero replicas while idle and wakes it to its configured replica count on the next request. It defaults to off and has two prerequisites:

• Install the KEDA HTTP add-on, which provides the interceptor proxy and the external scaler.
• Enable the scaling engine. The server fails fast at startup if you enable on-demand mode without it.

The server does no runtime detection of the add-on. Install the add-on first, then enable on-demand mode:

config:
  deploymentServer:
    keda:
      enabled: true
      onDemand:
        enabled: true

When the scaling engine and on-demand mode are both enabled, the chart also routes on-demand deployments through the interceptor proxy automatically; you do not configure that route by hand.

The server locates the interceptor and scaler through built-in defaults that match the add-on's standard install. If your add-on install diverges from those defaults (for example, a different namespace), override the coordinates under config.deploymentServer.keda.onDemand:

Value	Description
interceptorServiceName	Service name of the interceptor proxy that receives traffic for idle on-demand deployments.
interceptorServiceNamespace	The namespace the interceptor proxy runs in.
interceptorServicePort	Interceptor proxy service port.
scalerAddress	Address (host:port) of the external scaler.
hostSuffix	Host suffix used to build the synthetic per-deployment host the interceptor keys on.

Each value applies only when set. Left empty, the server falls back to its built-in default.

The server also bounds and defaults the per-deployment timeouts. The chart does not expose values for these settings, which stay at the server defaults in this release:

Setting	Default	Description
Default idle timeout	30m	Idle time before a deployment scales to zero, for deployments that do not set their own.
Default cold start timeout	30s	How long the first request after an idle period waits while the deployment wakes, for deployments that do not set their own.
Lowest idle timeout	5m	Users cannot set an idle timeout below this.
Highest cold start timeout	2m	Users cannot set a cold start timeout longer than this.

While on-demand mode is off, enabling it for a deployment fails with FAILED_PRECONDITION and the message on-demand mode is not enabled on this server.

For what on-demand mode does to a running deployment, see On-demand mode.

Define schedule presets

A preset is a named set of scaling windows that users apply as a starting point for a deployment schedule. Define presets under config.deploymentServer.scheduling.presets, keyed by preset name:

config:
  deploymentServer:
    scheduling:
      presets:
        weekday-business-hours:
          windows:
            - name: weekday-daytime
              startCron: "0 8 * * 1-5"
              endCron: "0 19 * * 1-5"
              desiredReplicas: 3
              timezone: "America/New_York"
            - name: weekday-evening
              startCron: "0 19 * * 1-5"
              endCron: "0 23 * * 1-5"
              desiredReplicas: 1
              timezone: "America/New_York"

Every window requires name, startCron, and endCron; the chart refuses to render a window that omits one. timezone is optional and defaults to UTC.

The server validates every preset at startup and names the offending preset if one fails. A cron expression the server cannot parse, an unknown timezone, a duplicate window name within a preset, or a window that exceeds deployer.kubernetes.deployment-max-replicas stops the server from starting. Presets never load partially, so one bad preset takes the server down rather than reaching users half-configured.

Applying a preset copies its windows into the deployment's schedule. The schedule does not stay linked to the preset: editing a preset later, or removing it from the configuration, leaves schedules already created from it untouched. The preset name recorded on a schedule is informational.

Users list the presets you define with mlops.configs.deployment_schedule_presets. For how they apply one, see Scheduled scaling for deployments.

Set workspace defaults

Two workspace annotations give a new deployment a schedule or an on-demand configuration at creation time. They apply only to deployments created after you set the annotation.

Annotation	Accepted values
mlops.h2o.ai/deployment-schedule	preset:<name>, a JSON array of windows, or disabled
mlops.h2o.ai/deployment-on-demand	enabled, a JSON object of timeout overrides, or disabled

Each annotation resolves independently, in this order:

1. The annotation on the deployment's own workspace.
2. The annotation on the global workspace.
3. No default.

The literal value disabled is an opt-out sentinel. It ends the lookup at that level with no default, which is how a single workspace escapes a default set on the global workspace. Any other unrecognized value fails deployment creation, so keep the value exact.

An on-demand annotation on a server where on-demand mode is off is a special case: the server skips it, logs the reason, and creates the deployment anyway. A default is a convenience, so it degrades rather than failing every create while the feature is off.

Set a workspace default schedule

Use set_default_schedule() with an explicit list of windows:

from h2o_mlops import options

workspace = mlops.workspaces.list(name="fraud-scoring")[0]
workspace.set_default_schedule(
   windows=[
       options.ScheduledWindowOptions(
           name="weekday-daytime",
           start_cron="0 8 * * 1-5",
           end_cron="0 19 * * 1-5",
           desired_replicas=3,
           timezone="America/New_York",
       ),
   ],
)

To remove the default and keep every other annotation on the workspace:

workspace.clear_default_schedule()

note

The client does not yet have a matching helper for the on-demand default. Until one exists, set mlops.h2o.ai/deployment-on-demand through the workspace annotations directly, and merge it into the annotations the workspace already carries. workspace.update(annotations=...) replaces the whole annotation map rather than patching it.

For workspace basics, see Manage workspaces.

Permissions

The following permissions govern scheduling and on-demand configuration:

Action	Permission
Read deployment schedule	actions/mlops/deploymentSchedules/GET
Create, update, or delete deployment schedule	actions/mlops/deploymentSchedules/UPDATE
Read on-demand configuration	actions/mlops/deploymentOnDemand/GET
Update on-demand configuration	actions/mlops/deploymentOnDemand/UPDATE
Read endpoint schedule	actions/mlops/endpoints/GET
Update endpoint schedule	actions/mlops/endpoints/UPDATE

The UI follows these permissions. Without the GET permission it hides the Schedule tab and the Schedule column; with GET but not UPDATE it renders the schedule read-only.

API reference

The following operations manage schedules and on-demand configuration:

Operation	Method and path	Update mask
Get deployment schedule	GET /v2/workspaces/{ws}/deployments/{dep}/schedule
Update deployment schedule	PATCH /v2/workspaces/{ws}/deployments/{dep}/schedule	enabled,preset,windows
List schedule presets	GET /v2/deploymentSchedulePresets
Get endpoint schedule	GET /v2/workspaces/{ws}/endpoints/{ep}/schedule
Update endpoint schedule	PATCH /v2/workspaces/{ws}/endpoints/{ep}/schedule	enabled,windows
Get on-demand configuration	GET /v2/workspaces/{ws}/deployments/{dep}/onDemand
Update on-demand configuration	PATCH /v2/workspaces/{ws}/deployments/{dep}/onDemand	enabled,idle_timeout,cold_start_timeout

Each Get returns 200 even when the resource does not exist. An empty response means the deployment or endpoint has no schedule or on-demand configuration, not that the request failed.

---

Feedback

• Submit and view feedback for this page
• Send feedback about H2O MLOps to cloud-feedback@h2o.ai