Scheduling overview
H2O MLOps gives you three ways to control when a deployment uses compute and where an endpoint sends traffic:
- Scheduled scaling changes a deployment's replica count during recurring time windows.
- Scheduled target switching changes which deployment an endpoint routes to during recurring time windows.
- On-demand mode rests a deployment at zero replicas while nothing calls it, then wakes it on the next request.
Scheduled scaling and scheduled target switching share one model: the cron-driven window. On-demand mode has no windows and reacts to traffic instead, but it combines with scheduled scaling on the same deployment.
Scheduling concepts
Windows apply to scheduled scaling and scheduled target switching. On-demand mode uses neither windows nor cron.
What a window is
A window is a named interval with two five-field cron expressions, start_cron and end_cron, and an IANA time zone that defaults to UTC. The five cron fields are minute, hour, day of month, month, and day of week.
What a window does while active depends on the capability. A scaling window holds a replica count. A target window holds the deployment an endpoint routes to.
When a window is active
A window is active if and only if its start_cron fired more recently than its end_cron.
H2O MLOps stores no window state. It recomputes the answer from the two cron expressions every time it evaluates the schedule, which has two consequences worth knowing:
- A window whose
start_cronhas never fired is inactive. - A window whose
start_cronhas fired but whoseend_cronhas not stays active.
One schedule per deployment or endpoint
A schedule is a sub-resource of the deployment or endpoint it belongs to, and each one gets at most a single schedule. A missing schedule means the deployment or endpoint has none, not that it has an empty one. Clearing every window and turning the schedule off deletes it.
Turn a schedule on or off
A schedule takes effect only when you enable it and it holds at least one window. You cannot enable a schedule that has no windows. H2O MLOps rejects the attempt with the error an enabled schedule must have at least one window.
Turning a schedule off does not discard it. H2O MLOps keeps the windows and stops enforcing them, so you can turn scheduling off and back on without rebuilding it.
Scheduled scaling
While one of a deployment's scaling windows is active, the deployment runs at that window's desired_replicas. Outside every window it falls back to its configured replica count, the number of replicas you set when you create a deployment. A window with desired_replicas set to 0 scales the deployment to zero for as long as that window stays active.
For the full setup, see Scheduled scaling for deployments.
Scheduled target switching
An endpoint has a default target: the deployment it routes to when no window is active. While one of its target windows is active, the endpoint routes to that window's target instead.
The endpoint's effective target is whatever it routes to at a given moment. An endpoint with no default target and no active window detaches and serves nothing.
For the full setup, see Scheduled target switching for endpoints.
On-demand mode
With on-demand mode enabled, a deployment scales to zero after a stretch with no requests, then wakes to its full configured replica count when the next request arrives. H2O MLOps holds that first request while the workload starts, then forwards it. On-demand mode has no UI: configure it through the H2O MLOps API or the Python client.
For the full setup, see On-demand mode.
An administrator enables on-demand mode for the H2O MLOps server. Until then, requests to configure it fail with a failed-precondition error. See Configure scheduling and on-demand mode.
Choose an approach
The following table maps common goals to the capability that fits:
| Goal | Capability |
|---|---|
| Run more replicas during known busy hours and fewer outside them | Scheduled scaling |
| Scale a deployment to zero during known quiet hours | Scheduled scaling, with a window whose desired_replicas is 0 |
| Send an endpoint's traffic to a different deployment during a known period | Scheduled target switching |
| Scale to zero based on idleness rather than the clock | On-demand mode |
| Keep a deployment warm during known busy hours and let idleness handle everything else | On-demand mode together with scheduled scaling |
Enabling on-demand mode and scheduled scaling on the same deployment changes what a scaling window means. Every window becomes a keep-warm floor that can raise the replica count but never lower it, because scaling down on idleness stays the job of on-demand mode. For the rule H2O MLOps enforces, see Combine on-demand mode with scheduled scaling.
Shared behavior
- Window boundaries are not instantaneous. A window can open or close up to a minute after its cron time, so avoid scheduling for changes that have to land at an exact second. H2O MLOps evaluates endpoint schedules on a 60-second cycle. It evaluates deployment schedules on the same cycle unless your administrator enabled KEDA, in which case KEDA applies the windows as cron triggers.
- Each window carries its own time zone. The time zone defaults to
UTC. H2O MLOps evaluates the cron expressions in the IANA zone you name, so a window that opens at 09:00 local time keeps opening at 09:00 local time across a daylight saving change. - Pausing a deployment suspends its schedule and its on-demand configuration. H2O MLOps holds both rather than discarding them, so resuming the deployment restores what you set. While a deployment stays paused, H2O MLOps rejects changes to either configuration.
When more than one window is active
The two capabilities resolve an overlap differently.
| Capability | Rule |
|---|---|
| Scheduled scaling | The highest desired_replicas among the active windows wins. |
| Scheduled target switching | The first active window in list order wins. H2O MLOps never combines targets. |
Reordering the windows on an endpoint schedule can change where traffic goes. Reordering the windows on a deployment schedule changes nothing.
- Submit and view feedback for this page
- Send feedback about H2O MLOps to cloud-feedback@h2o.ai