Scheduled target switching for endpoints

An endpoint gives a deployment a stable URL. Scheduled target switching changes which deployment sits behind that URL on a recurring weekly schedule, so callers keep scoring the same path while the deployment underneath changes.

Cost drives most of these schedules. Route the endpoint to a GPU deployment during business hours when latency matters, and to a cheaper CPU deployment overnight. Nothing changes for the caller.

This page covers endpoint schedules. To schedule replica counts on a single deployment, see Scheduled scaling for deployments. To scale a deployment to zero while idle and wake it on the next request, see On-demand mode.

For how the scheduling features relate to each other, see Scheduling overview. For administrator settings, workspace defaults, and permissions, see Configure scheduling and on-demand mode.

Prerequisites

• An endpoint. See Configure endpoint.
• At least one deployment to route to. Every window needs a target deployment in the same workspace as the endpoint.

Default target and effective target

An endpoint has a default target: the deployment it routes to when no window is active. Set it with endpoint.update(target_deployment=...). See Manage endpoint.

The effective target names the deployment the endpoint routes to right now. A schedule never overwrites the default target. It layers an override on top of it for as long as a window is active.

Endpoint state	Effective target
A window is active	That window's target deployment
No window is active, and the endpoint has a default target	The default target
No window is active, and the endpoint has no default target	None. The endpoint becomes detached and serves nothing

Give an endpoint a default target unless you intend it to serve nothing between windows.

In the Python client, endpoint.target_deployment returns the default target and endpoint.effective_target_deployment returns the effective target.

Window order and overlaps

H2O MLOps evaluates windows in list order, top to bottom. When two or more windows are active at the same time, the first active window in the list wins. Targets are not additive: exactly one deployment sits behind the endpoint at any moment.

Reorder the list to change priority. In the UI, use the move up and move down actions on a window row. In the Python client, the order of the list you pass to update(windows=[...]) is the priority order.

caution

Endpoint windows and deployment windows resolve overlap differently. Scheduled scaling picks the window with the highest desired_replicas. Scheduled target switching picks the first active window in list order. Do not carry the deployment rule across: position in the list decides the winner here, not the value of the target.

Health checks at window boundaries

A switch onto a window's target happens only when that deployment is healthy.

If the target is unhealthy when the window opens, the endpoint keeps its current target: the previous window's target, or the default. H2O MLOps retries on each evaluation, so the endpoint moves across as soon as the target reports healthy and the window is still active. A window whose target never becomes healthy never takes effect.

A window whose target deployment you paused or deleted counts as unhealthy, so that window never routes traffic and the endpoint holds its current target. Removing a deployment that a schedule still names degrades the schedule quietly rather than breaking the endpoint. Keep the windows in step with the deployments they name.

The check runs at the switch, not throughout the window. Once the endpoint moves onto a window target, it stays there even if that deployment turns unhealthy mid-window.

Returning to the default target carries no health check. When a window ends, the endpoint leaves the window target on schedule, even if the default is unhealthy. If you deleted the default target, the endpoint detaches and serves nothing once the window ends.

Bring the target deployment up before its window opens. A deployment that is still starting at the boundary keeps traffic on the old target until it passes its health check, which delays the switch for the length of the startup.

When changes take effect

• Evaluation runs once a minute. Window boundaries are not instantaneous. A switch lands within about a minute of the boundary.
• Changing the default target during an active window does not disturb live routing. The endpoint keeps serving the window's target, and the new default takes effect when the window ends.
• Disabling a schedule during an active window drops the override immediately. The endpoint returns to its default target, or detaches if it has none. The schedule never strands it on the last window's target.
• Disabling keeps the windows. They stay stored but unenforced until you enable the schedule again.

Configure scheduled target switching in the UI

To create a schedule:

1. Click Endpoints on the left navigation menu.
2. Find the endpoint's row and open its action menu.
3. Click Schedule. The Scheduled target switching side sheet opens.
4. Turn on Enable scheduled target switching.
5. Click Add window.
6. Enter a Name for the window. The name must be unique within the schedule. The dialog suggests one from the times you pick.
7. Select a Target deployment. The list holds the deployments in the endpoint's workspace.
8. Set when the window opens and closes:
   • In Simple mode, pick Days, Opens at, and Closes at.
   • In Cron mode, enter Opens (cron) and Closes (cron).
9. Select a Timezone.
10. Click Add.
11. Repeat steps 5 to 10 for each window.
12. Set the priority order. Use the move up and move down actions on a window row, because the first active window in the list wins.
13. Check the Weekly preview. It draws one track per window across the week, plus an Effective target row showing which deployment wins at each point. Times appear in your local time zone, and the preview flags any overlap between windows.
14. Click Save schedule.

note

Without permission to update endpoint schedules, the action menu reads View schedule and the sheet opens read-only.

Schedule states in the endpoints list

The Schedule column shows a pill for every endpoint that has a schedule. Endpoints without one show nothing.

Pill	Meaning
Active	The schedule is on, and a window routes the endpoint right now.
Enabled	The schedule is on, but no window is active right now.
Disabled	The endpoint has windows, but the schedule is off.

Point to the pill to see the windows and the deployment each one routes to.

Remove a window or a schedule

To remove a single window, open the schedule sheet, point to the window row, and click the remove action. Confirm at Remove window?. The removal takes effect when you click Save schedule.

To remove the whole schedule, click Remove schedule and confirm at Remove endpoint schedule?. The confirmation states what happens to the endpoint next:

• With a default target, the endpoint returns to it.
• Without one, the endpoint detaches and the path serves nothing after the removal.

Schedule status on a deployment page

Open a deployment and select the Endpoints tab. Any endpoint with a schedule carries a one-line note saying whether a window is active. When a window routes the endpoint to a different deployment, the note names that deployment and links to it.

Click Manage schedule in the note to open the schedule sheet for that endpoint.

Configure scheduled target switching with the Python client

Set the default target

Set the deployment the endpoint routes to outside every window:

cpu_deployment = workspace.deployments.get(uid="48ece40f-8608-473a-92a6-388e164e9952")
endpoint = workspace.endpoints.get(uid="6ab2b92e-98c0-4c38-b4ef-4f6d9ba66e6f")

endpoint.update(target_deployment=cpu_deployment)

Add a target window

Route the endpoint to a GPU deployment during weekday business hours:

from h2o_mlops import options

gpu_deployment = workspace.deployments.get(uid="9f21c7d4-52ab-4f0e-9c3e-1b7a6d0e4c88")

endpoint.schedule.update(
   enabled=True,
   windows=[
       options.TargetWindowOptions(
           name="weekday-gpu",
           start_cron="0 9 * * 1-5",
           end_cron="0 17 * * 1-5",
           target_deployment=gpu_deployment,
           timezone="Europe/Berlin",
       ),
   ],
)

The endpoint now routes to gpu_deployment from 09:00 to 17:00 Monday to Friday, Berlin time, and to cpu_deployment at all other times.

note

target_deployment takes a deployment object, not a name or a UID. Retrieve the deployment with workspace.deployments.get() first.

update() applies a partial update. Pass windows to replace the window list, pass enabled to turn the schedule on or off, and omit either one to leave it unchanged.

Passing windows=[] on its own succeeds even on a schedule you left enabled, because H2O MLOps checks for windows only when you set enabled in the same call. The schedule stays enabled with nothing to enforce, and the endpoint routes to its default target until you add a window.

Check where an endpoint routes

Input:

print("Default:", endpoint.target_deployment.name)
print("Effective:", endpoint.effective_target_deployment.name)
print("Active window:", endpoint.schedule.active_window)

Output:

Default: churn-scorer-cpu
Effective: churn-scorer-gpu
Active window: weekday-gpu

Each property returns None when it has nothing to report: target_deployment when the endpoint has no default target, effective_target_deployment when the endpoint detaches, and active_window when no window is active.

Schedule properties and methods

endpoint.schedule exposes the following:

• enabled: Whether the schedule is on.
• windows: The configured windows as options.TargetWindowOptions, in priority order.
• active_window: The name of the window governing the target right now, or None.
• active_target_deployment: The deployment the schedule routes to right now, or None when no window is active.
• last_modified_time: When the schedule last changed.
• refresh(): Re-read the schedule from the server.
• clear(): Remove every window and turn the schedule off.

note

Read properties fetch the schedule on first access and then cache it. Call refresh() to pick up a change made elsewhere, such as an edit in the UI or a window opening.

Two properties look alike but sit on different objects. endpoint.schedule.active_target_deployment reports what the schedule alone resolves to, and returns None between windows. endpoint.effective_target_deployment reports what the endpoint serves, falling back to the default target in endpoint.target_deployment.

Disable or remove a schedule

To turn the schedule off and keep the windows:

endpoint.schedule.update(enabled=False)

To remove the schedule:

endpoint.schedule.clear()

clear() removes every window and turns the schedule off, which deletes it from the server. In both cases the endpoint returns to its default target, or detaches if it has none.

Window rules

Every target window must meet these rules:

• name: Required. Must be unique within the schedule.
• target_deployment: Required. The deployment must exist in the same workspace as the endpoint.
• start_cron and end_cron: Required. Both take five-field cron expressions covering minute, hour, day of month, month, and day of week. Neither takes a seconds field.
• start_cron and end_cron must differ from each other.
• timezone: An IANA time zone name such as Europe/Berlin. Defaults to UTC.
• Each cron expression must fire at least once within the next 31 days. H2O MLOps measures the horizon from the moment you save, so it rejects an expression such as 0 0 29 2 *, which fires only on 29 February, unless you save the schedule within 31 days of that date.

A window is active whenever its start_cron fired more recently than its end_cron. You cannot enable a schedule that has no windows, because it has nothing to switch to.

note

Deployment scaling windows carry three restrictions that do not apply to endpoint target windows: they cannot cross midnight, their day fields must match, and scale-to-zero windows cannot restrict the day of month or the month. Those limits come from the way deployment windows drive replica scaling.

An endpoint target window can cross midnight. For an overnight window, set start_cron="0 22 * * *" and end_cron="0 6 * * *".

For the deployment-side rules, see Scheduled scaling for deployments.

---

Feedback

• Submit and view feedback for this page
• Send feedback about H2O MLOps to cloud-feedback@h2o.ai