Monitoring options
This page describes how to configure monitoring options for new and existing deployments.
Connect to H2O MLOps
import h2o_mlops
import time
mlops = h2o_mlops.Client()
New deployments
Set up the prerequisites for creating a deployment.
project = mlops.projects.create(name="demo")
experiment = project.experiments.create(
data="/Users/jgranados/models/mojo.zip",
name="experiment-from-client"
)
model = project.models.create(name="model-from-client")
model.register(experiment=experiment)
environment = project.environments.list(name="DEV")[0]
scoring_runtime = mlops.runtimes.scoring.list(
artifact_type="dai_mojo_pipeline",
uid="dai_mojo_runtime",
)[0]
Configure the options you want for monitoring
- When
enable
isFalse
, H2O MLOps does not store any information about scoring requests and the other options have no effect. - When
enable
isTrue
, H2O MLOps will always capture operational data such as request count and scoring latency. - When
enable
andsave_scoring_inputs
areTrue
, the data sent to the model is stored and used for further evaluation such as detecting data drift.
monitoring_options = h2o_mlops.options.MonitoringOptions(
enable=True,
save_scoring_inputs=True
)
Pass the monitoring options to the deployment creation method
deployment = environment.deployments.create_single(
name = "deployment-from-client",
model = model,
scoring_runtime = scoring_runtime,
monitoring_options = monitoring_options
)
while not deployment.is_healthy():
deployment.raise_for_failure()
time.sleep(5)
deployment.status()
Output:
'HEALTHY'
View the current configured options for the deployment
print(deployment.monitoring_options)
Output:
enable: True
save_scoring_inputs: True
Existing deployments
If you already have a deployment, you can use the update_monitoring_options
method to change it. Note that the deployment will report itself as UNHEALTHY
until the updated options have taken effect.
deployment.update_monitoring_options(enable=False)
while not deployment.is_healthy():
deployment.raise_for_failure()
time.sleep(5)
deployment.status()
Output:
'HEALTHY'
Confirm that the current configured options for the deployment have been updated
print(deployment.monitoring_options)
Output:
enable: False
save_scoring_inputs: True
Feedback
- Submit and view feedback for this page
- Send feedback about H2O MLOps to cloud-feedback@h2o.ai