Skip to main content
Version: v1.1.0

Configure monitoring

This guide explains how to enable and configure model monitoring during deployment using the H2O MLOps UI.

Step 1: Enable model monitoring​

To enable model monitoring for your deployment:

  1. In the left navigation panel, click Real-time deployments.
  2. Click Create deployment. Create deployment
  3. On the Create new deployment page, click Advanced settings. Click advanced settings
  4. Toggle Enable monitoring to Yes. Enable monitoring

Step 2: Configure and deploy​

During model deployment, configure monitoring to collect and analyze data.

  1. If Kafka is available in your environment, provide a pre-created Kafka topic where raw data will be sent. For more information, see Raw data export to Kafka. Kafka topic
  2. Select the columns you want to monitor. Column selection
  3. Provide baseline data for comparison:
    • Numerical features: Numerical baseline data
    • Categorical features: Categorical baseline data
    • Text features: Text columns are monitored using character length distributions and vocabulary analysis. For details, see Text column feature monitoring. Text baseline data
  4. Click Deploy.

Step 3: Start scoring​

Once the deployment is Healthy, you can begin scoring.

  1. In the left navigation panel, click Real-time deployments.
  2. Select the deployment you created. Select deployment
  3. Go to the Quick scoring tab.
  4. Click Score. Select quick scoring and click Score

Step 4: View aggregated data​

To view the scoring aggregates for each monitored column:

  1. After scoring completes, go to the Monitoring tab.
  2. Wait 3–5 minutes to see data under SCORING AGGREGATES. Monitoring tab

Step 5: Open the monitoring UI​

To view and analyze model drift:

  1. Click View in monitoring UI. View in monitoring UI

  2. The Superset UI opens.

  3. From the SQL drop-down, select SQL Lab. SQL Lab

  4. Schema and table selection:

    • Schema name is the Workspace ID from the H2O AI Cloud Workspaces UI. Copy the Workspace ID and select that schema in Superset. See Workspaces in H2O AI Cloud.
    • Table name is the Deployment ID. From the MLOps Deployments page, copy the deployment ID (it appears in the default scorer URL). In Superset, select the table named aggregates_<deployment_id>.
    • Alternatively, in SQL Lab, use the dropdowns labeled Schema and See table schema to choose the workspace (schema) and aggregates_<deployment_id> (table). Superset schema and table selectors
Schema access control

In Superset, you can only see and query schemas for workspaces you have access to. If a schema is not visible in the dropdown, contact your administrator to verify your workspace permissions.

If you attempt to query a schema you do not have access to, you will receive a permission error listing the unauthorized schema names. This means your account has not been granted access to that workspace. Contact your administrator to request workspace access.

After completing these steps, you can proceed to analyze drift in Superset.

Configure monitoring with the Python client​

To learn how to configure monitoring for your deployment using the H2O MLOps Python client, see Monitoring setup.

Feedback