Raw data export to Kafka
Monitoring supports exporting raw scoring data to Kafka. This feature allows you to process scoring data in ways required by your internal regulations. Request and response data from scoring operations can be sent to a specified Kafka topic for downstream processing, auditing, or debugging. The MLOps administrator enables this feature.
During model deployment with monitoring enabled, scoring data and response data are sent to a default topic configured by the MLOps administrator. Users can optionally specify a custom Kafka topic where the data is sent for this particular deployment. This allows separating data streams per deployment for improved observability. This configuration has no effect when the Kafka integration is disabled by the administrator.
The following image shows the Kafka topic configuration during deployment setup:
The custom Kafka topic must exist before deploying the model. Monitoring will not attempt to create the topic automatically.
Once configured, the monitoring captures raw request and response data from scoring operations and forwards it to the configured Kafka topic (global or deployment-specific).
Common use cases
- Debugging and inspecting raw scoring payloads
- Auditing input/output for compliance
- Real-time analytics via stream processing systems
- Submit and view feedback for this page
- Send feedback about H2O MLOps to cloud-feedback@h2o.ai