Skip to content

Third-party model support

In H2O MLOps, the Bring Your Own Model (BYOM) feature lets you upload and deploy third-party models. The following sections describe this feature.

Supported third-party models

The following is a list of tested and supported third-party Python models.

  • fastai==2.7.10
  • gluon==1.1.0
  • keras==2.10.0
  • lightgbm==3.3.3
  • mlflow==1.29.0
  • onnx==1.12.0
  • scikit-learn==1.1.3
  • statsmodels==0.13.5
  • tensorflow==2.10.1
  • torch==1.12.1
  • torchvision==0.13.1
  • xgboost==1.7.1

Understanding BYOM

Experiments in H2O MLOps

In MLOps, an experiment is defined as the output of a training job. Many different experiments can be rapidly created by modifying specific parameters and hyperparameters. Experiments can be imported from Driverless AI, H2O-3 open source, MLflow, or as a serialized Python file. Before being deployed, imported experiments must first be registered as a model version.

For more information, see Key terms.

Experiment metadata

Each experiment in the H2O.ai Storage can have multiple key-value pairs attached to it. These values are not interpreted by the storage itself but can be interpreted by the clients or client services that access data in the Storage.

Experiments that are added to H2O MLOps from the MLflow Model Registry include both the MLflow model name (source_model_name) and MLflow version number (source_model_version) as part of the experiment metadata.

Model schema

Each model can be described by its input and output column names and their types. Knowing the model schema is essential for monitoring purposes. Currently, only models using the known schema can be deployed by MLOps.

Model schema is represented by the experiment metadata attached to the experiment. Deployer expects the model schema to be stored in the json_value of the input_schema and output_schema keys.

Note

Natively supported Driverless AI MOJO2 and H2O-3 MOJO2 models are not required to contain the schema as the schema is an integral part of the MOJO2 artifacts.

Schema format

The following example shows how model schema is formatted:

[{"name": "<column name>", "type": "<column type>"}, ... ]

Column types

The following is a list of supported column types:

  • Boolean
  • Time64
  • Float32
  • Float64
  • Int32
  • Int64
  • String

Note

Column type names are not case-sensitive.

Artifacts in MLOps

Defining artifacts and experiment artifacts

  • Artifact: An arbitrary binary large object (BLOB) attached to a particular entity in the H2O.ai Storage.

  • Experiment Artifact: Any artifact that is attached to the experiment entity.

Artifact type

Because any entity can have multiple artifacts attached to it, specific artifacts must be identified by their type. Type is an arbitrary string. Artifact type is recognized by and relevant to MLOps deployments.

The following is a list of artifact types:

  • dai/mojo_pipeline (Natively supported, ingestion supported)

  • h2o3/mojo (Natively supported, ingestion supported)

  • Python/MLflow (Ingestion supported)

Deployable artifact type

A deployable artifact type is an artifact type that the Deployer knows how to process and deploy. Each deployable artifact type consists of the name, readable name, and reference to the artifact type.

Artifact processor

Artifact processor is the routine that takes the raw artifact data and transforms it into the format that is digestible by the runtime. In this routine, an artifact is defined by its name, the model type that it produces, and a container image reference.

Artifact processors can be any container image that can be pulled from the target deployment environment. Each processor needs to recognize and use two environmental variables.

The following is a list of artifact processor environment variables:

  • SOURCE_PATH: Path to the file containing the raw data of the artifact

  • TARGET_PATH: Path where the processor saves its output. This path is passed as MODEL_PATH to the runtime

Model type

Model types enumerate types outputted by artifact processors. This indirection is included due to the fact that one particular artifact type can contain multiple internal artifacts, each of which may be consumed by different runtimes. One particular artifact type can be processed in different ways, producing different outputs consumable by different runtimes.

A model type defines what runtime can be used for artifact deployment.

Runtime

The runtime defines the way how the experiment is deployed and exposed. The runtime is defined by its name, the model type it can digest and a reference to the container image. Each runtime can be configured with environment variables.

The runtime consists of the server exposing the scorer API (see below) and all of the dependencies required to serve a particular model type.

Runtime interface

  • The runtime must expose HTTP/1.1 server on the port 8080
  • Server needs to (partially) implement defined scorer API
  • GET /model/schema and GET /model/score are required
  • Runtime recognizes and uses MODEL_PATH environment variable and expects the supported artifact format on that path

Runtime environment variables

  • MODEL_PATH: The path at which a model of the supported model type is located.
  • MODEL_ID: A unique string identifying the model