Skip to main content
Version: v0.66.1

Add external registry model to MLOps

This example demonstrates how you can use the MLOps Python client to add experiments from an external model registry to MLOps using the ExternalRegisteredModelVersionService of the MLOps API.

Before you begin
ConstantValueDescription
MLOPS_API_URLUsually: https://api.mlops.my.domainDefines the URL for the MLOps Gateway component.
TOKEN_ENDPOINT_URL
https://mlops.keycloak.domain/auth/realms/[fill-in-realm-name]/protocol/openid-connect/token
Defines the token endpoint URL of the Identity Provider. This uses Keycloak as the Identity Provider. Keycloak Realm should be provided.
REFRESH_TOKEN<your-refresh-token>Defines the user's refresh token.
CLIENT_ID<your-client-id>Sets the client id for authentication. This is the client you will be using to connect to MLOps.
CLIENT_SECRET<your-client-secret>Sets the client secret.
PROJECT_NAME<project-name>Provide the project name in MLOps to which the model will be imported.
EXTERNAL_REGISTERED_MODEL_NAME<model-name>Provide the registered model name in the external model registry to import from.
EXTERNAL_REGISTERED_MODEL_VERSIONSeg: ["1", "2", "3"]Provide the versions of the EXTERNAL_REGISTERED_MODEL_NAME to import. This allows importing up to five versions. Version numbers should be provided as an array.

The following steps demonstrate how you can use the MLOps Python client to add experiments from an external model registry to MLOps.

  1. Download the AddExternalRegistryModelToMLOps.py file.

  2. Change the values of the following constants in your AddExternalRegistryModelToMLOps.py file as given in the preceding data table.

    AddExternalRegistryModelToMLOps.py
    ### Constants
    MLOPS_API_URL = "https://api.mlops.my.domain"
    TOKEN_ENDPOINT_URL="https://mlops.keycloak.domain/auth/realms/[fill-in-realm-name]/protocol/openid-connect/token"
    REFRESH_TOKEN="<your-refresh-token>"
    CLIENT_ID="<your-mlops-client>"
    CLIENT_SECRET = "<your-client-secret>"
    PROJECT_NAME = "<project-name>"
    EXTERNAL_REGISTERED_MODEL_NAME = "<model-name>"
    EXTERNAL_REGISTERED_MODEL_VERSIONS = ["1", "2", "3"]
  3. Run the AddExternalRegistryModelToMLOps.py file.

    python3 AddExternalRegistryModelToMLOps.py
  4. This adds the experiments for the list of model versions represented by the request.

    {
    "responses" : [ {
    "description" : "description",
    "experimentId" : "experimentId",
    "status" : "status"
    }, {
    "description" : "description",
    "experimentId" : "experimentId",
    "status" : "status"
    } ]
    }
  5. Navigate to your project in MLOps and click the Experiments tab to view the experiments imported from the external model registry. This example uses MLflow as the external model registry.

    Add MLflow model to MLOps - Experiments

  6. Click the Models tab to view the imported model. It is now registered with the MLOps model registry. If the imported model was another version of an existing model, then the new version is added.

    Add MLflow model to MLOps - Models

Example walkthrough

This section provides a walkthrough of the AddExternalRegistryModelToMLOps.py file.

  1. Get the list of projects in MLOps and check if the given PROJECT_NAME exists.

    AddExternalRegistryModelToMLOps.py
    def _get_project_id(projects: List[mlops.StorageProject]) -> str:
    """Gets the ID of the selected project."""
    for p in projects:
    if p.display_name == PROJECT_NAME:
    return p
    else:
    raise LookupError("Requested project not found")
  2. Prepare the request required by the _list_all_versions_for_model function.

    AddExternalRegistryModelToMLOps.py
    def _form_list_model_version_request(model_name:str):
    return mlops.ModelregistryListRegisteredModelVersionsForModelRequest(
    source_key=model_name
    )
  3. Prepare the request required by the _import_external_model_into_mlops function.

    AddExternalRegistryModelToMLOps.py
    def _form_import_model_version_request(project_id: str, registered_model_name: str, registered_model_versions: List[str]):

    external_registered_model_versions = [
    mlops.ModelregistryExternalRegisteredModelVersion(
    source_key=registered_model_name, version=version
    )
    for version in registered_model_versions
    ]
    return mlops.ModelregistryImportRegisteredModelVersionRequest(
    external_registered_model_versions=external_registered_model_versions,
    project_id=project_id,
    )
  1. Prepare the request required by the _add_experiments_to_mlops function.

    AddExternalRegistryModelToMLOps.py
    def _form_add_experiments_request(versions: List[mlops.ModelregistryExternalRegisteredModelVersion]):

    return [
    mlops.ModelregistryAddExperimentRequest(
    model_version_id=version.adapter_model_version_id,
    experiment_name="imported_experiment",
    )
    for version in versions
    ]
  2. Create a new project in MLOps if a project by the given name does not exist.

    AddExternalRegistryModelToMLOps.py
    def _create_project(mlops_client: mlops.Client):
    return mlops_client.storage.project.create_project(
    mlops.StorageCreateProjectRequest(
    mlops.StorageProject(display_name=PROJECT_NAME)
    )
    ).project
  3. List the model versions for the given registered model by calling the ListRegisteredModelVersionsForModel endpoint of the ExternalRegisteredModelVersionService.

    AddExternalRegistryModelToMLOps.py
    def _list_all_versions_for_model(
    mlops_client: mlops.Client, model_name: str
    ):
    return mlops_client.external_registry.external_registered_model_version_service.list_registered_model_versions_for_model(
    _form_list_model_version_request(model_name)
    )
  4. Import the given model versions into MLOps by using the ImportRegisteredModel endpoint of the ExternalRegisteredModelVersionService.

    AddExternalRegistryModelToMLOps.py
    def _import_external_model_into_mlops(
    mlops_client: mlops.Client, project_id: str, registered_model_name: str, registered_model_versions: List[str]
    ):
    try:
    response = mlops_client.external_registry.external_registered_model_version_service.import_registered_model(
    _form_import_model_version_request(project_id, registered_model_name, registered_model_versions)
    )
    return response
    except Exception:
    raise
  5. Add the experiments to MLOps by calling the BatchAddExperiments endpoint of the ExternalRegisteredModelVersionService.

    AddExternalRegistryModelToMLOps.py
    def _add_experiments_to_mlops(
    mlops_client: mlops.Client, versions: mlops.ModelregistryImportRegisteredModelVersionResponse
    ):
    try:
    response = mlops_client.external_registry.external_registered_model_version_service.batch_add_experiments(
    mlops.ModelregistryAddExperimentsRequest(
    _form_add_experiments_request(versions)
    )
    )
    return response
    except Exception:
    raise
  6. In the main function, set up the token provider using an existing refresh token and client secret, and then set up the MLOps client.

  7. Call the _get_project_id function to get the id of the given project name if a project by that name exists in MLOps. If not, call the _create_project function to create a new project with that name.

    AddExternalRegistryModelToMLOps.py
    prj: mlops.StorageProject
    projects: mlops.StorageListProjectsResponse = mlops_client.storage.project.list_projects(
    mlops.StorageListProjectsRequest()
    )
    try:
    prj = _get_project_id(projects.project)
    except LookupError:
    prj = _create_project(mlops_client)
  8. Call the _import_external_model_into_mlops function with the project id, EXTERNAL_REGISTERED_MODEL_NAME and EXTERNAL_REGISTERED_MODEL_VERSIONS to import the registered model versions from the external model registry. Then call the _add_experiments_to_mlops function with the imported versions.

    AddExternalRegistryModelToMLOps.py
    try:
    import_response = _import_external_model_into_mlops(mlops_client, prj.id, EXTERNAL_REGISTERED_MODEL_NAME, EXTERNAL_REGISTERED_MODEL_VERSIONS)

    imported_versions = import_response.external_registered_model_versions.copy()
    for version in import_response.external_registered_model_versions:
    if version.status == "IMPORTED":
    print(version.source_key + " " + "Version:" +version.version +"--"+ version.status_description)
    imported_versions.remove(version)
    elif version.status_description:
    print(version.source_key + " " + "Version:" + version.version + "--" + version.status_description)
    imported_versions.remove(version)

    if len(imported_versions) > 0:
    add_experiment_response = _add_experiments_to_mlops(mlops_client, imported_versions)
    print(add_experiment_response)
    else:
    print("No experiments available to add")
    except Exception:
    raise

Feedback