Skip to main content
Version: v0.67.0

Driverless AI example

This tutorial is an example of how to perform a standard Driverless AI flow using the MLOps Python Client. It fetches a dataset from the Driverless AI instance and starts an experiment. It then creates a project and links both the dataset and the experiment to that project. Finally, it deploys the project to the DEV environment in MLOps.

Before you begin
  • Install Driverless AI.
  • Launch a new instance of Driverless AI.
  • You will need the values for the following constants in order to successfully carry out the task. Contact your administrator to obtain deployment specific values.
ConstantValueDescription
MLOPS_API_URLUsually: https://api.mlops.my.domainDefines the URL for the MLOps Gateway component. You can verify the correct URL by navigating to the API URL in your browser. It should provide a page with a list of available routes.
TOKEN_ENDPOINT_URL
https://mlops.keycloak.domain/auth/realms/[fill-in-realm-name]/protocol/openid-connect/token
Defines the token endpoint URL of the Identity Provider. This uses Keycloak as the Identity Provider. Keycloak Realm should be provided.
REFRESH_TOKEN<your-refresh-token>Defines the user's refresh token
CLIENT_ID<your-client-id>Sets the client id for authentication. This is the client you will be using to connect to MLOps.
PROJECT_NAMEStandardFlowExampleDefines a project name that the script will be using.
DRIVERLESS_URL<your-dai-instance-url>Defines the URL of the Driverless AI instance you have launched.
EXPERIMENT_NAMECreditCardExampleDefines a name for the experiment that will be created.
TRAIN_FILE_URL
s3://h2o-public-test-data/smalldata/kaggle/CreditCard/creditcard_train_cat.csv
Defines the training dataset.
TARGET_COLUMNDEFAULT_PAYMENT_NEXT_MONTHDefines the target column for the experiment.
DEPLOYMENT_ENVIRONMENTDEVDefines the target deployment environment.
REFRESH_STATUS_INTERVAL1.0Defines a refresh interval for the deployment health check.
MAX_WAIT_TIME300Defines maximum waiting time for the deployment to become healthy.

The following steps demonstrate how you can use MLOps Python client to carry out a flow in Driverless AI.

  1. Download the StandardFlowExample.py file.

  2. Change the values of the following constants in your StandardFlowExample.py file as given in the preceding data table.

    StandardFlowExample.py
    ### Constants
    MLOPS_API_URL = <MLOPS_API_URL>
    TOKEN_ENDPOINT_URL = <TOKEN_ENDPOINT_URL>
    REFRESH_TOKEN = <REFRESH_TOKEN>
    CLIENT_ID = <CLIENT_ID>
    PROJECT_NAME = <PROJECT_NAME>
    DRIVERLESS_URL = <DRIVERLESS_URL>
    EXPERIMENT_NAME = <EXPERIMENT_NAME>
    TRAIN_FILE_URL = <TRAIN_FILE_URL>
    TARGET_COLUMN = <TARGET_COLUMN>
    DEPLOYMENT_ENVIRONMENT = <DEPLOYMENT_ENVIRONMENT>
    REFRESH_STATUS_INTERVAL = <REFRESH_STATUS_INTERVAL>
    MAX_WAIT_TIME = <MAX_WAIT_TIME>
    StandardFlowExample.py
    ### Constants
    MLOPS_API_URL = "https://api.mlops.my.domain"
    TOKEN_ENDPOINT_URL="https://mlops.keycloak.domain/auth/realms/[fill-in-realm-name]/protocol/openid-connect/token"
    REFRESH_TOKEN="<your-refresh-token>"
    CLIENT_ID="<your-mlops-client>"
    DRIVERLESS_URL = "<your-dai-instance-url>"
    PROJECT_NAME = "StandardFlowExample"
    EXPERIMENT_NAME = "CreditCardExample"
    TRAIN_FILE_URL = "s3://h2o-public-test-data/smalldata/kaggle/CreditCard/creditcard_train_cat.csv"
    TARGET_COLUMN = "DEFAULT_PAYMENT_NEXT_MONTH"
    DEPLOYMENT_ENVIRONMENT = "DEV"
    REFRESH_STATUS_INTERVAL = 1.0
    MAX_WAIT_TIME = 300
  1. Run the StandardFlowExample.py file.

    python3 StandardFlowExample.py
    Complete 100.00% - [4/4] Computed stats for column DEFAULT_PAYMENT_NEXT_MONTH
    Experiment launched at: <your-dai-url>/experiment?key=77960ca4-0cbe-11ed-afae-0a7e293afcca
    Complete 100.00% - Status: Complete
    Deployment has become healthy
  2. Navigate to the launched Driverless AI instance from your browser and click the Experiments tab. Then select the name of the created experiment CreditCardExample in order to view the details of the experiment.

    Python client Driverless AI example experiment

    Note

    For more information about experiments in Driverless AI, refer Driverless AI documentation.

  3. Click the Projects tab in the Driverless AI instance to view the created project StandardFlowExample.

    Python client Driverless AI example project

    Note

    For more information about projects in Driverless AI, see Driverless AI documentation.

  4. Finally, navigate to MLOps and click the project name StandardFlowExample under Projects to view the deployed model.

    Python client Driverless AI example MLOps

    Note

    For more information about model deployments in MLOps, see Understand deployments in MLOps.

Example walkthrough

The following steps provide a walkthrough of each of the sections in the StandardFlowExample.py file.

  1. Include the helper function, which waits for the deployment to be healthy.

  2. Set up the token provider using an existing refresh token.

  3. Set up the Driverless AI client.

    StandardFlowExample.py
    dai_client = driverlessai.Client(
    address=DRIVERLESS_URL, token_provider=mlops_token_provider
    )
  4. Set up the MLOps client.

  5. Fetch the credit card dataset from S3 as the training dataset ds_train and start a basic credit card experiment exp_cc with minimal parameters.

    StandardFlowExample.py
    # Fetching the credit card dataset from S3.
    ds_train = dai_client.datasets.create(data=TRAIN_FILE_URL, data_source="s3")

    # Starting a basic credit card experiment with minimal parameters.
    exp_cc = dai_client.experiments.create(
    name=EXPERIMENT_NAME,
    train_dataset=ds_train,
    target_column=TARGET_COLUMN,
    task="classification",
    accuracy=1,
    time=1,
    interpretability=1,
    scorer="AUC",
    enable_gpus=False,
    seed=1234,
    cols_to_drop=["ID"],
    )
  6. Create a project in Driverless AI named prj and link the credit card dataset ds_train and experiment exp_cc to the project.

    StandardFlowExample.py
    # Creating a project in DAI.
    prj = dai_client.projects.create(PROJECT_NAME)

    # Linking the dataset to the project.
    prj.link_dataset(ds_train, dataset_type="train_dataset")

    # Linking the experiment to the project.
    prj.link_experiment(exp_cc)
  7. Fetch the available deployment environments and look up the ID of the selected deployment environment.

  8. Look for the ID of the artifact to deploy and customize the composition of the deployment. Then specify the deployment as a single deployment.

    StandardFlowExample.py
    # Looking for the ID of the artifact to deploy.
    artifact_list = mlops_client.storage.artifact.list_entity_artifacts(
    mlops.StorageListEntityArtifactsRequest(entity_id=exp_cc.key)
    ).artifact

    for ar in artifact_list:
    if ar.type == "dai/mojo_pipeline":
    artifact_id = ar.id
    break
    else:
    raise LookupError(f"Could not find 'dai/mojo_pipeline' artifact.")

    # Customize the composition of the deployment
    composition = mlops.DeployDeploymentComposition(
    experiment_id=exp_cc.key,
    artifact_id=artifact_id,
    deployable_artifact_type_name="dai/mojo_pipeline",
    artifact_processor_name="dai_mojo_pipeline_extractor",
    runtime_name="dai_mojo_runtime",
    )

    # Specify the deployment as a single deployment (not champion/challenger or A/B test)
    to_deploy = mlops.DeployDeployment(
    project_id=prj.key,
    deployment_environment_id=deployment_env_id,
    single_deployment=mlops.DeploySingleDeployment(
    deployment_composition=composition
    ),
    )
  9. Finally, create a deployment and wait for the deployment to become healthy.


Feedback