Driverless AI example
This tutorial is an example of how to perform a standard Driverless AI flow using the MLOps Python Client. It fetches a dataset from the Driverless AI instance and starts an experiment. It then creates a project and links both the dataset and the experiment to that project. Finally, it deploys the project to the DEV
environment in MLOps.
- Install Driverless AI.
- Launch a new instance of Driverless AI.
- You will need the values for the following constants in order to successfully carry out the task. Contact your administrator to obtain deployment specific values.
Constant | Value | Description |
---|---|---|
MLOPS_API_URL | Usually: https://api.mlops.my.domain | Defines the URL for the MLOps Gateway component. You can verify the correct URL by navigating to the API URL in your browser. It should provide a page with a list of available routes. |
TOKEN_ENDPOINT_URL | https://mlops.keycloak.domain/auth/realms/[fill-in-realm-name]/protocol/openid-connect/token | Defines the token endpoint URL of the Identity Provider. This uses Keycloak as the Identity Provider. Keycloak Realm should be provided. |
REFRESH_TOKEN | <your-refresh-token> | Defines the user's refresh token |
CLIENT_ID | <your-client-id> | Sets the client id for authentication. This is the client you will be using to connect to MLOps. |
PROJECT_NAME | StandardFlowExample | Defines a project name that the script will be using. |
DRIVERLESS_URL | <your-dai-instance-url> | Defines the URL of the Driverless AI instance you have launched. |
EXPERIMENT_NAME | CreditCardExample | Defines a name for the experiment that will be created. |
TRAIN_FILE_URL | s3://h2o-public-test-data/smalldata/kaggle/CreditCard/creditcard_train_cat.csv | Defines the training dataset. |
TARGET_COLUMN | DEFAULT_PAYMENT_NEXT_MONTH | Defines the target column for the experiment. |
DEPLOYMENT_ENVIRONMENT | DEV | Defines the target deployment environment. |
REFRESH_STATUS_INTERVAL | 1.0 | Defines a refresh interval for the deployment health check. |
MAX_WAIT_TIME | 300 | Defines maximum waiting time for the deployment to become healthy. |
The following steps demonstrate how you can use MLOps Python client to carry out a flow in Driverless AI.
Change the values of the following constants in your
StandardFlowExample.py
file as given in the preceding data table.StandardFlowExample.py### Constants
MLOPS_API_URL = <MLOPS_API_URL>
TOKEN_ENDPOINT_URL = <TOKEN_ENDPOINT_URL>
REFRESH_TOKEN = <REFRESH_TOKEN>
CLIENT_ID = <CLIENT_ID>
PROJECT_NAME = <PROJECT_NAME>
DRIVERLESS_URL = <DRIVERLESS_URL>
EXPERIMENT_NAME = <EXPERIMENT_NAME>
TRAIN_FILE_URL = <TRAIN_FILE_URL>
TARGET_COLUMN = <TARGET_COLUMN>
DEPLOYMENT_ENVIRONMENT = <DEPLOYMENT_ENVIRONMENT>
REFRESH_STATUS_INTERVAL = <REFRESH_STATUS_INTERVAL>
MAX_WAIT_TIME = <MAX_WAIT_TIME>StandardFlowExample.py### Constants
MLOPS_API_URL = "https://api.mlops.my.domain"
TOKEN_ENDPOINT_URL="https://mlops.keycloak.domain/auth/realms/[fill-in-realm-name]/protocol/openid-connect/token"
REFRESH_TOKEN="<your-refresh-token>"
CLIENT_ID="<your-mlops-client>"
DRIVERLESS_URL = "<your-dai-instance-url>"
PROJECT_NAME = "StandardFlowExample"
EXPERIMENT_NAME = "CreditCardExample"
TRAIN_FILE_URL = "s3://h2o-public-test-data/smalldata/kaggle/CreditCard/creditcard_train_cat.csv"
TARGET_COLUMN = "DEFAULT_PAYMENT_NEXT_MONTH"
DEPLOYMENT_ENVIRONMENT = "DEV"
REFRESH_STATUS_INTERVAL = 1.0
MAX_WAIT_TIME = 300
Run the
StandardFlowExample.py
file.python3 StandardFlowExample.py
Complete 100.00% - [4/4] Computed stats for column DEFAULT_PAYMENT_NEXT_MONTH
Experiment launched at: <your-dai-url>/experiment?key=77960ca4-0cbe-11ed-afae-0a7e293afcca
Complete 100.00% - Status: Complete
Deployment has become healthyNavigate to the launched Driverless AI instance from your browser and click the Experiments tab. Then select the name of the created experiment
CreditCardExample
in order to view the details of the experiment.NoteFor more information about experiments in Driverless AI, refer Driverless AI documentation.
Click the Projects tab in the Driverless AI instance to view the created project
StandardFlowExample
.NoteFor more information about projects in Driverless AI, see Driverless AI documentation.
Finally, navigate to MLOps and click the project name
StandardFlowExample
under Projects to view the deployed model.NoteFor more information about model deployments in MLOps, see Understand deployments in MLOps.
Example walkthrough
The following steps provide a walkthrough of each of the sections in the StandardFlowExample.py
file.
Include the helper function, which waits for the deployment to be healthy.
Set up the Driverless AI client.
StandardFlowExample.pydai_client = driverlessai.Client(
address=DRIVERLESS_URL, token_provider=mlops_token_provider
)Fetch the credit card dataset from S3 as the training dataset
ds_train
and start a basic credit card experimentexp_cc
with minimal parameters.StandardFlowExample.py# Fetching the credit card dataset from S3.
ds_train = dai_client.datasets.create(data=TRAIN_FILE_URL, data_source="s3")
# Starting a basic credit card experiment with minimal parameters.
exp_cc = dai_client.experiments.create(
name=EXPERIMENT_NAME,
train_dataset=ds_train,
target_column=TARGET_COLUMN,
task="classification",
accuracy=1,
time=1,
interpretability=1,
scorer="AUC",
enable_gpus=False,
seed=1234,
cols_to_drop=["ID"],
)Create a project in Driverless AI named
prj
and link the credit card datasetds_train
and experimentexp_cc
to the project.StandardFlowExample.py# Creating a project in DAI.
prj = dai_client.projects.create(PROJECT_NAME)
# Linking the dataset to the project.
prj.link_dataset(ds_train, dataset_type="train_dataset")
# Linking the experiment to the project.
prj.link_experiment(exp_cc)Look for the ID of the artifact to deploy and customize the composition of the deployment. Then specify the deployment as a single deployment.
StandardFlowExample.py# Looking for the ID of the artifact to deploy.
artifact_list = mlops_client.storage.artifact.list_entity_artifacts(
mlops.StorageListEntityArtifactsRequest(entity_id=exp_cc.key)
).artifact
for ar in artifact_list:
if ar.type == "dai/mojo_pipeline":
artifact_id = ar.id
break
else:
raise LookupError(f"Could not find 'dai/mojo_pipeline' artifact.")
# Customize the composition of the deployment
composition = mlops.DeployDeploymentComposition(
experiment_id=exp_cc.key,
artifact_id=artifact_id,
deployable_artifact_type_name="dai/mojo_pipeline",
artifact_processor_name="dai_mojo_pipeline_extractor",
runtime_name="dai_mojo_runtime",
)
# Specify the deployment as a single deployment (not champion/challenger or A/B test)
to_deploy = mlops.DeployDeployment(
project_id=prj.key,
deployment_environment_id=deployment_env_id,
single_deployment=mlops.DeploySingleDeployment(
deployment_composition=composition
),
)Finally, create a deployment and wait for the deployment to become healthy.
- Submit and view feedback for this page
- Send feedback about H2O MLOps to cloud-feedback@h2o.ai