Configure scoring endpoints
This page describes how to use the H2O MLOps Python client to configure scoring endpoints.
Create an endpoint
This example assumes that you know the following values:
PROJECT_ID = "..."
ENVIRONMENT_ID = "..."
DEPLOYMENT_ID = "..."
Endpoints use resource names for configuration. The following is an example of how you can convert the unique IDs to resource names.
# environment_resource_name = f"projects/{PROJECT_ID}/environments/{ENVIRONMENT_ID}"
# deployment_resource_name = f"{environment_resource_name}/deployments/{DEPLOYMENT_ID}"
environment_resource_name = f"projects/{project_1.id}/environments/{env_1.id}"
deployment_resource_name = f"{environment_resource_name}/deployments/{deployment_1.id}"
The following is an example of the output from inputting
environment_resource_name
.environment_resource_name
'projects/96b2fffa-7932-4a3f-9db0-48d60c193838/environments/c682855f-4e8f-4027-91ad-dbe2c88b0ff7'The following is an example of the output from inputting
deployment_resource_name
.deployment_resource_name
'projects/96b2fffa-7932-4a3f-9db0-48d60c193838/environments/c682855f-4e8f-4027-91ad-dbe2c88b0ff7/deployments/d44760ec-8164-47c4-bc45-5b695380c229'
Define and request the creation of an endpoint
The following is an example of how you can define an endpoint and have it be accessible at the path abc/xyz
.
endpoint_definition = h2o_mlops_client.DeployConfigurableEndpoint(
display_name="My Custom Endpoint",
description="This is my custom endpoint.",
path="abc/xyz",
target=deployment_resource_name
)
You can then submit a request to create that endpoint in your target environment:
endpoint = mlops.deployer.endpoint.create_endpoint(
mlops.DeployCreateEndpointRequest(
parent=environment_resource_name,
endpoint=endpoint_definition
)
).endpoint
endpoint
{'create_time': datetime.datetime(2023, 2, 16, 19, 16, 35, 458084, tzinfo=tzlocal()),
'description': 'This is my custom endpoint.',
'display_name': 'My Custom Endpoint',
'name': 'projects/96b2fffa-7932-4a3f-9db0-48d60c193838/environments/c682855f-4e8f-4027-91ad-dbe2c88b0ff7/endpoints/f5df8be6-32de-4254-baa7-0b0c1549b753',
'path': 'abc/xyz',
'target': 'projects/96b2fffa-7932-4a3f-9db0-48d60c193838/environments/c682855f-4e8f-4027-91ad-dbe2c88b0ff7/deployments/d44760ec-8164-47c4-bc45-5b695380c229',
'update_time': datetime.datetime(2023, 2, 16, 19, 16, 35, 458415, tzinfo=tzlocal())}
Use endpoints
Once you have an endpoint, it provides a static URL that can be pointed to any deployment in the deployment environment of the endpoint.
Endpoints take a little time to become ready to recieve HTTP requests. We have a request in place for a status indicator of readiness.
static_url = f"https://model.cloud-qa.h2o.ai/{endpoint.path}/model"
static_url
'https://model.cloud-qa.h2o.ai/abc/xyz/model'
Any of the model methods available from the deployment (for example, capabilities
, schema
, id
, sample_request
, and score
) are available at the endpoint.
sample_request = httpx.get(f"{static_url}/sample_request").json()
httpx.post(
url=f"{static_url}/score",
json=sample_request
).json()
Sample return:
{'fields': ['default payment next month.0', 'default payment next month.1'],
'id': 'bbc16b62-7730-11ed-8a78-ca18b025b240',
'score': [['0.9472957', '0.052704293']]}
Update an endpoint
The following is an example of how you can update the endpoint so that same URI routes to a different model deployment. This example assumes that you know another model's deployment ID (referred to here as ANOTHER_DEPLOYMENT_ID
).
another_deployment_resource_name = f"{environment_resource_name}/deployments/{ANOTHER_DEPLOYMENT_ID}"
endpoint.target = another_deployment_resource_name
endpoint = mlops.deployer.endpoint.update_endpoint(
h2o_mlops_client.DeployUpdateEndpointRequest(
endpoint=endpoint
)
).endpoint
Scoring requests to the static https://model.cloud-qa.h2o.ai/abc/xyz/model
URI will now direct to the new deployment.
List all endpoints in an environment
The following is an example of how to list all the endpoints that exist in an environment.
mlops.deployer.endpoint.list_endpoints(
h2o_mlops_client.DeployListEndpointsRequest(
parent=environment_resource_name
)
)
Sample return:
{'endpoints': [{'create_time': datetime.datetime(2023, 2, 16, 19, 16, 35, 458084, tzinfo=tzlocal()),
'description': 'This is my custom endpoint.',
'display_name': 'My Custom Endpoint',
'name': 'projects/96b2fffa-7932-4a3f-9db0-48d60c193838/environments/c682855f-4e8f-4027-91ad-dbe2c88b0ff7/endpoints/f5df8be6-32de-4254-baa7-0b0c1549b753',
'path': 'abc/xyz',
'target': 'projects/96b2fffa-7932-4a3f-9db0-48d60c193838/environments/c682855f-4e8f-4027-91ad-dbe2c88b0ff7/deployments/d44760ec-8164-47c4-bc45-5b695380c229',
'update_time': datetime.datetime(2023, 2, 16, 19, 18, 40, 360409, tzinfo=tzlocal())}],
'next_page_token': ''}
Get an endpoint
The following is an example of how you can get information about individual endpoints by their resource name.
endpoint = mlops.deployer.endpoint.get_endpoint(
h2o_mlops_client.DeployGetEndpointRequest(
name=endpoint.name
)
).endpoint
Delete an endpoint
You can pass in the name of an endpoint to delete it. The following is an example of how to delete an endpoint.
mlops.deployer.endpoint.delete_endpoint(
h2o_mlops_client.DeployDeleteEndpointRequest(
name=endpoint.name
)
)
- Submit and view feedback for this page
- Send feedback about H2O MLOps to cloud-feedback@h2o.ai