Configure scoring endpoints
This page describes how to use the H2O MLOps Python client to configure scoring endpoints.
Create an endpoint​
This example assumes that you know the following values:
PROJECT_ID = "..."
ENVIRONMENT_ID = "..."
DEPLOYMENT_ID = "..."
Endpoints use resource names for configuration. The following is an example of how you can convert the unique IDs to resource names.
# environment_resource_name = f"projects/{PROJECT_ID}/environments/{ENVIRONMENT_ID}"
# deployment_resource_name = f"{environment_resource_name}/deployments/{DEPLOYMENT_ID}"
environment_resource_name = f"projects/{project_1.id}/environments/{env_1.id}"
deployment_resource_name = f"{environment_resource_name}/deployments/{deployment_1.id}"
-
The following is an example of the output from inputting
environment_resource_name
.environment_resource_name
'projects/96b2fffa-7932-4a3f-9db0-48d60c193838/environments/c682855f-4e8f-4027-91ad-dbe2c88b0ff7' -
The following is an example of the output from inputting
deployment_resource_name
.deployment_resource_name
'projects/96b2fffa-7932-4a3f-9db0-48d60c193838/environments/c682855f-4e8f-4027-91ad-dbe2c88b0ff7/deployments/d44760ec-8164-47c4-bc45-5b695380c229'
Define and request the creation of an endpoint​
The following is an example of how you can define an endpoint and have it be accessible at the path abc/xyz
.
endpoint_definition = h2o_mlops_client.DeployConfigurableEndpoint(
display_name="My Custom Endpoint",
description="This is my custom endpoint.",
path="abc/xyz",
target=deployment_resource_name
)
You can then submit a request to create that endpoint in your target environment:
endpoint = mlops.deployer.endpoint.create_endpoint(
mlops.DeployCreateEndpointRequest(
parent=environment_resource_name,
endpoint=endpoint_definition
)
).endpoint
endpoint
{'create_time': datetime.datetime(2023, 2, 16, 19, 16, 35, 458084, tzinfo=tzlocal()),
'description': 'This is my custom endpoint.',
'display_name': 'My Custom Endpoint',
'name': 'projects/96b2fffa-7932-4a3f-9db0-48d60c193838/environments/c682855f-4e8f-4027-91ad-dbe2c88b0ff7/endpoints/f5df8be6-32de-4254-baa7-0b0c1549b753',
'path': 'abc/xyz',
'target': 'projects/96b2fffa-7932-4a3f-9db0-48d60c193838/environments/c682855f-4e8f-4027-91ad-dbe2c88b0ff7/deployments/d44760ec-8164-47c4-bc45-5b695380c229',
'update_time': datetime.datetime(2023, 2, 16, 19, 16, 35, 458415, tzinfo=tzlocal())}
Use endpoints​
Once you have an endpoint, it provides a static URL that can be pointed to any deployment in the deployment environment of the endpoint.
Endpoints take a little time to become ready to recieve HTTP requests. We have a request in place for a status indicator of readiness.
static_url = f"https://model.cloud-qa.h2o.ai/{endpoint.path}/model"
static_url
'https://model.cloud-qa.h2o.ai/abc/xyz/model'
Any of the model methods available from the deployment (for example, capabilities
, schema
, id
, sample_request
, and score
) are available at the endpoint.
sample_request = httpx.get(f"{static_url}/sample_request").json()
httpx.post(
url=f"{static_url}/score",
json=sample_request
).json()
Sample return:
{'fields': ['default payment next month.0', 'default payment next month.1'],
'id': 'bbc16b62-7730-11ed-8a78-ca18b025b240',
'score': [['0.9472957', '0.052704293']]}
Update an endpoint​
The following is an example of how you can update the endpoint so that same URI routes to a different model deployment. This example assumes that you know another model's deployment ID (referred to here as ANOTHER_DEPLOYMENT_ID
).
another_deployment_resource_name = f"{environment_resource_name}/deployments/{ANOTHER_DEPLOYMENT_ID}"
endpoint.target = another_deployment_resource_name
endpoint = mlops.deployer.endpoint.update_endpoint(
h2o_mlops_client.DeployUpdateEndpointRequest(
endpoint=endpoint
)
).endpoint
Scoring requests to the static https://model.cloud-qa.h2o.ai/abc/xyz/model
URI will now direct to the new deployment.
List all endpoints in an environment​
The following is an example of how to list all the endpoints that exist in an environment.
mlops.deployer.endpoint.list_endpoints(
h2o_mlops_client.DeployListEndpointsRequest(
parent=environment_resource_name
)
)
Sample return:
{'endpoints': [{'create_time': datetime.datetime(2023, 2, 16, 19, 16, 35, 458084, tzinfo=tzlocal()),
'description': 'This is my custom endpoint.',
'display_name': 'My Custom Endpoint',
'name': 'projects/96b2fffa-7932-4a3f-9db0-48d60c193838/environments/c682855f-4e8f-4027-91ad-dbe2c88b0ff7/endpoints/f5df8be6-32de-4254-baa7-0b0c1549b753',
'path': 'abc/xyz',
'target': 'projects/96b2fffa-7932-4a3f-9db0-48d60c193838/environments/c682855f-4e8f-4027-91ad-dbe2c88b0ff7/deployments/d44760ec-8164-47c4-bc45-5b695380c229',
'update_time': datetime.datetime(2023, 2, 16, 19, 18, 40, 360409, tzinfo=tzlocal())}],
'next_page_token': ''}
Get an endpoint​
The following is an example of how you can get information about individual endpoints by their resource name.
endpoint = mlops.deployer.endpoint.get_endpoint(
h2o_mlops_client.DeployGetEndpointRequest(
name=endpoint.name
)
).endpoint
Delete an endpoint​
You can pass in the name of an endpoint to delete it. The following is an example of how to delete an endpoint.
mlops.deployer.endpoint.delete_endpoint(
h2o_mlops_client.DeployDeleteEndpointRequest(
name=endpoint.name
)
)
- Submit and view feedback for this page
- Send feedback about H2O MLOps to cloud-feedback@h2o.ai