Skip to main content
Version: v0.61.1

Configure scoring endpoints

This page describes how to use the H2O MLOps Python client to configure scoring endpoints.

Create an endpoint

This example assumes that you know the following values:

PROJECT_ID = "..."
ENVIRONMENT_ID = "..."
DEPLOYMENT_ID = "..."

Endpoints use resource names for configuration. The following is an example of how you can convert the unique IDs to resource names.

# environment_resource_name = f"projects/{PROJECT_ID}/environments/{ENVIRONMENT_ID}"
# deployment_resource_name = f"{environment_resource_name}/deployments/{DEPLOYMENT_ID}"
environment_resource_name = f"projects/{project_1.id}/environments/{env_1.id}"
deployment_resource_name = f"{environment_resource_name}/deployments/{deployment_1.id}"
  • The following is an example of the output from inputting environment_resource_name.

    environment_resource_name
    'projects/96b2fffa-7932-4a3f-9db0-48d60c193838/environments/c682855f-4e8f-4027-91ad-dbe2c88b0ff7'
  • The following is an example of the output from inputting deployment_resource_name.

    deployment_resource_name
    'projects/96b2fffa-7932-4a3f-9db0-48d60c193838/environments/c682855f-4e8f-4027-91ad-dbe2c88b0ff7/deployments/d44760ec-8164-47c4-bc45-5b695380c229'

Define and request the creation of an endpoint

The following is an example of how you can define an endpoint and have it be accessible at the path abc/xyz.

endpoint_definition = h2o_mlops_client.DeployConfigurableEndpoint(
display_name="My Custom Endpoint",
description="This is my custom endpoint.",
path="abc/xyz",
target=deployment_resource_name
)

You can then submit a request to create that endpoint in your target environment:

endpoint = mlops.deployer.endpoint.create_endpoint(
mlops.DeployCreateEndpointRequest(
parent=environment_resource_name,
endpoint=endpoint_definition
)
).endpoint
endpoint
{'create_time': datetime.datetime(2023, 2, 16, 19, 16, 35, 458084, tzinfo=tzlocal()),
'description': 'This is my custom endpoint.',
'display_name': 'My Custom Endpoint',
'name': 'projects/96b2fffa-7932-4a3f-9db0-48d60c193838/environments/c682855f-4e8f-4027-91ad-dbe2c88b0ff7/endpoints/f5df8be6-32de-4254-baa7-0b0c1549b753',
'path': 'abc/xyz',
'target': 'projects/96b2fffa-7932-4a3f-9db0-48d60c193838/environments/c682855f-4e8f-4027-91ad-dbe2c88b0ff7/deployments/d44760ec-8164-47c4-bc45-5b695380c229',
'update_time': datetime.datetime(2023, 2, 16, 19, 16, 35, 458415, tzinfo=tzlocal())}

Use endpoints

Once you have an endpoint, it provides a static URL that can be pointed to any deployment in the deployment environment of the endpoint.

note

Endpoints take a little time to become ready to recieve HTTP requests. We have a request in place for a status indicator of readiness.

static_url = f"https://model.cloud-qa.h2o.ai/{endpoint.path}/model"
static_url
'https://model.cloud-qa.h2o.ai/abc/xyz/model'

Any of the model methods available from the deployment (for example, capabilities, schema, id, sample_request, and score) are available at the endpoint.

sample_request = httpx.get(f"{static_url}/sample_request").json()

httpx.post(
url=f"{static_url}/score",
json=sample_request
).json()

Sample return:

{'fields': ['default payment next month.0', 'default payment next month.1'],
'id': 'bbc16b62-7730-11ed-8a78-ca18b025b240',
'score': [['0.9472957', '0.052704293']]}

Update an endpoint

The following is an example of how you can update the endpoint so that same URI routes to a different model deployment. This example assumes that you know another model's deployment ID (referred to here as ANOTHER_DEPLOYMENT_ID).

another_deployment_resource_name = f"{environment_resource_name}/deployments/{ANOTHER_DEPLOYMENT_ID}"
endpoint.target = another_deployment_resource_name
endpoint = mlops.deployer.endpoint.update_endpoint(
h2o_mlops_client.DeployUpdateEndpointRequest(
endpoint=endpoint
)
).endpoint

Scoring requests to the static https://model.cloud-qa.h2o.ai/abc/xyz/model URI will now direct to the new deployment.

List all endpoints in an environment

The following is an example of how to list all the endpoints that exist in an environment.

mlops.deployer.endpoint.list_endpoints(
h2o_mlops_client.DeployListEndpointsRequest(
parent=environment_resource_name
)
)

Sample return:

{'endpoints': [{'create_time': datetime.datetime(2023, 2, 16, 19, 16, 35, 458084, tzinfo=tzlocal()),
'description': 'This is my custom endpoint.',
'display_name': 'My Custom Endpoint',
'name': 'projects/96b2fffa-7932-4a3f-9db0-48d60c193838/environments/c682855f-4e8f-4027-91ad-dbe2c88b0ff7/endpoints/f5df8be6-32de-4254-baa7-0b0c1549b753',
'path': 'abc/xyz',
'target': 'projects/96b2fffa-7932-4a3f-9db0-48d60c193838/environments/c682855f-4e8f-4027-91ad-dbe2c88b0ff7/deployments/d44760ec-8164-47c4-bc45-5b695380c229',
'update_time': datetime.datetime(2023, 2, 16, 19, 18, 40, 360409, tzinfo=tzlocal())}],
'next_page_token': ''}

Get an endpoint

The following is an example of how you can get information about individual endpoints by their resource name.

endpoint = mlops.deployer.endpoint.get_endpoint(
h2o_mlops_client.DeployGetEndpointRequest(
name=endpoint.name
)
).endpoint

Delete an endpoint

You can pass in the name of an endpoint to delete it. The following is an example of how to delete an endpoint.

mlops.deployer.endpoint.delete_endpoint(
h2o_mlops_client.DeployDeleteEndpointRequest(
name=endpoint.name
)
)

Feedback