Skip to main content
Version: v0.64.0

Configure scoring endpoints

This page describes how to use the H2O MLOps Python client to configure scoring endpoints.

Create an endpoint​

This example assumes that you know the following values:

PROJECT_ID = "..."
ENVIRONMENT_ID = "..."
DEPLOYMENT_ID = "..."

Endpoints use resource names for configuration. The following is an example of how you can convert the unique IDs to resource names.

# environment_resource_name = f"projects/{PROJECT_ID}/environments/{ENVIRONMENT_ID}"
# deployment_resource_name = f"{environment_resource_name}/deployments/{DEPLOYMENT_ID}"
environment_resource_name = f"projects/{project_1.id}/environments/{env_1.id}"
deployment_resource_name = f"{environment_resource_name}/deployments/{deployment_1.id}"

  • The following is an example of the output from inputting environment_resource_name.

    environment_resource_name
    'projects/96b2fffa-7932-4a3f-9db0-48d60c193838/environments/c682855f-4e8f-4027-91ad-dbe2c88b0ff7'

  • The following is an example of the output from inputting deployment_resource_name.

    deployment_resource_name
    'projects/96b2fffa-7932-4a3f-9db0-48d60c193838/environments/c682855f-4e8f-4027-91ad-dbe2c88b0ff7/deployments/d44760ec-8164-47c4-bc45-5b695380c229'

Define and request the creation of an endpoint​

The following is an example of how you can define an endpoint and have it be accessible at the path abc/xyz.

endpoint_definition = h2o_mlops_client.DeployConfigurableEndpoint(
    display_name="My Custom Endpoint",
    description="This is my custom endpoint.",
    path="abc/xyz",
    target=deployment_resource_name
)

You can then submit a request to create that endpoint in your target environment:

endpoint = mlops.deployer.endpoint.create_endpoint(
    mlops.DeployCreateEndpointRequest(
        parent=environment_resource_name,
        endpoint=endpoint_definition
    )
).endpoint
endpoint
{'create_time': datetime.datetime(2023, 2, 16, 19, 16, 35, 458084, tzinfo=tzlocal()),
 'description': 'This is my custom endpoint.',
 'display_name': 'My Custom Endpoint',
 'name': 'projects/96b2fffa-7932-4a3f-9db0-48d60c193838/environments/c682855f-4e8f-4027-91ad-dbe2c88b0ff7/endpoints/f5df8be6-32de-4254-baa7-0b0c1549b753',
 'path': 'abc/xyz',
 'target': 'projects/96b2fffa-7932-4a3f-9db0-48d60c193838/environments/c682855f-4e8f-4027-91ad-dbe2c88b0ff7/deployments/d44760ec-8164-47c4-bc45-5b695380c229',
 'update_time': datetime.datetime(2023, 2, 16, 19, 16, 35, 458415, tzinfo=tzlocal())}

Use endpoints​

Once you have an endpoint, it provides a static URL that can be pointed to any deployment in the deployment environment of the endpoint.

note

Endpoints take a little time to become ready to recieve HTTP requests. We have a request in place for a status indicator of readiness.

static_url = f"https://model.cloud-qa.h2o.ai/{endpoint.path}/model"
static_url
'https://model.cloud-qa.h2o.ai/abc/xyz/model'

Any of the model methods available from the deployment (for example, capabilities, schema, id, sample_request, and score) are available at the endpoint.

sample_request = httpx.get(f"{static_url}/sample_request").json()

httpx.post(
    url=f"{static_url}/score",
    json=sample_request
).json()

Sample return:

{'fields': ['default payment next month.0', 'default payment next month.1'],
 'id': 'bbc16b62-7730-11ed-8a78-ca18b025b240',
 'score': [['0.9472957', '0.052704293']]}

Update an endpoint​

The following is an example of how you can update the endpoint so that same URI routes to a different model deployment. This example assumes that you know another model's deployment ID (referred to here as ANOTHER_DEPLOYMENT_ID).

another_deployment_resource_name = f"{environment_resource_name}/deployments/{ANOTHER_DEPLOYMENT_ID}"
endpoint.target = another_deployment_resource_name
endpoint = mlops.deployer.endpoint.update_endpoint(
    h2o_mlops_client.DeployUpdateEndpointRequest(
        endpoint=endpoint
    )
).endpoint

Scoring requests to the static https://model.cloud-qa.h2o.ai/abc/xyz/model URI will now direct to the new deployment.

List all endpoints in an environment​

The following is an example of how to list all the endpoints that exist in an environment.

mlops.deployer.endpoint.list_endpoints(
    h2o_mlops_client.DeployListEndpointsRequest(
        parent=environment_resource_name
    )
)

Sample return:

{'endpoints': [{'create_time': datetime.datetime(2023, 2, 16, 19, 16, 35, 458084, tzinfo=tzlocal()),
                'description': 'This is my custom endpoint.',
                'display_name': 'My Custom Endpoint',
                'name': 'projects/96b2fffa-7932-4a3f-9db0-48d60c193838/environments/c682855f-4e8f-4027-91ad-dbe2c88b0ff7/endpoints/f5df8be6-32de-4254-baa7-0b0c1549b753',
                'path': 'abc/xyz',
                'target': 'projects/96b2fffa-7932-4a3f-9db0-48d60c193838/environments/c682855f-4e8f-4027-91ad-dbe2c88b0ff7/deployments/d44760ec-8164-47c4-bc45-5b695380c229',
                'update_time': datetime.datetime(2023, 2, 16, 19, 18, 40, 360409, tzinfo=tzlocal())}],
 'next_page_token': ''}

Get an endpoint​

The following is an example of how you can get information about individual endpoints by their resource name.

endpoint = mlops.deployer.endpoint.get_endpoint(
    h2o_mlops_client.DeployGetEndpointRequest(
        name=endpoint.name
    )
).endpoint

Delete an endpoint​

You can pass in the name of an endpoint to delete it. The following is an example of how to delete an endpoint.

mlops.deployer.endpoint.delete_endpoint(
    h2o_mlops_client.DeployDeleteEndpointRequest(
        name=endpoint.name
    ) 
)
Feedback