Shapley values support
Support for Shapley values in MLOps requires a model created with Driverless AI version 1.10 or later, and is supported for MOJO or Python pipeline artifacts. The following steps describe how to enable and request Shapley values.
Enable Shapley values when deploying a model (for DAI MOJO pipeline only)
If the Driverless AI MOJO pipeline artifact type (
dai/mojo_pipeline
) is selected when deploying a model, several runtimes that enable support for Shapley values are available. The selected runtime determines the type of Shapley value you can request in the following step. Depending on the selected runtime option, deploying with Shapley support doubles or triples the RAM requirements of the runtime. Note that you can skip this step if you're using a DAI Python pipeline.H2O.ai MOJO scorer with Shapley values for transformed features (
mojo_runtime_shapley_transformed
): Generate Shapley values for features or columns that have been transformed by DAI.H2O.ai MOJO scorer with Shapley values for all features (
mojo_runtime_shapley_all
): Generate Shapley values for either original or transformed features.H2O.ai MOJO scorer with Shapley values for original features (
mojo_runtime_shapley_original
): Generate Shapley values for features or columns that existed as part of the original dataset or experiment.
The following steps describe how to check which Shapley values have been enabled:
- Copy the endpoint URL of the deployment.
- In the endpoint URL, replace
/score
with/capabilities
. - Paste the endpoint URL in a browser window. One to three different terms are displayed that indicate whether the deployment supports Shapley values for original features and/or transformed features: [ SCORE, CONTRIBUTION_ORIGINAL, CONTRIBUTION_TRANSFORMED ]
Request Shapley values in a
curl
requestBy default, Shapley values aren't returned in a
curl
request. To get Shapley values (that is, the Shapley type enabled in the preceding step), you must include therequestShapleyValueType
argument in thecurl
request and set the value as eitherORIGINAL
orTRANSFORMED
. Note that the specified value must correlate with the runtime selected in the preceding step.ORIGINAL
: Enables the ability to generate Shapley values for features or columns that existed as part of the original dataset or experiment.Example usage:
"requestShapleyValueType": "ORIGINAL"
TRANSFORMED
: Enables the ability to generate Shapley values for features or columns that have been transformed by DAI.Example usage:
"requestShapleyValueType": "TRANSFORMED"
By default, this value is set to NONE
, which is the equivalent of not providing the requestShapleyValueType
argument in the curl
request.
The following is a sample curl
request and response with Shapley values enabled for original features:
curl -X POST -H "Content-Type: application/json" -d @- DEPLOYMENT_ENDPOINT_URL << EOF
{
"fields": [
"LIMIT_BAL",
"SEX",
"EDUCATION",
"MARRIAGE",
"AGE",
"PAY_0",
"PAY_2",
"PAY_3",
"PAY_4",
"PAY_5",
"PAY_6",
"BILL_AMT1",
"BILL_AMT2",
"BILL_AMT3",
"BILL_AMT4",
"BILL_AMT5",
"BILL_AMT6",
"PAY_AMT2",
"PAY_AMT3",
"PAY_AMT4",
"PAY_AMT5",
"PAY_AMT6"
],
"rows": [
[
"0",
"0",
"0",
"0",
"0",
"0",
"0",
"0",
"0",
"0",
"0",
"0",
"0",
"0",
"0",
"0",
"0",
"0",
"0",
"0",
"0",
"0"
]
], "requestShapleyValueType": "ORIGINAL"
}
EOF
{
"featureShapleyContributions":
{
"contributionGroups":
[
{
"contributions":
[
[
"0.3031580540597976",
"0.05037104158009451",
"0.01197491002508829",
"-0.09613404645427222",
"0.03349942127192829",
"-0.19629869420775475",
"-0.05457586577961132",
"-0.016488709633310006",
"0.03022179422131117",
"-0.010334700480366232",
"-0.001831092307318766",
"0.24078515169214335",
"0.059130207887999234",
"-0.03856367964221913",
"-4.1371086615778267E-4",
"0.02549452684520287",
"-0.012600605263304759",
"0.17373018794999764",
"0.19368473683713824",
"0.1887846519524733",
"0.08455862402217218",
"0.04817053716929957",
"-1.4823175495435195"
]
]
}
],
"features":
[
"contrib_LIMIT_BAL",
"contrib_SEX",
"contrib_EDUCATION",
"contrib_MARRIAGE",
"contrib_AGE",
"contrib_PAY_0",
"contrib_PAY_2",
"contrib_PAY_3",
"contrib_PAY_4",
"contrib_PAY_5",
"contrib_PAY_6",
"contrib_BILL_AMT1",
"contrib_BILL_AMT2",
"contrib_BILL_AMT3",
"contrib_BILL_AMT4",
"contrib_BILL_AMT5",
"contrib_BILL_AMT6",
"contrib_PAY_AMT2",
"contrib_PAY_AMT3",
"contrib_PAY_AMT4",
"contrib_PAY_AMT5",
"contrib_PAY_AMT6",
"contrib_bias"
]
},
"fields":
[
"default payment next month.0",
"default payment next month.1"
],
"id": "f0395bc4-47d0-11ec-b7eb-fad6d6e23f65",
"score":
[
[
"0.6144353",
"0.38556466"
]
]
}
- Submit and view feedback for this page
- Send feedback about H2O MLOps to cloud-feedback@h2o.ai