Skip to main content
Version: v1.0.0

Shapley values support

Shapley value support in MLOps requires a model created with H2O-3 or Driverless AI version 1.10 or later, and is available for both MOJO and Python pipeline artifacts.

The following steps describe how to enable and request Shapley values.

Step 1: Enable Shapley values when deploying a model​

For DAI experiments​

If the Driverless AI MOJO pipeline artifact type (dai/mojo_pipeline) is selected when deploying a model, several runtimes that enable support for Shapley values are available. The selected runtime determines the type of Shapley value you can request in the following step. Depending on the selected runtime option, deploying with Shapley support doubles or triples the RAM requirements of the runtime.

note

You can skip this step if you're using a DAI Python pipeline.

  • H2O.ai MOJO scorer with Shapley values for transformed features (mojo_runtime_shapley_transformed): Generate Shapley values for features or columns that have been transformed by DAI.

  • H2O.ai MOJO scorer with Shapley values for all features (mojo_runtime_shapley_all): Generate Shapley values for either original or transformed features.

  • H2O.ai MOJO scorer with Shapley values for original features (mojo_runtime_shapley_original): Generate Shapley values for features or columns that existed as part of the original dataset or experiment.

For H2O-3 MOJO experiments​

If the H2O-3 MOJO artifact type (h2o3/mojo) is selected when deploying a model, only one runtime is available that enables support for Shapley values: h2o3_mojo_runtime_shapley_transformed.

  • H2O-3 MOJO scorer with Shapley values for transformed features (h2o3_mojo_runtime_shapley_transformed): Generate Shapley values for features or columns transformed during training.
note
  • H2O MLOps supports Shapley TRANSFORMED values in H2O-3 for MOJO experiments.
  • Shapley values are supported only for the following model types:
    • DRF
    • GBM
    • XGBoost
  • Shapley values are not supported for the following models:
    • Multinominal classification models
    • Binominal models with Binominal Double Trees parameter set
    • GLM models

Step 2: Request Shapley values in a curl request​

By default, Shapley values aren't returned in a curl request. To get Shapley values (that is, the Shapley type enabled in the preceding step), you must include the requestShapleyValueType argument in the curl request and set the value as either ORIGINAL or TRANSFORMED.

Note

  • The specified value must correlate with the runtime selected in the preceding step.

  • The following steps describe how to check which Shapley values have been enabled:

    1. Copy the endpoint URL of the deployment.
    2. In the endpoint URL, replace /score with /capabilities.
    3. Paste the endpoint URL in a browser window. One to three different terms are displayed that indicate whether the deployment supports Shapley values for original features and/or transformed features: [ SCORE, CONTRIBUTION_ORIGINAL, CONTRIBUTION_TRANSFORMED ]

  • To try this using the H2O MLOps Python client, see View scorer capabilities.

ORIGINAL​

ORIGINAL enables the ability to generate Shapley values for features or columns that existed as part of the original dataset or experiment.

Example usage:

"requestShapleyValueType": "ORIGINAL"

TRANSFORMED​

TRANSFORMED enables the ability to generate Shapley values for features or columns that have been transformed by DAI.

Example usage:

"requestShapleyValueType": "TRANSFORMED"

note
  • By default, this value is set to NONE, which is the equivalent of not providing the requestShapleyValueType argument in the curl request.
  • To try this using the H2O MLOps Python client, see Shapley values.

The following is a sample curl request and response with Shapley values enabled for original features:

curl -X POST -H "Content-Type: application/json" -d @- <SCORING_ENDPOINT_URL> << EOF
    {
    "fields": [
        "LIMIT_BAL",
        "SEX",
        "EDUCATION",
        "MARRIAGE",
        "AGE",
        "PAY_0",
        "PAY_2",
        "PAY_3",
        "PAY_4",
        "PAY_5",
        "PAY_6",
        "BILL_AMT1",
        "BILL_AMT2",
        "BILL_AMT3",
        "BILL_AMT4",
        "BILL_AMT5",
        "BILL_AMT6",
        "PAY_AMT2",
        "PAY_AMT3",
        "PAY_AMT4",
        "PAY_AMT5",
        "PAY_AMT6"
    ],
    "rows": [
        [
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0"
        ]
    ], "requestShapleyValueType": "ORIGINAL"
    }
    EOF

    {
        "featureShapleyContributions":
        {
            "contributionGroups":
            [
                {
                    "contributions":
                    [
                        [
                            "0.3031580540597976",
                            "0.05037104158009451",
                            "0.01197491002508829",
                            "-0.09613404645427222",
                            "0.03349942127192829",
                            "-0.19629869420775475",
                            "-0.05457586577961132",
                            "-0.016488709633310006",
                            "0.03022179422131117",
                            "-0.010334700480366232",
                            "-0.001831092307318766",
                            "0.24078515169214335",
                            "0.059130207887999234",
                            "-0.03856367964221913",
                            "-4.1371086615778267E-4",
                            "0.02549452684520287",
                            "-0.012600605263304759",
                            "0.17373018794999764",
                            "0.19368473683713824",
                            "0.1887846519524733",
                            "0.08455862402217218",
                            "0.04817053716929957",
                            "-1.4823175495435195"
                        ]
                    ]
                }
            ],
            "features":
            [
                "contrib_LIMIT_BAL",
                "contrib_SEX",
                "contrib_EDUCATION",
                "contrib_MARRIAGE",
                "contrib_AGE",
                "contrib_PAY_0",
                "contrib_PAY_2",
                "contrib_PAY_3",
                "contrib_PAY_4",
                "contrib_PAY_5",
                "contrib_PAY_6",
                "contrib_BILL_AMT1",
                "contrib_BILL_AMT2",
                "contrib_BILL_AMT3",
                "contrib_BILL_AMT4",
                "contrib_BILL_AMT5",
                "contrib_BILL_AMT6",
                "contrib_PAY_AMT2",
                "contrib_PAY_AMT3",
                "contrib_PAY_AMT4",
                "contrib_PAY_AMT5",
                "contrib_PAY_AMT6",
                "contrib_bias"
            ]
        },
        "fields":
        [
            "default payment next month.0",
            "default payment next month.1"
        ],
        "id": "f0395bc4-47d0-11ec-b7eb-fad6d6e23f65",
        "score":
        [
            [
                "0.6144353",
                "0.38556466"
            ]
        ]
    }
Feedback