Skip to content

Shapley values support

Support for Shapley values in MLOps requires a model created with Driverless AI version 1.10 or later, and is supported for MOJO or Python pipeline artifacts. The following steps describe how to enable and request Shapley values.

  1. Enable Shapley values when deploying a model (for DAI MOJO pipeline only)

    If the Driverless AI MOJO pipeline artifact type (dai/mojo_pipeline) is selected when deploying a model, several runtimes that enable support for Shapley values are available. The selected runtime determines the type of Shapley value you can request in the following step. Depending on the selected runtime option, deploying with Shapley support doubles or triples the RAM requirements of the runtime. Note that you can skip this step if you're using a DAI Python pipeline.

    • H2O.ai MOJO scorer with Shapley values for transformed features (mojo_runtime_shapley_transformed): Generate Shapley values for features or columns that have been transformed by DAI.

    • H2O.ai MOJO scorer with Shapley values for all features (mojo_runtime_shapley_all): Generate Shapley values for either original or transformed features.

    • H2O.ai MOJO scorer with Shapley values for original features (mojo_runtime_shapley_original): Generate Shapley values for features or columns that existed as part of the original dataset or experiment.

    Info

    The following steps describe how to check which Shapley values have been enabled:

    1. Copy the endpoint URL of the deployment.
    2. In the endpoint URL, replace /score with /capabilities.
    3. Paste the endpoint URL in a browser window. One to three different terms are displayed that indicate whether the deployment supports Shapley values for original features and/or transformed features: [ SCORE, CONTRIBUTION_ORIGINAL, CONTRIBUTION_TRANSFORMED ]

  2. Request Shapley values in a curl request

    By default, Shapley values aren't returned in a curl request. To get Shapley values (that is, the Shapley type enabled in the preceding step), you must include the requestShapleyValueType argument in the curl request and set the value as either ORIGINAL or TRANSFORMED. Note that the specified value must correlate with the runtime selected in the preceding step.

    • ORIGINAL: Enables the ability to generate Shapley values for features or columns that existed as part of the original dataset or experiment.

      Example usage:

      "requestShapleyValueType": "ORIGINAL"

    • TRANSFORMED: Enables the ability to generate Shapley values for features or columns that have been transformed by DAI.

      Example usage:

      "requestShapleyValueType": "TRANSFORMED"

    Note

    By default, this value is set to NONE, which is the equivalent of not providing the requestShapleyValueType argument in the curl request.

    The following is a sample curl request and response with Shapley values enabled for original features:

    curl -X POST -H "Content-Type: application/json" -d @- DEPLOYMENT_ENDPOINT_URL << EOF
    {
    "fields": [
        "LIMIT_BAL",
        "SEX",
        "EDUCATION",
        "MARRIAGE",
        "AGE",
        "PAY_0",
        "PAY_2",
        "PAY_3",
        "PAY_4",
        "PAY_5",
        "PAY_6",
        "BILL_AMT1",
        "BILL_AMT2",
        "BILL_AMT3",
        "BILL_AMT4",
        "BILL_AMT5",
        "BILL_AMT6",
        "PAY_AMT2",
        "PAY_AMT3",
        "PAY_AMT4",
        "PAY_AMT5",
        "PAY_AMT6"
    ],
    "rows": [
        [
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0"
        ]
    ], "requestShapleyValueType": "ORIGINAL"
    }
    EOF

    {
        "featureShapleyContributions":
        {
            "contributionGroups":
            [
                {
                    "contributions":
                    [
                        [
                            "0.3031580540597976",
                            "0.05037104158009451",
                            "0.01197491002508829",
                            "-0.09613404645427222",
                            "0.03349942127192829",
                            "-0.19629869420775475",
                            "-0.05457586577961132",
                            "-0.016488709633310006",
                            "0.03022179422131117",
                            "-0.010334700480366232",
                            "-0.001831092307318766",
                            "0.24078515169214335",
                            "0.059130207887999234",
                            "-0.03856367964221913",
                            "-4.1371086615778267E-4",
                            "0.02549452684520287",
                            "-0.012600605263304759",
                            "0.17373018794999764",
                            "0.19368473683713824",
                            "0.1887846519524733",
                            "0.08455862402217218",
                            "0.04817053716929957",
                            "-1.4823175495435195"
                        ]
                    ]
                }
            ],
            "features":
            [
                "contrib_LIMIT_BAL",
                "contrib_SEX",
                "contrib_EDUCATION",
                "contrib_MARRIAGE",
                "contrib_AGE",
                "contrib_PAY_0",
                "contrib_PAY_2",
                "contrib_PAY_3",
                "contrib_PAY_4",
                "contrib_PAY_5",
                "contrib_PAY_6",
                "contrib_BILL_AMT1",
                "contrib_BILL_AMT2",
                "contrib_BILL_AMT3",
                "contrib_BILL_AMT4",
                "contrib_BILL_AMT5",
                "contrib_BILL_AMT6",
                "contrib_PAY_AMT2",
                "contrib_PAY_AMT3",
                "contrib_PAY_AMT4",
                "contrib_PAY_AMT5",
                "contrib_PAY_AMT6",
                "contrib_bias"
            ]
        },
        "fields":
        [
            "default payment next month.0",
            "default payment next month.1"
        ],
        "id": "f0395bc4-47d0-11ec-b7eb-fad6d6e23f65",
        "score":
        [
            [
                "0.6144353",
                "0.38556466"
            ]
        ]
    }