MLI¶
MLIExplainerId ¶
Bases: Enum
Enumeration represeting different MLI explainer IDs.
ABSOLUTE_PERMUTATION_BASED_FEATURE_IMPORTANCE
class-attribute
instance-attribute
¶
ABSOLUTE_PERMUTATION_BASED_FEATURE_IMPORTANCE = "h2oai.mli.byor.recipes.permutation_feat_imp_absolute_explainer.AbsolutePermutationFeatureImportanceExplainer"
Absolute Permutation-Based Feature Importance
DECISION_TREE
class-attribute
instance-attribute
¶
DECISION_TREE = "h2oaicore.mli.byor.recipes.surrogates.dt_surrogate_explainer.DecisionTreeSurrogateExplainer"
Decision Tree
FRIEDMAN_H_STATISTIC
class-attribute
instance-attribute
¶
FRIEDMAN_H_STATISTIC = "h2oai.mli.byor.recipes.h2o_sonar_explainers_friedman_h_statistic_explainer_FriedmanHStatisticExplainer"
Friedman's H-statistic
KERNEL_SHAPLEY_VALUES_FOR_ORIGINAL_FEATURES
class-attribute
instance-attribute
¶
KERNEL_SHAPLEY_VALUES_FOR_ORIGINAL_FEATURES = "h2oaicore.mli.byor.recipes.orig_kernel_shap_explainer.OriginalKernelShapExplainer"
Shapley Values for Original Features (Kernel SHAP Method)
NLP_LEAVE_ONE_COVARIATE_OUT
class-attribute
instance-attribute
¶
NLP_LEAVE_ONE_COVARIATE_OUT = "h2oaicore.mli.byor.recipes.text.nlp_loco_explainer_v2.NlpLocoExplainerVersion2"
NLP Leave-one-covariate-out (LOCO)
NLP_PARTIAL_DEPENDENCE_PLOT
class-attribute
instance-attribute
¶
NLP_PARTIAL_DEPENDENCE_PLOT = "h2oaicore.mli.byor.recipes.text.nlp_dai_pd_ice_explainer.NlpDaiPdIceExplainer"
NLP Partial Dependence Plot
NLP_TOKENIZER
class-attribute
instance-attribute
¶
NLP_TOKENIZER = "h2oaicore.mli.byor.recipes.text.nlp_tokenizer_explainer.NlpTokenizerExplainer"
NLP Tokenizer
NLP_VECTORIZER_LINEAR_MODEL
class-attribute
instance-attribute
¶
NLP_VECTORIZER_LINEAR_MODEL = "h2oaicore.mli.byor.recipes.text.nlp_vectorizer_linear_model_explainer.NlpVecLmExplainer"
NLP Vectorizer + Linear Model (VLM) Text Feature Importance
ORIGINAL_FEATURE_IMPORTANCE
class-attribute
instance-attribute
¶
ORIGINAL_FEATURE_IMPORTANCE = "h2oaicore.mli.byor.recipes.surrogates.original_feat_imp_explainer.OriginalFeatureImportanceExplainer"
Original Feature Importance
PARTIAL_DEPENDENCE_PLOT
class-attribute
instance-attribute
¶
PARTIAL_DEPENDENCE_PLOT = (
"h2oaicore.mli.byor.recipes.dai_pd_ice_explainer.DaiPdIceExplainer"
)
Partial Dependence Plot
RANDOM_FOREST_FEATURE_IMPORTANCE
class-attribute
instance-attribute
¶
RANDOM_FOREST_FEATURE_IMPORTANCE = "h2oaicore.mli.byor.recipes.surrogates.rf_feat_imp_explainer.RandomForestFeatureImportanceExplainer"
Random Forest Feature Importance
RANDOM_FOREST_PARTIAL_DEPENDENCE_PLOT
class-attribute
instance-attribute
¶
RANDOM_FOREST_PARTIAL_DEPENDENCE_PLOT = "h2oaicore.mli.byor.recipes.surrogates.rf_pd_explainer.RandomForestPartialDependenceExplainer"
Random Forest Partial Dependence Plot
RELATIVE_PERMUTATION_BASED_FEATURE_IMPORTANCE
class-attribute
instance-attribute
¶
RELATIVE_PERMUTATION_BASED_FEATURE_IMPORTANCE = "h2oai.mli.byor.recipes.permutation_feat_imp_relative_explainer.RelativePermutationFeatureImportanceExplainer"
Relative Permutation-Based Feature Importance
SHAPLEY_SUMMARY_PLOT_FOR_ORIGINAL_FEATURES
class-attribute
instance-attribute
¶
SHAPLEY_SUMMARY_PLOT_FOR_ORIGINAL_FEATURES = "h2oaicore.mli.byor.recipes.shapley_summary_explainer.ShapleySummaryOrigFeatExplainer"
Shapley Summary Plot for Original Features (Naive Shapley Method)
SHAPLEY_VALUES_FOR_ORIGINAL_FEATURES
class-attribute
instance-attribute
¶
SHAPLEY_VALUES_FOR_ORIGINAL_FEATURES = "h2oaicore.mli.byor.recipes.original_contrib_explainer.NaiveShapleyExplainer"
Shapley Values for Original Features (Naive Method)
SHAPLEY_VALUES_FOR_TRANSFORMED_FEATURES
class-attribute
instance-attribute
¶
SHAPLEY_VALUES_FOR_TRANSFORMED_FEATURES = "h2oaicore.mli.byor.recipes.transformed_shapley_explainer.TransformedShapleyExplainer"
Shapley Values for Transformed Features
SURROGATE_RANDOM_FOREST_LEAVE_ONE_COVARIATE_OUT
class-attribute
instance-attribute
¶
SURROGATE_RANDOM_FOREST_LEAVE_ONE_COVARIATE_OUT = "h2oaicore.mli.byor.recipes.surrogates.rf_loco_explainer.RandomForestLocoExplainer"
Surrogate Random Forest Leave-one-covariate-out (LOCO)
TRANSFORMED_FEATURE_IMPORTANCE
class-attribute
instance-attribute
¶
TRANSFORMED_FEATURE_IMPORTANCE = "h2oaicore.mli.byor.recipes.transformed_feat_imp_explainer.TransformedFeatureImportanceExplainer"
Transformed Feature Importance
MLI ¶
Interact with MLI interpretations on the Driverless AI server.
iid
property
¶
iid: IIDMethods
Retrieve IID interpretations.
Deprecated API
A deprecated API that will be removed from v1.10.7 onwards. Please use 'driverlessai._mli.MLI' instead.
Returns:
-
IIDMethods
–
create ¶
create(
experiment: Experiment | None = None,
dataset: Dataset | None = None,
name: str | None = None,
force: bool = False,
**kwargs: Any
) -> Interpretation
Create an MLI interpretation on the Driverless AI server and return an Interpretation object corresponding to the created interpretation.
Parameters:
-
experiment
(Experiment | None
, default:None
) –The experiment to interpret. Will use training dataset if the
dataset
has not been specified. -
dataset
(Dataset | None
, default:None
) –The dataset to use for the interpretation (if dataset includes target and prediction columns, then an experiment will not be needed).
-
name
(str | None
, default:None
) –The display name for the interpretation.
-
force
(bool
, default:False
) –Create the new interpretation even if the interpretation with the same name already exists.
Other Parameters:
-
explainers
(List[ExplainerRecipe]
) –The list of explainer recipe objects.
-
test_dataset
(Dataset
) –Dataset object (timeseries only).
-
target_column
(str
) –The name of the column in
dataset
. -
prediction_column
(str
) –The name of the column in
dataset
. -
weight_column
(str
) –The name of the column in
dataset
. -
drop_columns
(List[str]
) –The names of the columns in
dataset
.
Note
Any expert setting can also be passed as a kwarg
.
To search possible expert settings for your server version,
use mli.search_expert_settings(search_term)
.
create_async ¶
create_async(
experiment: Experiment | None = None,
dataset: Dataset | None = None,
name: str | None = None,
force: bool = False,
validate_settings: bool = True,
**kwargs: Any
) -> Interpretation
Launch creation of an MLI interpretation on the Driverless AI server and return an Interpretation object to track the status.
Parameters:
-
experiment
(Experiment | None
, default:None
) –The experiment to interpret. Will use training dataset if the
dataset
has not been specified. -
dataset
(Dataset | None
, default:None
) –The dataset to use for the interpretation (if dataset includes target and prediction columns, then an experiment will not be needed).
-
name
(str | None
, default:None
) –The display name for the interpretation.
-
force
(bool
, default:False
) –Create the new interpretation even if the interpretation with the same name already exists.
Other Parameters:
-
explainers
(List[ExplainerRecipe]
) –The list of explainer recipe objects.
-
test_dataset
(Dataset
) –Dataset object (timeseries only).
-
target_column
(str
) –The name of the column in
dataset
. -
prediction_column
(str
) –The name of the column in
dataset
. -
weight_column
(str
) –The name of the column in
dataset
. -
drop_columns
(List[str]
) –The names of the columns in
dataset
.
Note
Any expert setting can also be passed as a kwarg
.
To search possible expert settings for your server version,
use mli.search_expert_settings(search_term)
.
get ¶
get(key: str) -> Interpretation
Initialize an Interpretation object but do not request information from the server (it is possible for the interpretation key to not exist on the server). This is useful for populating lists without making multiple network calls.
Parameters:
-
key
(str
) –Driverless AI server's unique ID for the MLI interpretation.
list ¶
list(start_index: int = 0, count: int = None) -> Sequence[Interpretation]
Interpretation ¶
Interact with an MLI interpretation on the Driverless AI server.
artifacts
property
¶
artifacts: InterpretationArtifacts
Interact with artifacts that are created when the interpretation completes.
Returns:
creation_timestamp
property
¶
creation_timestamp: float
Creation timestamp in seconds since the epoch (POSIX timestamp).
Returns:
-
float
–
dataset
property
¶
dataset: Dataset | None
Dataset for the interpretation.
Returns:
-
Dataset | None
–
experiment
property
¶
experiment: Experiment | None
Experiment for the interpretation.
Returns:
-
Experiment | None
–
explainers
property
¶
explainers: ExplainerList
Explainers that were ran as an ExplainerList
object.
Driverless AI version requirement
Requires Driverless AI server 1.10.5 or higher.
Beta API
A beta API that is subject to future changes.
Returns:
explanation_plots
property
¶
explanation_plots: Mapping[int | str, list[ExplanationPlot]]
Plots for explanations that were created for the interpretation.
Example: Retrieve global and local explanation plot
# get the list of available plots for the Decision Tree explainer
MLIExplainerId = driverlessai.MLIExplainerId
dt_plots = interpretation.explanation_plots[MLIExplainerId.DECISION_TREE]
# retrieve the global explanation
dt_plot = dt_plots[0].get_plot()
# get the local explanation (for row 44)
dt_local_plot = dt_plots[0].get_plot(row_number=44)
Driverless AI version requirement
Requires Driverless AI server 1.10.5 or higher.
Beta API
A beta API that is subject to future changes.
Returns:
-
Mapping[int | str, list[ExplanationPlot]]
–
gui ¶
gui() -> Hyperlink
Get the full URL for the interpretation's page on the Driverless AI server.
is_complete ¶
is_complete() -> bool
Whether the job has been completed successfully.
Returns:
-
bool
–True
if the job has been completed successfully, otherwiseFalse
.
is_running ¶
is_running() -> bool
Whether the job has been scheduled or is running, finishing, or syncing.
Returns:
-
bool
–True
if the job has not completed yet, otherwiseFalse
.
parameter_summary ¶
parameter_summary() -> Table
Get MLI summary.
Driverless AI version requirement
Requires Driverless AI server 1.10.5 or higher.
rename ¶
rename(name: str) -> Interpretation
Change the interpretation display name.
Parameters:
-
name
(str
) –New display name.
result ¶
result(silent: bool = False) -> Interpretation
Wait for the job to complete, then return an Interpretation object.
InterpretationArtifacts ¶
Interact with files created by an MLI interpretation on the Driverless AI server.
file_paths
property
¶
download ¶
download(
only: str | list[str] = None,
dst_dir: str = ".",
file_system: AbstractFileSystem | None = None,
overwrite: bool = False,
) -> dict[str, str]
Download interpretation artifacts from the Driverless AI server. Returns a dictionary of relative paths for the downloaded artifacts.
Parameters:
-
only
(str | list[str]
, default:None
) –Specify the specific artifacts to download, use
interpretation.artifacts.list()
to see the available artifacts on the Driverless AI server. -
dst_dir
(str
, default:'.'
) –The path to the directory where the interpretation artifacts will be saved.
-
file_system
(AbstractFileSystem | None
, default:None
) –FSSPEC based file system to download to, instead of local file system.
-
overwrite
(bool
, default:False
) –Overwrite existing files.
Explainer ¶
Interact with an MLI explainers on the Driverless AI server.
explanation_plots
property
¶
explanation_plots: list[ExplanationPlot]
The avaialbe plots for this explainer.
Returns:
frames
property
¶
frames: ExplainerFrames | None
An ExplainerFrames
object that contains the paths to the explainer
frames retrieved from Driverless AI Server. If the explainer frame is not
available, the value of this propertiy is None
.
Returns:
-
ExplainerFrames | None
–
get_data ¶
get_data(**kwargs: Any) -> ExplainerData
Retrieve the ExplainerData
from the Driverless AI server.
Raises a RuntimeError
exception if the explainer has not been completed
successfully.
Use help(explainer.get_data)
to view help on available keyword arguments.
is_complete ¶
is_complete() -> bool
Whether the job has been completed successfully.
Returns:
-
bool
–True
if the job has been completed successfully, otherwiseFalse
.
is_running ¶
is_running() -> bool
Whether the job has been scheduled or is running, finishing, or syncing.
Returns:
-
bool
–True
if the job has not completed yet, otherwiseFalse
.
ExplainerArtifacts ¶
Interact with artifacts created by an explainer during interpretation on the Driverless AI server.
file_paths
property
¶
download ¶
download(
only: str | list[str] = None,
dst_dir: str = ".",
file_system: AbstractFileSystem | None = None,
overwrite: bool = False,
timeout: float = 30,
) -> dict[str, str]
Download explainer artifacts from the Driverless AI server. Returns a dictionary of relative paths for the downloaded artifacts.
Parameters:
-
only
(str | list[str]
, default:None
) –Specify the specific artifacts to download, use
interpretation.artifacts.list()
to see the available artifacts on the Driverless AI server. -
dst_dir
(str
, default:'.'
) –The path to the directory where the interpretation artifacts will be saved.
-
file_system
(AbstractFileSystem | None
, default:None
) –FSSPEC based file system to download to, instead of local file system.
-
overwrite
(bool
, default:False
) –Overwrite existing files.
-
timeout
(float
, default:30
) –Connection timeout in seconds.
ExplainerData ¶
Interact with the result data of an explainer on the Driverless AI server.
data_as_dict ¶
Return the explainer result data as a dictionary.
ExplainerFrames ¶
Interact with explanation frames created by an explainer during interpretation on the Driverless AI server.
frame_paths
property
¶
download ¶
download(
frame_name: str | list[str] = None,
dst_dir: str = ".",
file_system: AbstractFileSystem | None = None,
overwrite: bool = False,
timeout: float = 30,
) -> dict[str, str]
Download explainer frames from the Driverless AI server. Returns a dictionary of relative paths for the downloaded artifacts.
Parameters:
-
frame_name
(str | list[str]
, default:None
) –Specify the specific frame to download, use
explainer.frames.list()
to see the available artifacts on the Driverless AI server. -
dst_dir
(str
, default:'.'
) –The path to the directory where the interpretation artifacts will be saved.
-
file_system
(AbstractFileSystem | None
, default:None
) –Optional["fsspec.spec.AbstractFileSystem"] = None.
-
overwrite
(bool
, default:False
) –Overwrite existing files.
-
timeout
(float
, default:30
) –Connection timeout in seconds.
frame_as_pandas ¶
frame_as_pandas(
frame_name: str,
custom_tmp_dir: str | None = None,
keep_downloaded: bool = False,
) -> DataFrame
Download a frame with the given frame name to a temporary directory and
return it as a pandas.DataFrame
.
Parameters:
-
frame_name
(str
) –The name of the frame to open.
-
custom_tmp_dir
(str | None
, default:None
) –If specified, use this directory as the temporary directory instead of the default.
-
keep_downloaded
(bool
, default:False
) –If
True
, do not delete the downloaded frame. Otherwise, the downloaded frame is deleted before returning from this method.
Beta API
A beta API that is subject to future changes.
ExplainerList ¶
ExplanationPlot ¶
This abstract class serves as the foundation for all explanation plot classes to inherit from.
ShapleyExplanationPlot ¶
Handles plots for the following explainers:
- Shapley Values for Original Features (Naive Method)
- Shapley Values for Transformed Features
- Shapley Values for Original Features (Kernel SHAP Method)
get_plot ¶
Plot this explanation
Parameters:
-
row_number
(int | None
, default:None
) –Local explanation for the given row_number.
-
class_name
(str | None
, default:None
) –The name of the class in multinomial classification, if not provided the first class from the set of available classes in the model will be selected (use the method
get_classes()
to view available classes).
Returns:
NLPTokenizerExplanationPlot ¶
Handles plots for the following explainers:
- NLP Tokenizer
FeatureImportanceGlobalExplanationPlot ¶
Handles plots for the following explainers:
- Absolute Permutation-Based Feature Importance
- Relative Permutation-Based Feature Importance
- Friedman's H-statistic
get_plot ¶
Plot this explanation
Parameters:
-
class_name
(str | None
, default:None
) –The name of the class in multinomial classification, if not provided the first class from the set of available classes in the model will be selected (use the method
get_classes()
to view available classes).
Returns:
DAIFeatureImportanceExplanationPlot ¶
RandomForestFeatureImportanceExplanationPlot ¶
Handles plots for the following explainers:
- Random Forest Feature Importance
- Surrogate Random Forest Leave-one-covariate-out (LOCO)
NLPFeatureImportanceExplanationPlot ¶
Handles plots for the following explainers:
- NLP Vectorizer + Linear Model (VLM) Text Feature Importance
- NLP Leave-one-covariate-out (LOCO)
get_plot ¶
get_plot(
*,
row_number: int | None = None,
class_name: str | None = None,
text_feature: str | None = None
) -> dict[str, Any]
Plot this explanation
Parameters:
-
row_number
(int | None
, default:None
) –Local explanation for the given row_number.
-
class_name
(str | None
, default:None
) –The name of the class in multinomial classification, if not provided the first class from the set of available classes in the model will be selected (use the method
get_classes()
to view available classes). -
text_features
–Select the text feature to plot, if not provided all text features will be selected (use the method
get_text_features
to view available text features)
Returns:
PDPExplanationPlot ¶
Handles plots for the following explainers:
- Partial Dependence Plot
- NLP Partial Dependence Plot
get_plot ¶
get_plot(
*,
row_number: int | None = None,
class_name: str | None = None,
feature_name: str | None = None,
partial_dependence_type: str | None = None
) -> dict[str, Any]
Plot this explanation
Parameters:
-
row_number
(int | None
, default:None
) –Local explanation for the given row_number.
-
class_name
(str | None
, default:None
) –The name of the class in multinomial classification, if not provided the first class from the set of available classes in the model will be selected (use the method
get_classes()
to view available classes). -
feature_name
(str | None
, default:None
) –The name of the feature to plot, if not provided the first feature from the set of avaialbe features in the model will be selected (use the method
get_feature_names()
to view available feature names) -
partial_dependence_type
(str | None
, default:None
) –Override default plot type, available options are categorical and numeric
Returns:
RandomForestPDPExplanationPlot ¶
Handles plots for the following explainer:
- Random Forest Partial Dependence Plot
get_plot ¶
Plot this explanation
Parameters:
-
row_number
(int | None
, default:None
) –Local explanation for the given row_number.
-
feature_name
(str | None
, default:None
) –The name of the feature to plot, if not provided the first feature from the set of avaialbe features in the model will be selected (use the method
get_feature_names()
to view available feature names)
Returns:
DecisionTreeExplanationPlot ¶
Handles plots for the following explainer:
- Decision Tree
get_plot ¶
Plot this explanation
Parameters:
-
row_number
(int | None
, default:None
) –Local explanation for the given row_number.
-
class_name
(str | None
, default:None
) –The name of the class in multinomial classification, if not provided the first class from the set of available classes in the model will be selected (use the method
get_classes()
to view available classes).
Returns:
ShapleySummaryExplanationPlot ¶
Handles plots for the following explainer:
- Shapley Summary Plot for Original Features
get_plot ¶
Plot this explanation
Parameters:
-
class_name
(str | None
, default:None
) –The name of the class in multinomial classification, if not provided the first class from the set of available classes in the model will be selected (use the method
get_classes()
to view available classes). -
page_num
(int
, default:1
) –Select the page (use the method
get_total_pages()
to view the total available number of pages)
Returns: