Model Diagnostics

ModelDiagnostics

Interact with model diagnostics in the Driverless AI server.

create

create(
    diagnose_experiment: Experiment, test_dataset: Dataset
) -> ModelDiagnostic

Creates a model diagnostic in the Driverless AI server.

Parameters:

• diagnose_experiment (Experiment) –

  Experiment to be diagnosed.

• test_dataset (Dataset) –

  Test dataset for the diagnosis.

Returns:

• ModelDiagnostic –

  Created model diagnostic.

create_async

create_async(
    diagnose_experiment: Experiment, test_dataset: Dataset
) -> ModelDiagnosticJob

Launches the creation of a model diagnostic in the Driverless AI server.

Parameters:

• diagnose_experiment (Experiment) –

  Experiment to be diagnosed.

• test_dataset (Dataset) –

  Test dataset for the diagnosis.

Returns:

• ModelDiagnosticJob –

  Started the model diagnostic job.

get

get(key: str) -> ModelDiagnostic

Retrieves a model diagnostic in the Driverless AI server.

Parameters:

• key (str) –

  The unique ID of the model diagnostic.

Returns:

• ModelDiagnostic –

  The model diagnostic corresponding to the key.

gui

gui() -> Hyperlink

Returns the full URL to the Model Diagnostics page in the Driverless AI server.

Returns:

• Hyperlink –

  The full URL to the Model Diagnostics page.

list

list(start_index: int = 0, count: int = None) -> Sequence[ModelDiagnostic]

Retrieves model diagnostics in the Driverless AI server.

Parameters:

• start_index (int, default: 0 ) –

  The index of the first model diagnostic to retrieve.

• count (int, default: None ) –

  The maximum number of model diagnostics to retrieve. If None, retrieves all available model diagnostics.

Returns:

• Sequence[ModelDiagnostic] –

  Model diagnostics.

ModelDiagnostic

A model diagnostic in the Driverless AI server.

experiment property

experiment: Experiment

Diagnosed experiment by the model diagnostic.

Returns:

• Experiment –

   

key property

key: str

Universally unique key of the entity.

Returns:

• str –

   

metric_plots property

metric_plots: ModelDiagnosticMetricPlots

Metric plots of the model diagnostic.

Beta API

A beta API that is subject to future changes.

Returns:

• ModelDiagnosticMetricPlots –

   

name property

name: str

Name of the entity.

Returns:

• str –

   

scores property

scores: Dict[str, Dict[str, float]]

Scores of the model diagnostic.

Returns:

• Dict[str, Dict[str, float]] –

   

test_dataset property

test_dataset: Dataset

Test dataset that was used for the model diagnostic.

Returns:

• Dataset –

   

delete

delete() -> None

Permanently deletes the model diagnostic from the Driverless AI server.

download_predictions

download_predictions(
    dst_dir: str = ".",
    dst_file: Optional[str] = None,
    file_system: Optional[AbstractFileSystem] = None,
    overwrite: bool = False,
) -> str

Downloads the predictions of the model diagnostic as a CSV file.

Parameters:

• dst_dir (str, default: '.' ) –

  The path where the CSV file will be saved.

• dst_file (Optional[str], default: None ) –

  The name of the CSV file (overrides the default file name).

• file_system (Optional[AbstractFileSystem], default: None ) –

  FSSPEC-based file system to download to instead of the local file system.

• overwrite (bool, default: False ) –

  Whether to overwrite or not if a file already exists.

Returns:

• str –

  Path to the downloaded CSV file.

gui

gui() -> Hyperlink

Returns the full URL to the model diagnostic page in the Driverless AI server.

Returns:

• Hyperlink –

  URL to the model diagnostic details page.

ModelDiagnosticMetricPlots

Interact with the metric plots of a model diagnostic in the Driverless AI server.

actual_vs_predicted_chart property

actual_vs_predicted_chart: Optional[Dict[str, Any]]

Actual vs predicted chart for the model.

Returns:

• Optional[Dict[str, Any]] –

  An actual vs predicted chart in Vega Lite (v3) format, or None is the model is a classification model.

gains_chart property

gains_chart: Optional[Dict[str, Any]]

Cumulative gains chart for the model.

Returns:

• Optional[Dict[str, Any]] –

  A cumulative gains chart in Vega Lite (v3) format, or None is the model is a classification model.

ks_chart property

ks_chart: Optional[Dict[str, Any]]

Kolmogorov-Smirnov chart of the model.

Returns:

• Optional[Dict[str, Any]] –

  A Kolmogorov-Smirnov chart in Vega Lite (v3) format, or None is the model is a classification model.

lift_chart property

lift_chart: Optional[Dict[str, Any]]

Lift chart of the model.

Returns:

• Optional[Dict[str, Any]] –

  A lift chart in Vega Lite (v3) format, or None is the model is a classification model.

prec_recall_curve property

prec_recall_curve: Optional[Dict[str, Any]]

Precision-recall curve of the model.

Returns:

• Optional[Dict[str, Any]] –

  A precision-recall curve in Vega Lite (v3) format, or None is the model is a classification model.

residual_histogram property

residual_histogram: Optional[Dict[str, Any]]

Residual Histogram of the model diagnostic.

Returns:

• Optional[Dict[str, Any]] –

  A residual histogram in Vega Lite (v3) format.

residual_plot property

residual_plot: Optional[Dict[str, Any]]

Residual plot with LOESS curve of the model.

Returns:

• Optional[Dict[str, Any]] –

  A residual plot in Vega Lite (v3) format, or None is the model is a classification model.

roc_curve property

roc_curve: Optional[Dict[str, Any]]

ROC curve of the model.

Returns:

• Optional[Dict[str, Any]] –

  A ROC curve in Vega Lite (v3) format, or None is the model is a classification model

confusion_matrix

confusion_matrix(threshold: float = None) -> Optional[List[List[Any]]]

Confusion matrix of the model.

Parameters:

• threshold (float, default: None ) –

  The threshold value.

Returns:

• Optional[List[List[Any]]] –

  A confusion matrix as a 2D list, or None is the model is a classification model