Skip to main content

Task 11: Experiment autoDocs

H2O Driverless AI makes it easy to download the results of your experiments, all at the click of a button.

  1. Let’s explore the auto generated documents for this experiment. On the Experiment page select DOWNLOAD AUTODOC. download-autoreport This report provides insight into the training data and any detected shifts in distribution, the validation schema selected, model parameter tuning, feature evolution, and the final set of features chosen during the experiment.

  2. Open the report.docx file, this auto-generated report contains the following information:

  • Experiment Overview
  • Data Overview
  • Methodology
  • Data Sampling
  • Validation Strategy
  • Model Tuning
  • Feature Evolution
  • Feature Transformation
  • Final Model
  • Alternative Models
  • Deployment
  • Partial Dependence Plots
  • Appendix

  1. Make sure to examine and go through the contents of the report to understand the experiment better.

  2. This section dives into Feature Evolution and Feature Transformation within the experiment summary report. How does this information differ from what's available on the Experiments page?

  • The Experiments page focuses on managing experiment configurations. It allows you to:

    • Set the experiment name.
    • Upload or select a dataset.
    • View dataset dimensions (number of rows and columns).
    • Drop unwanted columns.
    • Select a validation dataset (and perform other validation-related actions).

  • In contrast, the experiment summary report offers insights specifically related to the training data. This includes:

    • Identification of any shifts in the data distribution.
    • Details about the validation schema.
    • Information on feature evolution and feature transformation (covered below).

  • Understanding Feature Evolution and Transformation. The experiment summary report delves deeper into two key aspects of feature engineering:

    • Feature evolution: This section details the algorithms used to create the experiment's features.
    • Feature transformation: This section provides information on how the system automatically generated new features that are potentially valuable for the given dataset.
  1. Find the section titled Final Model on the report.docx and explore the following items:
  • Table titled Performance of Final Model and determine the logloss final test score.
  • Validation Confusion Matrix.
  • Test Confusion Matrix.
  • Validation and Test ROC, Prec-Recall, lift, and gains plots.
Feedback