Skip to main content

Tutorial 1B: Machine learning experiment scoring and analysis - financial focus

Introduction​

Overview​

Tools like ROC and Precision-Recall curves are used to assess how well classification models predict outcomes. In this tutorial, you will use a subset of the Freddie Mac Single-Family Loan-Level dataset to build a classification model and use it to predict if a loan will become delinquent. Through H2O Driverless AI's Diagnostic tool, you will examine the financial impacts the false positive and false negative predictions have while exploring tools like ROC Curve, Prec-Recall, Gain Charts, Lift Charts, and K-S Chart. Finally, we'll explore several metrics like AUC, F-Scores, GINI, MCC, and Log Loss to evaluate the generated model's performance.

note

To ensure familiarity with the content, it is recommended to go over the entire tutorial first before starting the experiment.

Objectives​

  • Model building: Construct a classification model using real-world loan data to predict loan delinquencies.
  • Performance evaluation: Employ various techniques to assess the effectiveness of the loan delinquency prediction model.
  • Financial impact analysis: Understand the financial implications of both accurate and inaccurate loan delinquency predictions.
  • Evaluation Metrics Mastery: Gain proficiency in utilizing key metrics like ROC-AUC, F1-scores, and Lift charts for model evaluation.

Prerequisites​

Feedback