Skip to main content

Impute data

The impute function lets you perform in-place imputation by filling missing values with aggregates computed on the na.rm’d vector. Additionally, you can perform imputation based on groupings of columns from within the dataset. These columns can be passed by index or by column name to the by parameter.

The impute function accepts the following arguments:

  • by: The group by columns.

  • column: A specific column to impute. The default of 0 specifies to impute the entire frame.

  • combine_method: (method must be set to median) This option will choose how to combine quantiles on even sample sizes. This parameter is ignored in all other cases. The available options are:

    • average
    • high
    • interpolate
    • low

  • dataset: The dataset containing the column to impute.

  • groupByFrame or group_by_frame: Impute the column with this pre-computed grouped frame.

  • method: The type of imputation to perform. The available options are:

    • mean: Replaces NA values with the column mean.
    • median: Replaces NA values with the column median.
    • mode: Replaces NA values with the most common factor (for factor columns only).
note

If a factor column is supplied, then the method must be mode.

  • values: A vector of impute values (one per column). NaN indicates to skip the column.
import h2o
h2o.init()

# Import the airlines dataset
air_path = "https://s3.amazonaws.com/h2o-public-test-data/smalldata/airlines/allyears2k.zip"
air = h2o.import_file(path=air_path)
air.dim
[43978, 31]

# Mean impute the DepTime column based on the Origin and Distance columns
DeptTime_impute = air.impute("DepTime", method = "mean", by = ["Origin", "Distance"])
DeptTime_impute
Origin      Distance    mean_DepTime
--------  ----------  --------------
ABE              253         1149.7
ABE              481          812
ABQ              223         1229.33
ABQ              277         1565
ABQ              289         1529
ABQ              321         1267.06
ABQ              328         1301.85
ABQ              332         1655
ABQ              349          813.28
ABQ              487         1536.14

[1497 rows x 3 columns]

# Revert imputations
air = h2o.import_file(path=air_path)

# Mode impute the TailNum column
mode_impute = air.impute("TailNum", method = "mode")
mode_impute
[nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, 3499.0, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan]

# Revert imputations
air = h2o.import_file(path=air_path)

# Mode impute the TailNum column based on the Month and Year columns
mode_impute = air.impute("TailNum", method = "mode", by=["Month", "Year"])
mode_impute
Year    Month    mode_TailNum
------  -------  --------------
  1987       10            3499
  1988        1            3499
  1989        1            3499
  1990        1            3499
  1991        1            3499
  1992        1            3499
  1993        1            3499
  1994        1            3499
  1995        1            3500
  1996        1             672

[22 rows x 3 columns]
Feedback