Skip to main content

Create a membership inference attack

Overview​

A membership inference attack enables you to evaluate the security of a Driverless AI model (experiment) on grounds of a membership inference attack.

Instructions​

To create a membership inference attack, consider the following instructions:

  1. Run a model inversion attack.
    Note

    To learn how to run a model inversion attack, see Create a model inversion attack.

  2. Click Run membership inference.
  3. Click Browse.... Or drag and drop the file (membership inference training dataset).
    Note
    • Recall: After executing a model inversion attack, the surrogate model created in such an attack is used to create a second-level surrogate model. This second-level surrogate model is trained to discriminate between rows of data in and not in the first-level surrogate training data. As a result, the second-level surrogate model can indicate whether a training row was in the original model's training data.
    • The format of the membership inference training dataset is as follows: A .csv file containing the following columns:
      • Columns representing the required features the endpoint URL needs to score new data
        • You need to populate these columns with the training information (rows) of the first-level surrogate model
          • To access the data (rows), the first-level surrogate model used for training, consider the following intruction:
            1. Click Download simulated training data (a .csv file its downloaded)
      • A column name in_train that needs to refer to whether H2O Model Security used the data in a row to train the first-level surrogate model
        • If H2O Model Security did not utilize a row as training data for the first-level surrogate model, assigned the column a value of 0, a 1 otherwise (if it was in the trianing dataset)
          • To access the data (rows), the first-level surrogate model used for training, consider the following intruction:
            • Click Download simulated training data (a .csv file its downloaded)
      • A column name Performance that needs to refer to the Adversarial loss the first-level surrogate model attributed to the row (data)
        • To access the Adversarial loss values H2O Model Security assigned to a particular row, consider the following instructions: Click Download full attack CSV (a .csv file its downloaded (the Adversarial loss values are in the Adversarial loss column)
          Note

          To observe a sample membership inference dataset, see Example membership inference training dataset.

  4. Click Upload data.
  5. In the Column sampling rate box, enter a column sampling rate for the second-level surrogate model.
  6. In the Row sampling rate box, enter a row sampling rate for the second-level surrogate model.
  7. In the Number of trees (Inference model)​ box, enter the number of trees for the second-level surrogate model.
  8. In the Maximum depth (Inference model) box, enter the maximum depth for the second-level surrogate model.
  9. In the Learning rate (Inference model) box, enter the learning rate for the second-level surrogate model.
  10. Click Begin inference test on training data.

Note
Feedback