Skip to main content

Create a new evaluation

Overview​

In H2O Eval Studio, you can evaluate a model and generate executive dashboards for model comparisons and advanced insights.
This page explains how to create a new evaluation.

Instructions​

To create a new evaluation:

  1. In the left navigation menu, click Evaluations.
  2. Click New evaluation.
  3. Enter a name for the evaluation.
  4. Enter a description for the evaluation.
  5. From the Model host drop-down menu, select the model you want to evaluate.
  6. Select the tests you want to use.
    For more information, see Tests.
  7. Select the LLM models you want to use for the evaluation.
  8. (Optional) From the Existing collection drop-down menu, select a collection if you want to reuse an H2OGPTe collection instead of creating a new one.
    A new collection is created only if no existing collection has the specified name.
  9. Select the evaluators you want to use.
    For more information, see Evaluators.
  10. (Optional) Set advanced model settings.
    For more details, see model host–specific Advanced settings.
  11. Click Create. Create a new evaluation
Feedback