Skip to main content

Create a dashboard

In H2O Eval Studio, dashboards compare LLMs based on metrics calculated by the 1 or more evaluators. (For more information on evaluators in H2O Eval Studio, see Evaluators.) You can view This page describes how to create a dashboard in H2O Eval Studio.

  1. In the main navigation, click Dashboards.

  2. Click the New Dashboard button.

  3. Enter a name for the dashboard.

  4. Enter a description of the dashboard.

  5. Select a connection to the model host of the LLM models you want to evaluate. Note that when creating dashboards, there are two types of connections: LLM and RAG. The list of available evaluators and tests depends on the type of connection you select. (For example, operating the RAGAs evaluator on a pure LLM model is not applicable.) For more information on adding a connection, see Add connection.

  6. Select the evaluators you want to use. For more information on the available evaluators, see Evaluators.

  7. Select the tests that you want to use. For more information on tests in H2O Eval Studio, see Tests.

  8. Select the LLM Models you want to use for the evaluation.

  9. (Optional) Set advanced settings for the dashboard. For more information on advanced settings, see model host specific Advanced settings.

  10. Click the Create button.

View a dashboard​

The table on the Dashboards page lists all of the dashboards that you have created. To view a dashboard, click the name of the dashboard you want to view.

When viewing a specific dashboard, you can view a visualization of the dashboard, obtain an HTML report, and download a zip archive with the evaluation results.

View a visualization of a dashboard​

The dashboard page features a visualization of evaluator result metrics as an evaluation eye. Dashboard visualizations can help you understand the evaluation results for a given metric and the LLM models being compared.

Obtain an HTML report of a dashboard​

To view an HTML report of a dashboard, click the Show Report button. This report provides in-depth information about potential problems with the model, evaluation parameters, the evaluated models, and more.

Download a zip archive with evaluation results​

To download a zip archive with evaluation results, click the Download Report button.

Delete a dashboard​

To delete a dashboard from the Dashboards page, select the checkbox next to the dashboard you want to delete, and then click the Delete button.

Feedback