Usage & Feedback
The Usage & Feedback page lets you track chat sessions, token costs, usage trends, and user feedback across all collections.
To open the page, click the profile icon at the bottom of the sidebar, then click Usage and Quota.
Usage & Feedback requires the Display chat insights page permission. The Default and User roles have this permission by default. The Guest role doesn't. To configure role permissions, see Roles and permissions.
The page has the following tabs:
- Observability (default)
- Guardrail Insights (administrator only)
- Dashboard
- Token Usage
- Feedback
An Auto-refresh toggle in the top-right corner updates data every 10 seconds when enabled. Click the refresh icon next to it to reload manually.
Observability​
The Observability tab shows live session status, errors, and message counts. This tab opens by default.
Filters​
- Filter: Text search across sessions.
- Status: All Status (default), Error, In Progress, Completed.
- Type: All Types (default), Agent, LLM.
- Time Range: Last 24 Hours, Last 7 Days (default), Last 30 Days, Last 90 Days.
- User (administrator only): Dropdown to filter by a specific user.
Summary cards​
Five cards appear before the session table:
| Card | Description |
|---|---|
| Total Sessions | Number of sessions in the selected time range. |
| Questions / Replies | Count of questions asked and replies received. |
| Total Errors | Number of sessions that ended with an error. |
| Agent Sessions | Number of sessions that used an agent. |
| Document References | Total document references across all sessions. |
Session table​
The table lists each session with the following columns:
- Chat Name: Clickable link that opens the chat session.
- Status: Badge showing Completed (green), In Progress (amber), or Error (red).
- Error Message: Truncated error text. Click to open a dialog with the full message.
- Type: Badge showing AI Assistant (purple), Agent (indigo), RAG (blue), or LLM (gray).
- Messages: Format "Q:1 A:1."
- Model: The LLM used for the session.
- Collection: Clickable link to the collection page.
- User (administrator only): The user who created the session.
- Last Active: Relative time. Hover for the absolute timestamp.
Each row has an Actions menu with the following options:
- Open Chat: Opens the chat session.
- View Details: Opens the session detail view.
- Copy ID: Copies the session ID to the clipboard.
- Open Collection: Opens the linked collection. Appears only when the session has a linked collection.
When no sessions match the current filters, the table displays "No chat sessions found," with the hint "Start chatting to see insights here."
Session detail view​
Click View Details on any session to open the detail view. The header shows the session name, a Back to list button, and an Open Chat link.
Four summary cards appear at the top:
| Card | Description |
|---|---|
| Messages | Total number of messages in the session. |
| References | Number of document references cited. |
| Model | The LLM used for the session. |
| Feedback | Count of thumbs-up and thumbs-down votes. |
Each message in the session appears as a card with:
- Badges: Question (blue) or Reply (green), plus Agent (indigo), Error (red), and Votes when applicable.
- Content: The message text. Errors appear in a separate section below.
- Config sections (question messages only): Collapsible sections for LLM, System Prompt, RAG Config, LLM Args, Self Reflection, Pre-prompt Query, and Prompt Query. Each section appears only when data exists.
- Metadata: Message type badges and reference count.
Guardrail Insights​
The Guardrail Insights tab gives administrators visibility into guardrail violations across the platform.
Guardrail Insights requires administrator access. To configure role permissions, see Roles and permissions.
Summary cards​
Four summary cards appear at the top:
| Card | Description |
|---|---|
| Total Violations | Total number of violations in the selected time period. |
| Past 24 Hours | Violations triggered in the last 24 hours. |
| Past 7 Days | Violations triggered in the last 7 days. |
| Violation Rate | Percentage of all messages that triggered a guardrail violation in the selected time period. |
Filters​
- Time Range: Last 24 Hours, Last 7 Days (default), Last 30 Days, Last 90 Days.
- Guardrail Stage: Multi-select dropdown. Defaults to All Guardrail Types.
- Affected Collection: Multi-select searchable dropdown. Defaults to All Collections.
- Violating User: Multi-select dropdown. Defaults to all users.
Charts​
Two charts appear side by side when data is available:
- Daily Violation Trend: Area chart showing the number of violations per day over the selected time range.
- By Guardrail Stage: Bar chart showing violation counts grouped by guardrail stage.
Violations list​
A paginated table lists individual violations with the following columns:
- Guardrail Stage: The stage that triggered the violation.
- Affected Collection: The collection linked to the chat, or "No associated collection."
- Violating User: The username and email of the user who triggered the violation.
- Violation Message: A truncated description of the violation.
- Violation Time: Relative time. Hover for the absolute timestamp.
When no violations match the selected filters, the table displays "No violations found in the selected time period." To configure guardrails, see Global Guardrails.
Dashboard​
The Dashboard tab shows combined cost and usage analytics. Administrators see extra filter controls and the Unique Users card.
Summary cards​
Five cards appear at the top:
| Card | Description | Visibility |
|---|---|---|
| Total Cost | Combined LLM cost for the selected period. | All users |
| Total Calls | Number of LLM API calls made. | All users |
| Total Tokens | Sum of input and output tokens consumed. | All users |
| Unique Users | Number of distinct users with activity. | Administrator only |
| Unique Models | Number of distinct models used. | All users |
Filter bar​
- Group by (administrator only): Model (default), User, or Group. The Group option appears only when user groups exist.
- Time Range: Last 24 Hours, Last 7 Days (default), Last 30 Days, Last 90 Days.
- All Models (administrator only): Multi-select dropdown to filter by specific models.
- All Users (administrator only): Multi-select dropdown to filter by specific users.
Daily cost chart​
A stacked bar chart displays daily costs. The chart title changes based on the Group by selection:
- Daily Cost by Model
- Daily Cost by User
- Daily Cost by Group
Model breakdown table​
A table below the chart breaks down per-model usage with the following columns: Model, Calls, Input Tokens, Output Tokens, Cost.
When no data exists for the selected time range, the chart area displays "No cost data available."
Token Usage​
The Token Usage tab tracks token consumption and cost per session.
Filters​
- Filter: Text search across sessions.
- Time Range: Last 24 Hours, Last 7 Days, Last 30 Days, Last 90 Days, All time (default).
- User (administrator only): Dropdown to filter by a specific user.
Summary cards​
Three cards appear before the table:
| Card | Description |
|---|---|
| Total Cost | Combined LLM cost across all sessions (for example, "$5.0325"). |
| Total Tokens | Sum of input and output tokens consumed (for example, "1.3M"). |
| My Quota | Cost used against your spending limit, shown with a progress bar. |
Usage table​
The table lists each session with the following columns:
- Chat Name: Clickable link that opens the chat session.
- Model
- Input Tokens
- Output Tokens
- Cost
- Last Active
Each row has an Actions menu with these options: View Details, Open Chat, Copy ID.
When no usage data exists, the table displays "No usage data found," with the hint "Cost and token usage appear here as sessions run."
Session detail view​
Click View Details on any session to open a cost-focused detail view. The header shows the session name and a Back to list button.
The detail view shows four summary cards: Model, Total Cost, Input Tokens, Output Tokens.
Session cost breakdown​
This section shows a per-model table with the following columns: Model, Input Tokens, Output Tokens, Cost, Time Taken, Tokens/sec.
Agent cost breakdown​
For agent sessions, a separate section shows agent-specific cost data:
- Agent Turns: Total number of turns the agent executed.
- Agent Query Analysis: The agent's interpretation of the query.
- A per-turn table with the following columns: Turn #, Turn Title, Model, Input Tokens, Output Tokens, Time Taken.
Manage user limits (administrator only)​
Administrators see a Manage User Limits button in the filter bar. Clicking it opens the User Limits dialog, which lists all users alongside their current spending status.
Each row shows:
- Daily (24h): The user's daily spending cap.
- Total Limit: The user's cumulative spending cap.
- Lifetime Spend: The user's total cost to date.
- Limit Status: Within Limit (green), Approaching (yellow), or Blocked (red).
From this dialog, administrators can:
- Edit daily or total limits per user. If a per-user override exists, a Reset to Global Default option is available.
- Reset total usage: Resets the user's cumulative spend to zero.
- Export Users CSV: Downloads a CSV of all users and their limit data.
Feedback​
The Feedback tab displays thumbs-up and thumbs-down votes that users have left on chat responses.
This tab shows feedback across all collections on the platform. To learn how to leave feedback on a response, see Feedback.
Filters​
- Collection: Searchable dropdown. Options include All collections (default), No collection, and all available collections.
- Date Range: Dropdown to filter by time range. Default: All time.
Export feedback​
Click Export CSV in the filter bar to export all feedback matching the current filters.
The export runs as a background job. A notification appears — "Feedback export started. Check the notification tray for progress." — and the notification tray opens automatically. The CSV file downloads automatically when the job completes.
The exported file is named feedback_export_YYYY-MM-DD.csv and contains the following columns:
| Column | Description |
|---|---|
| Date | Timestamp of the response. |
| Collection | Collection the chat belonged to, if any. |
| Chat ID | Unique identifier of the chat session. |
| Message ID | Unique identifier of the rated message. |
| Prompt | The question that was asked. |
| Response | The model's reply. |
| Feedback | Vote value: 1 for thumbs up, -1 for thumbs down. |
| Expected Answer | The ideal response entered by the user, if provided. |
| Comment | The user's comment, if provided. |
| Voted By | Username of the person who left the vote. |
Feedback table​
The table lists each feedback entry with the following columns:
- Prompt: Clickable. Opens the feedback detail pane.
- Collection: Clickable link to the collection page.
- User: The user who left the feedback.
- Vote: Thumbs up or thumbs down icon.
- Date: Relative time.
When no feedback exists, the table displays "You have no messages with feedback," with the hint "Chat messages with thumbs up or thumbs down feedback appear here."
Feedback detail pane​
Click any row in the Prompt column to open a detail pane on the right side of the page. The pane header reads "Chat message" with a timestamp ("Response received [date]").
The pane contains the following collapsible sections:
- Prompt: The full question text.
- Response: The model's reply rendered as text.
- User Feedback: Contains Expected response and User comment text areas.
- Prompt Settings (collapsed by default): Contains Personality (System Prompt), RAG prompt before context, and RAG prompt after context.
- LLM Settings (collapsed by default): Contains the LLM model name and RAG Config (JSON).
Related topics​
- Chats: Start and manage chat sessions.
- Agents: Build and run AI agents.
- Feedback: Leave thumbs-up or thumbs-down votes on chat responses.
- Roles and permissions: Configure access to Usage & Feedback.
- Submit and view feedback for this page
- Send feedback about Enterprise h2oGPTe to cloud-feedback@h2o.ai