Skip to main content
Version: v1.6.1 🚧

View usage metrics for each LLM used

Overview​

Users can view their usage metrics for each large language model (LLM) they've used, regardless of whether the LLM is currently configured or unconfigured.

Example​

from h2ogpte import H2OGPTE

client = H2OGPTE(
    address="https://h2ogpte.genai.h2o.ai",
    api_key='sk-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX',
)

# The available units for time intervals are:
# - minute / minutes (for example, 5 minutes)
# - hour / hours (for example, 2 hours)
# - day / days (for example, 3 days)
# - week / weeks (for example, 1 week)
# - year / years (for example, 1 year)
list = client.get_llm_usage_by_llm(interval="1 month")

for llm in list: 
    print(
    f"Call count: {llm.call_count}\n"
    f"Input tokens: {llm.input_tokens}\n"
    f"LLM cost: {llm.llm_cost}\n"
    f"LLM name: {llm.llm_name}\n"
    f"Model computed fields: {llm.model_computed_fields}\n"
    f"Model configuration: {llm.model_config}\n"
    f"Model fields: {llm.model_fields}\n"
    f"Output tokens: {llm.output_tokens}"
    )
    print("--" * 50)
Call count: 7
Input tokens: 256952
LLM cost: 0.998826
LLM name: claude-3-5-sonnet-20240620
Model computed fields: {}
Model configuration: {}
Model fields: {'llm_name': FieldInfo(annotation=str, required=True), 'llm_cost': FieldInfo(annotation=float, required=True), 'call_count': FieldInfo(annotation=int, required=True), 'input_tokens': FieldInfo(annotation=int, required=True), 'output_tokens': FieldInfo(annotation=int, required=True)}
Output tokens: 15198
----------------------------------------------------------------------------------------------------
Call count: 9
Input tokens: 76554
LLM cost: 0.02205975
LLM name: claude-3-haiku-20240307
Model computed fields: {}
Model configuration: {}
Model fields: {'llm_name': FieldInfo(annotation=str, required=True), 'llm_cost': FieldInfo(annotation=float, required=True), 'call_count': FieldInfo(annotation=int, required=True), 'input_tokens': FieldInfo(annotation=int, required=True), 'output_tokens': FieldInfo(annotation=int, required=True)}
Output tokens: 2337
....
Feedback