Skip to main content
Version: v1.6.1 🚧

View usage cost metrics for used LLMs in the past 24 hours

Example​

A user can retrieve specific metrics related to a particular large language model (LLM) and its usage costs within the past 24 hours. Specifically, the user can obtain certain usage cost metrics for all LLMs involved in making a specific LLM request within that timeframe.

Overview​

from h2ogpte import H2OGPTE

client = H2OGPTE(
    address="https://h2ogpte.genai.h2o.ai",
    api_key='sk-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX',
)

list = client.get_llm_usage_24h_by_llm()

print(f"""\
Call count: {list[0].call_count}
Input tokens: {list[0].input_tokens}
LLM cost: {list[0].llm_cost}
LLM name: {list[0].llm_name}
Model computed fields: {list[0].model_computed_fields}
Model config: {list[0].model_config}
Model fields: {list[0].model_fields}
Output tokens: {list[0].output_tokens}
""")
Call count: 2
Input tokens: 132
LLM cost: 5.02e-05
LLM name: h2oai/h2o-danube3-4b-chat
Model computed fields: {}
Model config: {}
Model fields: {'llm_name': FieldInfo(annotation=str, required=True), 'llm_cost': FieldInfo(annotation=float, required=True), 'call_count': FieldInfo(annotation=int, required=True), 'input_tokens': FieldInfo(annotation=int, required=True), 'output_tokens': FieldInfo(annotation=int, required=True)}
Output tokens: 148
Feedback