Version: v1.6.40-dev2 🚧

Agent Tool Configuration

Administrator Guide

This page provides integration details, configuration parameters, and requirements for all built-in agent tools available in Enterprise h2oGPTe. It is intended for administrators (DevOps, IT, MLOps, or platform engineers) who deploy, manage, or secure Enterprise h2oGPTe in production or enterprise environments.

Here you'll find:

Tool-by-tool configuration variables and deployment requirements
Default and optional settings, including cloud services and external API keys
Hardware and dependency notes (e.g., GPU requirements, tokens/credentials needed)
Best practices for enabling or restricting tools based on your organization's compliance, security, or data-access policies

Before onboarding end users or exposing agent capabilities, review this guide to ensure your environment is correctly configured and all necessary prerequisites—such as external integrations, cloud credentials, or on-prem services—are in place.

Tool configuration requirements

Enterprise h2oGPTe includes multiple agent tools, each with specific requirements. Basic agent functionality is enabled by default, but many tools require additional configuration.

Check the requirements for each tool you intend to use
Configure the necessary environment variables in your Helm deployment
Ensure any required external services or credentials are set up properly
Verify that hardware requirements (such as GPU for image generation) are met
Replace all <your-*-key> placeholders with actual keys/tokens

note

All configuration examples below are Helm chart configurations for administrators. Users will select and use tools through the h2oGPTe interface after administrators have properly configured them.

This basic agent configuration in your Helm chart enables several tools by default:

h2ogpt:
  config:
    externalLLM:
      secret:
        # Allow external API access
        ALLOW_API: "1"

        # Path to store OpenAI API logs
        H2OGPT_OPENAI_LOG_PATH: "/workspace/save/h2ogpt_openai/"

        # Path to cache Hugging Face transformer models
        TRANSFORMERS_CACHE: "/workspace/.cache"

Tools

Data analysis capabilities

H2O Driverless AI Data Science: Build automated machine learning models and analyze data to generate actionable insights. This tool connects to H2O Driverless AI to run experiments, create predictions, and perform data analysis.

Configure the tool:

For DAI Cloud:
For DAI on-premises:

h2ogpt:
  config:
    externalLLM:
      secret:
        ENABLE_DAI: "1"
        DAI_INSTANCE_TYPE: cloud
        DAI_CLIENT_ID: hac-platform-public
        DAI_ENGINE: <your-dai-engine>
        DAI_ENVIRONMENT: <your-dai-environment-url>
        DAI_TOKEN: <your-dai-token>
        DAI_TOKEN_ENDPOINT_URL: <your-token-endpoint-url>

h2ogpt:
  config:
    externalLLM:
      secret:
        ENABLE_DAI: "1"
        DAI_INSTANCE_TYPE: onprem
        DAI_ADDRESS: <your-dai-address>
        DAI_USERNAME: <your-username>
        DAI_PASSWORD: <your-password>

Configuration reference

Parameter	Type	Deployment Type (Cloud/MC and On-Premises)	Description
`ENABLE_DAI`	String	Both	Set to `"1"` to enable Driverless AI integration
`DAI_INSTANCE_TYPE`	String	Both	Specify `"cloud"` or `"onprem"`
`DAI_CLIENT_ID`	String	Cloud/MC	Client ID for cloud instances. Default: `hac-platform-public`
`DAI_ENGINE`	String	Cloud/MC	Engine identifier for cloud deployments
`DAI_ENVIRONMENT`	String	Cloud/MC	Cloud environment URL
`DAI_TOKEN`	String	Cloud/MC	Authentication token for cloud access
`DAI_TOKEN_ENDPOINT_URL`	String	Cloud/MC	Token endpoint URL for cloud authentication
`DAI_ADDRESS`	String	On-Premises	Server address for on-premises installations
`DAI_USERNAME`	String	On-Premises	Username for on-premises authentication
`DAI_PASSWORD`	String	On-Premises	Password for on-premises authentication
`DAI_MAX_RUNTIME_MINUTES` (optional)	Integer	Both	Maximum runtime limit for experiments in minutes (optional)

How end users interact with the tool

Once configured, end users can interact with the H2O Driverless AI Data Science tool through natural language prompts. They don't need to write code directly. For example:

"Use Driverless AI to create a classification experiment with the Titanic dataset, targeting the 'Survived' column with accuracy setting 2"
"Run a regression experiment on my sales data with high interpretability"
"Create a DAI experiment with time setting 3 and accuracy setting 5"

The agent will automatically invoke the underlying tool function:

# This code is executed automatically by the agent - not by end users
from api_server.agent_tools.driverless_ai_data_science import driverless_ai_data_science

driverless_ai_data_science(
    data="titanic.csv",
    dataset_name="titanic_test",
    target_column="Survived",
    task="classification",
    experiment_name="titanic_test_experiment",
    accuracy=2,
    time=2,
    interpretability=7
)

Secure your credentials

To securely access Driverless AI credentials:

Navigate to Agents > Authentication in the Enterprise h2oGPTe UI.
Create secure keys for DAI access.
Assign the keys to the H2O Driverless AI Data Science tool.

Access the credentials in your code:

# Access credentials securely via environment variables
enable_dai = os.getenv("ENABLE_DAI")
dai_max_runtime = os.getenv("DAI_MAX_RUNTIME_MINUTES")

Before you begin

Ensure your system has sufficient computational resources for experiments
For cloud instances: Verify authentication tokens and network connectivity
For on-premises: Confirm the Driverless AI (DAI) server is accessible from h2oGPTe
Use DAI_MAX_RUNTIME_MINUTES to prevent long-running experiments

Python Coding: Execute Python code snippets for custom data computations, machine learning tasks, or to automate data preprocessing.

Requirements:
- Enabled with basic agent configuration in Helm chart:
```
h2ogpt:
  config:
    externalLLM:
      secret:
        ALLOW_API: "1"
```
Advanced Reasoning: Perform context-aware reasoning to enhance decision-making processes.

Requirements:
- Requires models that support reasoning capabilities.
- Configure in Helm chart:
```
agent:
  enabled: true
  overrideConfig:
    visible_reasoning_models:
      - meta-llama/Llama-3.1-8B-Instruct
```
tip
To enhance reasoning depth and accuracy, you can set the AGENT_ACCURACY parameter to Max in your agent configuration. This setting may increase resource usage but improves the quality of context-aware reasoning.

Information retrieval and research

Scholar Papers Search: Locate scholarly papers, research articles, and academic content.

Requirements:
- Requires the Internet Access tool to be enabled.
Google search: Access Google's search capabilities to gather timely and accurate information.

Requirements:
- Requires the Internet Access tool to be enabled.
- Optional:
  - Configure the Google API key in Helm chart:
    h2ogpt: config: externalLLM: secret: GOOGLE_API_KEY: <your-google-api-key>
Bing search: Access Bing's search capabilities to gather timely and accurate information.

Requirements:
- Requires the Internet Access tool to be enabled.
- Optional:
  - Configure the Bing API key in Helm chart:
    h2ogpt: config: externalLLM: secret: BING_API_KEY: <your-bing-api-key>
Internet Access: Utilize general internet access for a wide range of tasks, from data retrieval to live updates.

Requirements:
- Basic agent configuration in Helm chart:
```
h2ogpt:
  config:
    externalLLM:
      secret:
        ALLOW_API: "1"
```
Wolfram|Alpha Math Science Search: Solve scientific and mathematical problems using the computational knowledge engine for complex computations.

Requirements:
- Requires the Internet Access tool to be enabled.
- Optional:
  - Configure the Wolfram|Alpha API key in Helm chart:
    h2ogpt: config: externalLLM: secret: WOLFRAM_ALPHA_APPID: <your-wolfram-alpha-appid>
Wikipedia Articles Search: Retrieve information from Wikipedia for learning or to support data analysis and research.

Requirements:
- Requires the Internet Access tool to be enabled.
Wayback Machine Search: Access archived versions of websites for research, historical context, or to reference past web content.

Requirements:
- Requires the Internet Access tool to be enabled.
Web Image Search: Find relevant images from the web to support visual analysis, presentations, or content creation.

Requirements:
- Requires the Internet Access tool to be enabled.

Browser Navigation: Automate web browsing tasks, including form filling, authentication, and complex web interactions across multiple platforms and services.

Requirements:

Requires the Internet Access tool to be enabled.

Basic agent configuration in Helm chart:

h2ogpt:
  config:
    externalLLM:
      secret:
        ALLOW_API: "1"

Optional API keys and credentials to configure in Helm chart for enhanced functionality:

h2ogpt:
  config:
    externalLLM:
      secret:
        # Search Engine APIs
        BING_API_KEY: <your-bing-api-key>
        SERPAPI_API_KEY: <your-serpapi-key>
        GOOGLE_API_KEY: <your-google-api-key>
        
        # AI Service APIs
        STT_OPENAI_API_KEY: <your-openai-stt-key>
        STT_OPENAI_BASE_URL: <your-openai-stt-base-url>
        STT_OPENAI_MODEL: <your-stt-model>
        ANTHROPIC_API_KEY: <your-anthropic-key>
        ANTHROPIC_BASE_URL: <your-anthropic-base-url>
        
        # Platform Credentials
        GOOGLE_USERNAME: <your-google-username>
        GOOGLE_PASSWORD: <your-google-password>
        GOOGLE_BIRTHDAY: <your-google-birthday>
        GOOGLE_HOME_ADDRESS: <your-google-home-address>
        GOOGLE_PHONE_NUMBER: <your-google-phone-number>
        
        # Data Platform APIs
        HUGGING_FACE_HUB_TOKEN: <your-huggingface-token>
        KAGGLE_API_KEY: <your-kaggle-api-key>
        KAGGLE_USERNAME: <your-kaggle-username>
        
        # RapidAPI Credentials
        RAPIDAPI_EMAIL: <your-rapidapi-email>
        RAPIDAPI_PASSWORD: <your-rapidapi-password>

note

Store sensitive credentials securely and ensure proper access controls are in place. Consider using environment-specific configurations for different deployment environments.

Ask Question About Documents: Extract or query information from uploaded documents for detailed insights.

Requirements:
- Basic agent configuration in Helm chart:
```
h2ogpt:
  config:
    externalLLM:
      secret:
        ALLOW_API: "1"
```

Content generation and transformation

Image Generation: Create custom images based on textual descriptions. Integrate image generation capabilities from different providers, including local models, OpenAI, Azure OpenAI, and Black Forest Labs.

Requirements:

Requires GPU in agent pod.
Shell Scripting tool must be enabled as a prerequisite.

Configuration in Helm chart:

h2ogpt:
  config:
    externalLLM:
      enabled: true
      secret:
        ALLOW_API: "1"
        
        # Black Forest Labs Configuration
        BFL_API_KEY: <your-black-forest-labs-api-key>
        
        # OpenAI Configuration
        IMAGEGEN_OPENAI_BASE_URL: https://api.openai.com/v1
        IMAGEGEN_OPENAI_API_KEY: <your-openai-api-key>
        
        # Azure OpenAI Configuration
        AZURE_OPENAI_BASE_URL: openai.azure.com
        AZURE_OPENAI_API_KEY: <your-azure-openai-api-key>

agent:
  enabled: true
  additionalConfig:
    function_api_key: <your-function-api-key>
    enforce_h2ogpt_api_key: false
    enforce_h2ogpt_ui_key: false
    enable_image: true
    visible_image_models:
      - "flux.1-schnell"  # Local model for image generation
      
      # Optional: Additional image generation models
      # Uncomment and replace with actual model names when using external providers
      # - "openai-model"  # OpenAI model
      # - "azure-model"   # Azure model

Audio-Video Transcription: Convert spoken content from audio or video files into text.

Requirements:

Basic agent configuration in Helm chart:

h2ogpt:
  config:
    externalLLM:
      secret:
        ALLOW_API: "1"

Optional:

Configure the OpenAI base URL and API key for STT in Helm chart:

h2ogpt:
  config:
    externalLLM:
      secret:
        STT_OPENAI_BASE_URL: <your-stt-openai-base-url>
        STT_OPENAI_API_KEY: <your-stt-openai-api-key>
    additionalConfig:
      asr_model: "distil-whisper/distil-large-v3"  # Model for converting audio/video transcription
      enable_stt: true  # Enable speech-to-text for audio/video transcription

note

Ensure you have selected Agents Only when uploading.

Convert Document to Text: Transform scanned or digital documents into editable text formats.

Requirements:
- Basic agent configuration in Helm chart:
```
h2ogpt:
  config:
    externalLLM:
      secret:
        ALLOW_API: "1"
```
Screenshot Webpage: Capture screenshots of webpages.

Requirements:
- Requires the Internet Access tool to be enabled.
Mermaid Chart-Diagram Renderer: Generate flowcharts and diagrams.

Requirements:
- Enabled with the default configuration.

Automation and scripting

Shell Scripting: Automate system-level tasks to streamline operations.

Requirements:
- Basic agent configuration in Helm chart:
```
h2ogpt:
  config:
    externalLLM:
      secret:
        ALLOW_API: "1"
```

Specialized AI features

RAG Text: Enhance text-based tasks with retrieval-augmented generation.

Requirements:
- Uses default models in ModelLock.
- Basic agent configuration in Helm chart:
```
h2ogpt:
  config:
    externalLLM:
      secret:
        ALLOW_API: "1"
```
RAG Vision: Apply retrieval-augmented generation for vision-based tasks.

Requirements:
- Uses default vision models in ModelLock.

Multimedia processing

Download Web Video: Download web videos.

Requirements:
- Requires the Internet Access tool to be enabled.
- Shell Scripting tool must be enabled as a prerequisite.
Ask Question About Image: Query and analyze visual data.

Requirements:
- Requires a vision model configured in ModelLock.

Software Development Tools

Aider Code Generation: Automatically generate code snippets to accelerate development tasks, particularly for data science and machine learning models.

Requirements:
- Models are fetched from ModelLock.
- No additional configuration required.
GitHub: Integrates with the GitHub platform for version control, code repository browsing, and collaboration.

Requirements:
- Requires configuring GITHUB_TOKEN as a secret in Helm chart:
```
h2ogpt:
  config:
    externalLLM:
      secret:
        GITHUB_TOKEN: <your-github-token>
```
Evaluate Answer: Provides automated evaluation of responses generated by agents.

Requirements:
- Basic agent configuration in Helm chart:
```
h2ogpt:
  config:
    externalLLM:
      secret:
        ALLOW_API: "1"
```
SWEBench Evaluate Patch: Evaluates software patches by running automated tests, quality checks, and performance benchmarks.

Requirements:
- Basic agent configuration in Helm chart:
```
h2ogpt:
  config:
    externalLLM:
      secret:
        ALLOW_API: "1"
```

note

To enable tools that require internet access, such as GitHub, Wikipedia Articles Search, Web Image Search, etc., administrators must ensure that the Internet Access tool is properly configured in their Helm deployment. Once configured by administrators, users can then select and use these tools through the Enterprise h2oGPTe interface.

Feedback

Submit and view feedback for this page
Send feedback about Enterprise h2oGPTe to cloud-feedback@h2o.ai

Tool configuration requirements​

Tools​

Data analysis capabilities​

Configure the tool:​

Configuration reference​

How end users interact with the tool​

Secure your credentials​

Requirements:​

Requirements:​

Information retrieval and research​

Requirements:​

Requirements:​

Requirements:​

Requirements:​

Requirements:​

Requirements:​

Requirements:​

Requirements:​

Requirements:​

Requirements:​

Content generation and transformation​

Requirements:​

Requirements:​

Requirements:​

Requirements:​

Requirements:​

Automation and scripting​

Requirements:​

Specialized AI features​

Requirements:​

Requirements:​

Multimedia processing​

Requirements:​

Requirements:​

Software Development Tools​

Requirements:​

Requirements:​

Requirements:​

Requirements:​

Tool configuration requirements

Tools

Data analysis capabilities

Configure the tool:

Configuration reference

How end users interact with the tool

Secure your credentials

Requirements:

Requirements:

Information retrieval and research

Requirements:

Requirements:

Requirements:

Requirements:

Requirements:

Requirements:

Requirements:

Requirements:

Requirements:

Requirements:

Content generation and transformation

Requirements:

Requirements:

Requirements:

Requirements:

Requirements:

Automation and scripting

Requirements:

Specialized AI features

Requirements:

Requirements:

Multimedia processing

Requirements:

Requirements:

Software Development Tools

Requirements:

Requirements:

Requirements:

Requirements: