Skip to main content
Version: v1.6.43-dev4 🚧

Agent tool calling and Driverless AI integration

Overview​

Agent tool calling allows Enterprise h2oGPTe agents to execute functions and interact with external services. The system supports integration with H2O's Driverless AI platform and other external tools to automate complex workflows through natural language requests.

Get started with Agent tool calling​

To get started with Agent tool calling:

  1. Start a new chat session in Enterprise h2oGPTe.
  2. Ensure you customize your chat configuration to use Agents
  3. Use natural language to request tasks that require external tools.

Set up Agent tool calling​

Prerequisites check​

To validate your setup before starting, consider the following checks:

  1. Verify Enterprise h2oGPTe is running: Check that your instance is accessible and Agent tools are enabled
  2. Validate Driverless AI (DAI) connection: Confirm with your administrator that Driverless AI integration is configured
  3. Check permissions: Verify your user account has permissions to access Agents and use tools

Available tools and tool Configuration​

Enterprise h2oGPTe includes several built-in tools. To see the full list of available tools and its configurations, see the Agent tool Configuration page.

Example 1: Basic Agent tool calling workflow​

This example demonstrates how to use tools for web search and data analysis.

Step 1: Start Enterprise h2oGPTe with tool support​

Ensure Enterprise h2oGPTe is running with Agent tool calling enabled.

Step 2: Use natural language requests​

Ask Enterprise h2oGPTe to perform tasks using natural language:

Web search and analysis:

Search for the latest machine learning trends in 2024 and summarize the key findings

Document processing:

Convert this PDF document to text and analyze the key themes: https://example.com/report.pdf

Audio transcription:

Transcribe this meeting recording and extract action items: meeting_audio.mp3

Step 3: Tool auto-selection​

Enterprise h2oGPTe Agents will automatically determine which tools to use and when to use them. Common use cases include:

Example 2: Loan amount prediction with DAI regression experiment​

This example demonstrates creating a regression model to predict loan amounts using Driverless AI through Agent tool calling. The dataset contains loan application data including applicant demographics, income, credit history, and loan details.

Step 1: Verify Driverless AI connection​

note

When using Driverless AI Agent tool calling, ensure you enable the Data Science Agent:

Enable Data Science Agent

Agent turn configuration

Complex DAI experiments or other complex Agent tool calling workflows may require multiple agent turns to complete. If your experiment stops prematurely with a message about reaching the turn limit, increase the agent turns setting in your chat configuration. The default max turns vary by agent accuracy level: Quick (5 turns), Basic (10 turns), Standard (20 turns), and Maximum (80 turns). For complex DAI experiments, consider using the Standard or Maximum accuracy setting, or manually increasing max turns to ensure the agent can complete the full workflow including data loading, experiment creation, monitoring, and results analysis. See Max Agent Turns for details.

Ensure your Driverless AI instance is accessible through Enterprise h2oGPTe:

Check my Driverless AI connection status and list available datasets

Step 2: Create a regression experiment​

In this example, the Loan Prediction dataset contains 614 loan applications with 13 features:

  • Applicant details: Gender, Married, Dependents, Education, Self_Employed
  • Financial information: ApplicantIncome, CoapplicantIncome, LoanAmount (target), Loan_Amount_Term
  • Other factors: Credit_History, Property_Area, Loan_Status

Request loan amount prediction experiment with natural language:

I need to build a model to predict loan amounts for applicants. Please use the loan prediction training dataset from https://s3.amazonaws.com/data.h2o.ai/DAI-Tutorials/loan_prediction/loan_prediction_train.csv and create a regression experiment that predicts the LoanAmount column. Name it 'Loan Amount Prediction' and prioritize interpretability so we can explain the results to stakeholders. Use RMSE to evaluate performance.

Alternative conversational prompts:

Build a regression model to predict how much loan amount applicants should receive based on their income, credit history, and other factors from this dataset: https://s3.amazonaws.com/data.h2o.ai/DAI-Tutorials/loan_prediction/loan_prediction_train.csv. I want a highly interpretable model.
Help me create a loan amount prediction model using the loan_prediction_train.csv dataset. The target is LoanAmount and I need the model to be easy to explain for regulatory purposes.

Understanding experiment settings:

  • Accuracy: Controls model accuracy and feature evolution complexity. Higher values lead to more confident model performance through extensive feature engineering and model tuning.
  • Time: Specifies experiment duration. Higher values allow more iterations and model exploration. Early stopping activates for Time > 1.
  • Interpretability: Controls model complexity. Higher values prioritize explainability by restricting complex features and models. Lower values enable more sophisticated feature engineering for potentially better accuracy.
  • Scorer: Metric for evaluating the model. RMSE is commonly used for regression to measure prediction error.

Expected detailed output​

The agent will provide a comprehensive analysis that includes:

  • Dataset analysis of 614 loan applications with 13 features
  • Model comparison showing Linear Regression as best performer (RMSE: 48.72, R²: 0.36)
  • Feature importance highlighting Total Income, Applicant Income, and Credit History as top predictors
  • Recommendation to use Linear Regression for optimal balance of performance and interpretability
  • Deliverables including visualizations, feature importance plots, model diagnostics, and prediction artifacts

Step 3: Analyze experiment results​

After the experiment completes, you can request additional analysis:

Show me the variable importance for the loan amount prediction experiment
Create a model diagnostic using the test dataset and show me the actual vs predicted plot
Generate an MLI report to interpret the model and analyze feature contributions

Some of the available analysis options include:

  • Variable importance: Shows which features most impact predictions (original and transformed features)
  • Model diagnostics: Provides comprehensive scoring across all regression metrics with visualization plots
  • MLI interpretation: Includes Disparate Impact Analysis, Sensitivity Analysis, Shapley values, and surrogate decision trees
  • Predictions: Download training/test predictions with confidence intervals
  • Artifacts: Access MOJO scoring pipeline, AutoDoc reports, and experiment summaries

Step 4: Deploy and use the model​

Download the MOJO scoring pipeline for my loan amount prediction model

Access Driverless AI instances​

Web interface access​

You can directly access your Driverless AI instance through the web interface using these methods:

  1. Navigate to your H2O AI Cloud environment:

    https://your-cloud-instance.h2o.ai

  2. Find your engine in the dashboard:

    • Look for your engine name (e.g., "your-engine-name")
    • Click on the engine to access the Driverless AI interface

  3. Access experiment directly:

    https://your-engine-name.h2o.ai:12345
note

Contact your system administrator for specific connection credentials if needed. Alternatively, you can access Driverless AI through Enterprise h2oGPTe by following Step 1: Verify Driverless AI connection.

Use experiment references​

Enterprise h2oGPTe remembers your experiments within the conversation:

Use my Loan Amount Prediction experiment to make predictions on this new dataset: new_loan_applications.csv
Compare the Ridge Regression and Driverless AI models from my loan prediction experiment and recommend which one to deploy

Advanced Agent tool calling features​

Multi-tool workflows​

Enterprise h2oGPTe can combine multiple tools for complex tasks:

Search for recent research on loan default prediction best practices, summarize the key features and modeling approaches, then enhance my Loan Amount Prediction experiment with those insights
Analyze this loan portfolio performance report, extract key metrics about approval rates and defaults by demographic segments, create visualizations, and build a model to predict optimal loan amounts

This request does the following:

  1. Use web search to find relevant research
  2. Analyze and summarize findings
  3. Apply insights to improve or create a Driverless AI experiment

Document analysis workflows​

Download and analyze this research paper, then create visualizations of the key concepts using Mermaid diagrams: https://arxiv.org/pdf/example.pdf

Tool usage control​

You can guide which tools to use or avoid:

Analyze this dataset but only use statistical methods, don't create any machine learning models
Create a comprehensive analysis using all available tools relevant except web search

Experiment management best practices​

1. Use descriptive names​

# Good: Descriptive experiment names
experiment_name="Q4_Customer_Churn_XGBoost_v2"

# Avoid: Generic names
experiment_name="Experiment1"

2. Reference previous work​

Enterprise h2oGPTe remembers experiments within your conversation:

Use the same settings from my Customer Churn experiment but apply them to this new dataset

3. Download results​

Request specific outputs from your experiments:

Download the model artifacts and scoring pipeline from my latest experiment
Generate a comprehensive model report with all visualizations and save it as a PDF

This guide covers Agent tool calling and Driverless AI integration in Enterprise h2oGPTe. For advanced use cases and custom tool development, refer to the Agent Tool Configuration developer documentation.

Feedback