Linux DEBs

For Linux machines that will not use the Docker image or RPM, a deb installation is available for x86_64 Ubuntu 16.04/18.04/20.04/22.04.

The following installation steps assume that you have a valid license key for Driverless AI. For information on how to obtain a license key for Driverless AI, visit https://www.h2o.ai/products/h2o-driverless-ai/. Once obtained, you will be prompted to paste the license key into the Driverless AI UI when you first log in, or you can save it as a .sig file and place it in the license folder that you will create during the installation process.

Note

  • To ensure that AutoDoc pipeline visualizations are generated correctly on native installations, installing fontconfig is recommended.

  • When using systemd, remove the dai-minio, dai-h2o, dai-redis, dai-procsy, and dai-vis-server services. When upgrading, you can use the following commands to deactivate these services:

systemctl stop dai-minio
systemctl disable dai-minio
systemctl stop dai-h2o
systemctl disable dai-h2o
systemctl stop dai-redis
systemctl disable dai-redis
systemctl stop dai-procsy
systemctl disable dai-procsy
systemctl stop dai-vis-server
systemctl disable dai-vis-server

Environment

Operating System

Min Mem

Ubuntu with GPUs

64 GB

Ubuntu with CPUs

64 GB

Requirements

  • Ubuntu 16.04/Ubuntu 18.04/Ubuntu 20.04/Ubuntu 22.04

  • NVIDIA drivers >= 471.68 is recommended (GPU only). Note that if you are using K80 GPUs, the minimum required NVIDIA driver version is 450.80.02

  • OpenCL (Required for full LightGBM support on GPU-powered systems)

  • Driverless AI deb, available from https://www.h2o.ai/download/

Note: CUDA 11.8.0 (for GPUs) and cuDNN (required for TensorFlow support on GPUs) are included in the Driverless AI package.

About the Install

  • The ‘dai’ service user is created locally (in /etc/passwd) if it is not found by ‘getent passwd’. You can override the user by providing the DAI_USER environment variable during rpm or dpkg installation.

  • The ‘dai’ service group is created locally (in /etc/group) if it is not found by ‘getent group’. You can override the group by providing the DAI_GROUP environment variable during rpm or dpkg installation.

  • Configuration files are placed in /etc/dai and owned by the ‘root’ user:

    • /etc/dai/config.toml: Driverless AI config file (See Using the config.toml File section for details).

    • /etc/dai/User.conf: systemd config file specifying the service user.

    • /etc/dai/Group.conf: systemd config file specifying the service group.

    • /etc/dai/EnvironmentFile.conf: systemd config file specifying (optional) environment variable overrides.

  • Software files are placed in /opt/h2oai/dai and owned by the ‘root’ user

  • The following directories are owned by the service user so that they can be updated by the running software:

    • /opt/h2oai/dai/home: The application’s home directory (license key files are stored here).

    • /opt/h2oai/dai/tmp: Experiments and imported data are stored here.

    • /opt/h2oai/dai/log: Log files go here if you are not using systemd (if you are using systemd, then the use the standard journalctl tool).

  • By default, for Docker or DEB/RPM installs, Driverless AI looks for a license key in /opt/h2oai/dai/home/.driverlessai/license.sig. If you are installing Driverless AI programmatically, you can copy a license key file to that location. For TAR SH installs, the equivalent location is <tar.sh dir>/home/.driverlessai, and after the license is imported, it is copied under ~/.driverlessai. If no license key is found, the application guides you through the process of adding one through the UI.

  • systemd unit files are placed in /usr/lib/systemd/system.

  • Symbolic links to the configuration files in /etc/dai files are placed in /etc/systemd/system.

If your environment is running an operational systemd, that is the preferred way to manage Driverless AI. The package installs the following systemd services and a wrapper service:

  • dai: Wrapper service that starts/stops the other services.

  • dai-dai: Main Driverless AI process.

If you don’t have systemd, refer to Linux TAR SH for install instructions.

Starting NVIDIA Persistence Mode (GPU only)

If you have NVIDIA GPUs, you must run the following NVIDIA command. This command needs to be run every reboot. For more information: http://docs.nvidia.com/deploy/driver-persistence/index.html.

sudo nvidia-smi -pm 1

Installing OpenCL

OpenCL is required for full LightGBM support on GPU-powered systems. To install OpenCL, run the following as root:

mkdir -p /etc/OpenCL/vendors && echo "libnvidia-opencl.so.1" > /etc/OpenCL/vendors/nvidia.icd && chmod a+r /etc/OpenCL/vendors/nvidia.icd && chmod a+x /etc/OpenCL/vendors/ && chmod a+x /etc/OpenCL

Note

If OpenCL is not installed, then CUDA LightGBM is automatically used. CUDA LightGBM is only supported on Pascal-powered (and later) systems, and can be enabled manually with the enable_lightgbm_cuda_support config.toml setting.

Installing the Driverless AI Linux DEB

Run the following commands to install the Driverless AI DEB.

# Install Driverless AI.
sudo dpkg -i dai_1.11.1.1_amd64.deb

By default, the Driverless AI processes are owned by the ‘dai’ user and ‘dai’ group. You can optionally specify a different service user and group as shown below. Replace <myuser> and <mygroup> as appropriate.

# Temporarily specify service user and group when installing Driverless AI.
# dpkg saves these for systemd in the /etc/dai/User.conf and /etc/dai/Group.conf files.
sudo DAI_USER=myuser DAI_GROUP=mygroup dpkg -i dai_1.11.1.1_amd64.deb

You may now optionally make changes to /etc/dai/config.toml.

Starting Driverless AI

To start Driverless AI, use the following command:

# Start Driverless AI.
sudo systemctl start dai

Note: If you don’t have systemd, refer to Linux TAR SH for install instructions.

View Driverless AI log files

For RPM or DEB package installations, log files are located in the following directories:

  • Startup and Daemon logs are accessible through journalctl.

  • Experiment logs are located in /opt/h2oai/dai/tmp/<user_logged_in_DAI>/.

  • Redis/server-related standard output and standard error logs are located in /opt/h2oai/dai/tmp/server_log/.

The process IDs (PIDs) of the services are stored in /opt/h2oai/dai/tmp/pids/.

Stopping Driverless AI

If you have systemd (preferred):

# Stop Driverless AI.
sudo systemctl stop dai

# The processes should now be stopped.  Verify.
sudo ps -u dai

If you do not have systemd:

# Stop Driverless AI.
sudo pkill -U dai

# The processes should now be stopped.  Verify.
sudo ps -u dai

Upgrading Driverless AI

WARNINGS:

  • This release deprecates experiments and MLI models from 1.7.0 and earlier.

  • Experiments, MLIs, and MOJOs reside in the Driverless AI tmp directory and are not automatically upgraded when Driverless AI is upgraded. We recommend you take the following steps before upgrading.

    • Build MLI models before upgrading.

    • Build MOJO pipelines before upgrading.

    • Stop Driverless AI and make a backup of your Driverless AI tmp directory before upgrading.

If you did not build MLI on a model before upgrading Driverless AI, then you will not be able to view MLI on that model after upgrading. Before upgrading, be sure to run MLI jobs on models that you want to continue to interpret in future releases. If that MLI job appears in the list of Interpreted Models in your current version, then it will be retained after upgrading.

If you did not build a MOJO pipeline on a model before upgrading Driverless AI, then you will not be able to build a MOJO pipeline on that model after upgrading. Before upgrading, be sure to build MOJO pipelines on all desired models and then back up your Driverless AI tmp directory.

The upgrade process inherits the service user and group from /etc/dai/User.conf and /etc/dai/Group.conf. You do not need to manually specify the DAI_USER or DAI_GROUP environment variables during an upgrade.

Requirements

We recommend to have NVIDIA driver >= 471.68 installed (GPU only) in your host environment for a seamless experience on all architectures, including Ampere. Driverless AI ships with CUDA 11.2.2 for GPUs, but the driver must exist in the host environment.

Go to NVIDIA download driver to get the latest NVIDIA Tesla A/T/V/P/K series drivers. For reference on CUDA Toolkit and Minimum Required Driver Versions and CUDA Toolkit and Corresponding Driver Versions, see here .

Note

If you are using K80 GPUs, the minimum required NVIDIA driver version is 450.80.02.

Upgrade Steps

If you have systemd (preferred):

# Stop Driverless AI.
sudo systemctl stop dai

# Make a backup of /opt/h2oai/dai/tmp directory at this time.

# Upgrade Driverless AI.
sudo dpkg -i dai_1.11.1.1_amd64.deb
sudo systemctl daemon-reload
sudo systemctl start dai

If you do not have systemd:

# Stop Driverless AI.
sudo pkill -U dai

# The processes should now be stopped.  Verify.
sudo ps -u dai

# Make a backup of /opt/h2oai/dai/tmp directory at this time. If you do not, all previous data will be lost.

# Upgrade and restart.
sudo dpkg -i dai_1.11.1.1_amd64.deb
sudo -H -u dai /opt/h2oai/dai/run-dai.sh

Uninstalling Driverless AI

If you have systemd (preferred):

# Stop Driverless AI.
sudo systemctl stop dai

# The processes should now be stopped.  Verify.
sudo ps -u dai

# Uninstall Driverless AI.
sudo dpkg -r dai

# Purge Driverless AI.
sudo dpkg -P dai

If you do not have systemd:

# Stop Driverless AI.
sudo pkill -U dai

# The processes should now be stopped.  Verify.
sudo ps -u dai

# Uninstall Driverless AI.
sudo dpkg -r dai

# Purge Driverless AI.
sudo dpkg -P dai

CAUTION! At this point you can optionally completely remove all remaining files, including the database (this cannot be undone):

sudo rm -rf /opt/h2oai/dai
sudo rm -rf /etc/dai

Note: The UID and GID are not removed during the uninstall process. These can be removed with userdel and usergroup. However, we DO NOT recommend removing the UID and GID if you plan to re-install Driverless AI. If you remove the UID and GID and then reinstall Driverless AI, the UID and GID will likely be re-assigned to a different (unrelated) user/group in the future; this may cause confusion if there are any remaining files on the filesystem referring to the deleted user or group.

Common Problems

Start of Driverless AI fails on the message ``Segmentation fault (core dumped)`` on Ubuntu 18.

This problem is caused by the font NotoColorEmoji.ttf, which cannot be processed by the Python matplotlib library. A workaround is to disable the font by renaming it. (Do not use fontconfig because it is ignored by matplotlib.) The following will print out the command that should be executed.

sudo find / -name "NotoColorEmoji.ttf" 2>/dev/null | xargs -I{} echo sudo mv {} {}.backup