Install on Ubuntu

This section describes how to install the Driverless AI Docker image on Ubuntu. The installation steps vary depending on whether your system has GPUs or if it is CPU only.

Environment

Operating System	GPUs?	Min Mem
Ubuntu with GPUs	Yes	64 GB
Ubuntu with CPUs	No	64 GB

Install on Ubuntu with GPUs

Important: The Chainguard image requires Docker 19.03+ with the --gpus all option. Legacy nvidia-docker setups are no longer supported.

Note: Driverless AI is supported on Ubuntu 16.04 or later.

Open a Terminal and ssh to the machine that will run Driverless AI. Once you are logged in, perform the following steps.

Retrieve the Driverless AI Docker image from https://www.h2o.ai/download/. (Note that the contents of this Docker image include a CentOS kernel and CentOS packages.)
Install and run Docker on Ubuntu (if not already installed):

# Install and run Docker on Ubuntu
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo apt-key fingerprint 0EBFCD88 sudo add-apt-repository \
 "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-get update
sudo apt-get install docker-ce
sudo systemctl start docker

Install the NVIDIA Container Toolkit (if not already installed). This allows Docker to access NVIDIA GPUs. For more details, see the NVIDIA Container Toolkit install guide.

To use NVIDIA inside the container, follow these steps on Ubuntu:

# Configure the production repository:
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
&& curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit

Verify that the NVIDIA driver is up and running. If the driver is not up and running, log on to http://www.nvidia.com/Download/index.aspx?lang=en-us to get the latest NVIDIA Tesla V/P/K series driver:

nvidia-smi

Set up a directory for the version of Driverless AI on the host machine:

# Set up directory with the version name
mkdir dai-2.0.0

Change directories to the new folder, then load the Driverless AI Docker image inside the new directory:

# cd into the new directory
cd dai-2.0.0

# Load the Driverless AI docker image
docker load < dai-docker-ubi8-x86_64-2.0.0.tar.gz

Enable persistence of the GPU. Note that this needs to be run once every reboot. Refer to the following for more information: http://docs.nvidia.com/deploy/driver-persistence/index.html.

sudo nvidia-smi -pm 1

Set up the data, log, and license directories on the host machine:

# Set up the data, log, license, and tmp directories on the host machine (within the new directory)
mkdir data
mkdir log
mkdir license
mkdir tmp

At this point, you can copy data into the data directory on the host machine. The data will be visible inside the Docker container.
Run docker images to find the image tag.
Start the Driverless AI Docker image and replace TAG below with the image tag. Use docker run --gpus all for Docker 19.03+ or podman run --hooks-dir for Podman. Note that from version 1.10 DAI docker image runs with internal tini that is equivalent to using --init from docker. If both are enabled in the launch command, tini will print a (harmless) warning message.

Using --shm-size=2g --cap-add=SYS_NICE --ulimit nofile=131071:131071 --ulimit nproc=16384:16384 in the launch command is recommended. If you plan to build image auto model extensively, use --shm-size=4g instead.

Note: Use the docker version command to check which version of Docker you are using.

Note: Podman requires the --hooks-dir=/usr/share/containers/oci/hooks.d option to access GPUs.

# Start the Driverless AI Docker image
docker run --gpus all \
   --pid=host \
   --rm \
   --shm-size=2g --cap-add=SYS_NICE --ulimit nofile=131071:131071 --ulimit nproc=16384:16384 \
   -u `id -u`:`id -g` \
   -p 12345:12345 \
   -v `pwd`/data:/data \
   -v `pwd`/log:/log \
   -v `pwd`/license:/license \
   -v `pwd`/tmp:/tmp \
   h2oai/dai-ubi8-x86_64:|tag:

# Start the Driverless AI Podman image
podman run --hooks-dir=/usr/share/containers/oci/hooks.d --gpus all \
   --pid=host \
   --rm \
   --shm-size=2g --cap-add=SYS_NICE --ulimit nofile=131071:131071 --ulimit nproc=16384:16384 \
   -u `id -u`:`id -g` \
   -p 12345:12345 \
   -v `pwd`/data:/data \
   -v `pwd`/log:/log \
   -v `pwd`/license:/license \
   -v `pwd`/tmp:/tmp \
   h2oai/dai-ubi8-x86_64:2.0.0-cuda11.8.0.xx

Driverless AI will begin running:

--------------------------------
Welcome to H2O.ai's Driverless AI
---------------------------------

- Put data in the volume mounted at /data
- Logs are written to the volume mounted at /log/20180606-044258
- Connect to Driverless AI on port 12345 inside the container
- Connect to Jupyter notebook on port 8888 inside the container

Connect to Driverless AI with your browser:

http://Your-Driverless-AI-Host-Machine:12345

Optionally, verify GPU access from within the container:

# Execute inside the running container
docker exec -it <container_id> nvidia-smi

# For Podman
podman exec -it <container_id> nvidia-smi

Install on Ubuntu with CPUs

Note: Driverless AI is supported on Ubuntu 16.04 or later.

This section describes how to install and start the Driverless AI Docker image on Ubuntu. Note that this uses docker and not nvidia-docker. GPU support will not be available.

Watch the installation video here. Note that some of the images in this video may change between releases, but the installation steps remain the same.

Open a Terminal and ssh to the machine that will run Driverless AI. Once you are logged in, perform the following steps.

Retrieve the Driverless AI Docker image from https://www.h2o.ai/download/.
Install and run Docker on Ubuntu (if not already installed):

# Install and run Docker on Ubuntu
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo apt-key fingerprint 0EBFCD88 sudo add-apt-repository \
 "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-get update
sudo apt-get install docker-ce
sudo systemctl start docker

Set up a directory for the version of Driverless AI on the host machine:

# Set up directory with the version name
mkdir dai-2.0.0

Change directories to the new folder, then load the Driverless AI Docker image inside the new directory:

# cd into the new directory
cd dai-2.0.0

# Load the Driverless AI docker image
docker load < dai-docker-ubi8-x86_64-2.0.0.tar.gz

Set up the data, log, license, and tmp directories on the host machine (within the new directory):

# Set up the data, log, license, and tmp directories
mkdir data
mkdir log
mkdir license
mkdir tmp

At this point, you can copy data into the data directory on the host machine. The data will be visible inside the Docker container.
Run docker images to find the new image tag.
Start the Driverless AI Docker image. Note that GPU support will not be available. Note that from version 1.10 DAI docker image runs with internal tini that is equivalent to using --init from docker, if both are enabled in the launch command, tini will print a (harmless) warning message.

We recommend --shm-size=2g --cap-add=SYS_NICE --ulimit nofile=131071:131071 --ulimit nproc=16384:16384 in docker launch command. But if user plans to build image auto model extensively, then --shm-size=4g is recommended for Driverless AI docker command.

# Start the Driverless AI Docker image
docker run \
    --pid=host \
    --rm \
    --shm-size=2g --cap-add=SYS_NICE --ulimit nofile=131071:131071 --ulimit nproc=16384:16384 \
    -u `id -u`:`id -g` \
    -p 12345:12345 \
    -v `pwd`/data:/data \
    -v `pwd`/log:/log \
    -v `pwd`/license:/license \
    -v `pwd`/tmp:/tmp \
    -v /etc/passwd:/etc/passwd:ro \
    -v /etc/group:/etc/group:ro \
    h2oai/dai-ubi8-x86_64:2.0.0-cuda11.8.0.xx

Driverless AI will begin running:

--------------------------------
Welcome to H2O.ai's Driverless AI
---------------------------------

- Put data in the volume mounted at /data
- Logs are written to the volume mounted at /log/20180606-044258
- Connect to Driverless AI on port 12345 inside the container
- Connect to Jupyter notebook on port 8888 inside the container

Connect to Driverless AI with your browser:

http://Your-Driverless-AI-Host-Machine:12345

Stopping the Docker Image

To stop the Driverless AI Docker image, type Ctrl + C in the Terminal (Mac OS X) or PowerShell (Windows 10) window that is running the Driverless AI Docker image.

Upgrading the Docker Image

This section provides instructions for upgrading Driverless AI versions that were installed in a Docker container. These steps ensure that existing experiments are saved.

WARNING: Experiments, MLIs, and MOJOs reside in the Driverless AI tmp directory and are not automatically upgraded when Driverless AI is upgraded.

Build MLI models before upgrading.

Build MOJO pipelines before upgrading.

Stop Driverless AI and make a backup of your Driverless AI tmp directory before upgrading.

If you did not build MLI on a model before upgrading Driverless AI, then you will not be able to view MLI on that model after upgrading. Before upgrading, be sure to run MLI jobs on models that you want to continue to interpret in future releases. If that MLI job appears in the list of Interpreted Models in your current version, then it will be retained after upgrading.

If you did not build a MOJO pipeline on a model before upgrading Driverless AI, then you will not be able to build a MOJO pipeline on that model after upgrading. Before upgrading, be sure to build MOJO pipelines on all desired models and then back up your Driverless AI tmp directory.

Note: Stop Driverless AI if it is still running.

Requirements

We recommend to have NVIDIA driver >= 471.68 installed (GPU only) in your host environment for a seamless experience on all architectures, including Ampere. Driverless AI ships with CUDA 11.8.0 for GPUs, but the driver must exist in the host environment.

Go to NVIDIA download driver to get the latest NVIDIA Tesla A/T/V/P/K series drivers. For reference on CUDA Toolkit and Minimum Required Driver Versions and CUDA Toolkit and Corresponding Driver Versions, see here .

备注

If you are using K80 GPUs, the minimum required NVIDIA driver version is 450.80.02.

Upgrade Steps

SSH into the IP address of the machine that is running Driverless AI.
Set up a directory for the version of Driverless AI on the host machine:

# Set up directory with the version name
mkdir dai-2.0.0

# cd into the new directory
cd dai-2.0.0

Retrieve the Driverless AI package from https://www.h2o.ai/download/ and add it to the new directory.
Load the Driverless AI Docker image inside the new directory:

# Load the Driverless AI docker image
docker load < dai-docker-ubi8-x86_64-2.0.0.tar.gz

Copy the data, log, license, and tmp directories from the previous Driverless AI directory to the new Driverless AI directory:

# Copy the data, log, license, and tmp directories on the host machine
cp -a dai_rel_1.4.2/data dai-2.0.0/data
cp -a dai_rel_1.4.2/log dai-2.0.0/log
cp -a dai_rel_1.4.2/license dai-2.0.0/license
cp -a dai_rel_1.4.2/tmp dai-2.0.0/tmp

At this point, your experiments from the previous versions will be visible inside the Docker container.

Use docker images to find the new image tag.
Start the Driverless AI Docker image.
Connect to Driverless AI with your browser at http://Your-Driverless-AI-Host-Machine:12345.