IBM Docker Images

To simplify local installation, Driverless AI is provided as a Docker image for the following system combination:

Host OS	Docker Version	Host Architecture	Min Mem
RHEL or CentOS 7.4 or later	Docker CE	ppc64le	64 GB

Notes:

• CUDA 10 or later with NVIDIA drivers >= 410 (GPU only)

• To use LightGBM for GPUs on Power, enable the LightGBM CUDA expert setting. LightGBM OpenCL is not currently supported on Power.

• For RHEL 8, Docker is not included and not supported by Red Hat (although it is still available from other sources). For more information, see https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html-single/building_running_and_managing_containers/index.

For the best performance, including GPU support, use nvidia-docker2. For a lower-performance experience without GPUs, use regular docker (with the same docker image).

These installation steps assume that you have a license key for Driverless AI. For information on how to obtain a license key for Driverless AI, visit https://www.h2o.ai/products/h2o-driverless-ai/. Once obtained, you will be prompted to paste the license key into the Driverless AI UI when you first log in, or you can save it as a .sig file and place it in the license folder that you will create during the installation process.

Install on IBM with GPUs

This section describes how to install and start the Driverless AI Docker image on RHEL for IBM Power LE systems with GPUs. Note that nvidia-docker has limited support for ppc64le machines. More information about nvidia-docker support for ppc64le machines is available here.

Open a Terminal and ssh to the machine that will run Driverless AI. Once you are logged in, perform the following steps.

1. Retrieve the Driverless AI Docker image from https://www.h2o.ai/download/.

2. Add the following to cuda-rhel7.repo in /etc/yum.repos.d/:

[cuda]
name=cuda
baseurl=http://developer.download.nvidia.com/compute/cuda/repos/rhel7/ppc64le
enabled=1
gpgcheck=1
gpgkey=http://developer.download.nvidia.com/compute/cuda/repos/rhel7/ppc64le/7fa2af80.pub

3. Add the following to nvidia-container-runtime.repo in /etc/yum.repos.d/:

[libnvidia-container]
name=libnvidia-container
baseurl=https://nvidia.github.io/libnvidia-container/centos7/$basearch
repo_gpgcheck=1
gpgcheck=0
enabled=1
gpgkey=https://nvidia.github.io/libnvidia-container/gpgkey
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt

[nvidia-container-runtime]
name=nvidia-container-runtime
baseurl=https://nvidia.github.io/nvidia-container-runtime/centos7/$basearch
repo_gpgcheck=1
gpgcheck=0
enabled=1
gpgkey=https://nvidia.github.io/nvidia-container-runtime/gpgkey
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt

4. Install the latest drivers and the latest version of CUDA:

yum -y install nvidia-driver-latest-dkms cuda --nogpgcheck

5. Install Docker on RedHat:

yum -y install docker

6. Install NVIDIA hook. (See https://github.com/NVIDIA/nvidia-docker#rhel-docker for more information.) This automatically switches Docker’s runtime to nvidia-runtime:

yum -y install nvidia-container-runtime-hook

7. Set up a directory for the version of Driverless AI on the host machine, replacing VERSION below with your Driverless AI Docker image version:

# Set up directory with the version name
mkdir dai_rel_VERSION

8. Change directories to the new folder, then load the Driverless AI Docker image inside the new directory. This example shows how to load Driverless AI. Replace VERSION with your image.

# cd into the new directory
cd dai_rel_VERSION

# Load the Driverless AI docker image
docker load < dai-docker-centos7-ppc64le-VERSION.tar.gz

9. Enable persistence of the GPU. Note that this needs to be run once every reboot. Refer to the following for more information: http://docs.nvidia.com/deploy/driver-persistence/index.html.

sudo nvidia-persistenced --persistence-mode

10. Set up the data, log, and license directories on the host machine (within the new directory):

# Set up the data, log, license, and tmp directories on the host machine
mkdir data
mkdir log
mkdir license
mkdir tmp

11. At this point, you can copy data into the data directory on the host machine. The data will be visible inside the Docker container.

12. Run docker images to find the image tag.

13. Start the Driverless AI Docker image with nvidia-docker and replace TAG below with the image tag:

# Start the Driverless AI Docker image
nvidia-docker run \
    --pid=host \
    --init \
    --rm \
    --shm-size=256m \
    -u `id -u`:`id -g` \
    -p 12345:12345 \
    -v `pwd`/data:/data \
    -v `pwd`/log:/log \
    -v `pwd`/license:/license \
    -v `pwd`/tmp:/tmp \
    -v /etc/passwd:/etc/passwd:ro \
    -v /etc/group:/etc/group:ro \
    h2oai/dai-centos7-ppc64le:TAG

Driverless AI will begin running:

--------------------------------
Welcome to H2O.ai's Driverless AI
---------------------------------

- Put data in the volume mounted at /data
- Logs are written to the volume mounted at /log/20180606-044258
- Connect to Driverless AI on port 12345 inside the container
- Connect to Jupyter notebook on port 8888 inside the container

14. Connect to Driverless AI with your browser at http://Your-Driverless-AI-Host-Machine:12345.

Install on IBM with CPUs

This section describes how to install and start the Driverless AI Docker image on RHEL for IBM Power LE systems with CPUs. Note that this uses Docker and not NVIDIA Docker. GPU support will not be available.

Watch the installation video here. Note that some of the images in this video may change between releases, but the installation steps remain the same.

Note: As of this writing, Driverless AI has only been tested on RHEL version 7.4.

Open a Terminal and ssh to the machine that will run Driverless AI. Once you are logged in, perform the following steps.

1. Install and start Docker CE.

sudo yum install -y yum-utils
sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo yum makecache fast
sudo yum -y install docker-ce
sudo systemctl start docker

2. On the machine that is running Docker EE, retrieve the Driverless AI Docker image from https://www.h2o.ai/driverless-ai-download/.

3. Set up a directory for the version of Driverless AI on the host machine, replacing VERSION below with your Driverless AI Docker image version:

# Set up directory with the version name
mkdir dai_rel_VERSION

4. Load the Driverless AI Docker image inside the new directory. The following example shows how to load Driverless AI. Replace VERSION with your image.

# Load the Driverless AI Docker image
docker load < dai-docker-centos7-ppc64le-VERSION.tar.gz

5. Set up the data, log, license, and tmp directories (within the new directory):

# cd into the directory associated with the selected version of Driverless AI
cd dai_rel_VERSION

# Set up the data, log, license, and tmp directories on the host machine
mkdir data
mkdir log
mkdir license
mkdir tmp

6. Copy data into the data directory on the host. The data will be visible inside the Docker container at /<user-home>/data.

7. Run docker images to find the image tag.

8. Start the Driverless AI Docker image and replace TAG below with the image tag. Note that GPU support will not be available.

$ docker run \
    --pid=host \
    --init \
    --rm \
    -u `id -u`:`id -g` \
    -p 12345:12345 \
    -v `pwd`/data:/data \
    -v `pwd`/log:/log \
    -v `pwd`/license:/license \
    -v `pwd`/tmp:/tmp \
    -v /etc/passwd:/etc/passwd:ro \
    -v /etc/group:/etc/group:ro \
    h2oai/dai-centos7-ppc64le:TAG

Driverless AI will begin running:

--------------------------------
Welcome to H2O.ai's Driverless AI
---------------------------------

- Put data in the volume mounted at /data
- Logs are written to the volume mounted at /log/20180606-044258
- Connect to Driverless AI on port 12345 inside the container
- Connect to Jupyter notebook on port 8888 inside the container

9. Connect to Driverless AI with your browser at http://Your-Driverless-AI-Host-Machine:12345.

Stopping the Docker Image

To stop the Driverless AI Docker image, type Ctrl + C in the Terminal (Mac OS X) or PowerShell (Windows 10) window that is running the Driverless AI Docker image.

Upgrading the Docker Image

This section provides instructions for upgrading Driverless AI versions that were installed in a Docker container. These steps ensure that existing experiments are saved.

WARNING: Experiments, MLIs, and MOJOs reside in the Driverless AI tmp directory and are not automatically upgraded when Driverless AI is upgraded.

• Build MLI models before upgrading.

• Build MOJO pipelines before upgrading.

• Stop Driverless AI and make a backup of your Driverless AI tmp directory before upgrading.

If you did not build MLI on a model before upgrading Driverless AI, then you will not be able to view MLI on that model after upgrading. Before upgrading, be sure to run MLI jobs on models that you want to continue to interpret in future releases. If that MLI job appears in the list of Interpreted Models in your current version, then it will be retained after upgrading.

If you did not build a MOJO pipeline on a model before upgrading Driverless AI, then you will not be able to build a MOJO pipeline on that model after upgrading. Before upgrading, be sure to build MOJO pipelines on all desired models and then back up your Driverless AI tmp directory.

Note: Stop Driverless AI if it is still running.

Requirements

As of 1.7.0, CUDA 9 is no longer supported. Your host environment must have CUDA 10.0 or later with NVIDIA drivers >= 410 installed (GPU only). Driverless AI ships with its own CUDA libraries, but the driver must exist in the host environment. Go to https://www.nvidia.com/Download/index.aspx to get the latest NVIDIA Tesla V/P/K series driver.

Upgrade Steps

1. SSH into the IP address of the machine that is running Driverless AI.

2. Set up a directory for the version of Driverless AI on the host machine:

# Set up directory with the version name
mkdir dai_rel_VERSION

# cd into the new directory
cd dai_rel_VERSION

3. Retrieve the Driverless AI package from https://www.h2o.ai/download/ and add it to the new directory.

4. Load the Driverless AI Docker image inside the new directory. This example shows how to load Driverless AI version. If necessary, replace VERSION with your image.

# Load the Driverless AI docker image
docker load < dai-docker-centos7-x86_64-VERSION.tar.gz

5. Copy the data, log, license, and tmp directories from the previous Driverless AI directory to the new Driverless AI directory:

# Copy the data, log, license, and tmp directories on the host machine
cp -a dai_rel_1.4.2/data dai_rel_VERSION/data
cp -a dai_rel_1.4.2/log dai_rel_VERSION/log
cp -a dai_rel_1.4.2/license dai_rel_VERSION/license
cp -a dai_rel_1.4.2/tmp dai_rel_VERSION/tmp

At this point, your experiments from the previous versions will be visible inside the Docker container.

6. Use docker images to find the new image tag.

7. Start the Driverless AI Docker image.

8. Connect to Driverless AI with your browser at http://Your-Driverless-AI-Host-Machine:12345.