Install on Google Compute with GPUs

This section describes how to install and start Driverless AI in a Google Compute environment with GPUs.

  1. In your browser, log in to the Google Compute Engine Console at https://console.cloud.google.com/.
  2. In the left navigation panel, select Compute Engine > VM Instances.
../_images/gce_newvm_instance.png
  1. Click Create Instance.
../_images/gce_create_instance.png
  1. Specify the following at a minimum:
  • A unique name for this instance.
  • The desired zone. Note that not all zones and user accounts can select zones with GPU instances. Refer to the following for information on how to add GPUs: https://cloud.google.com/compute/docs/gpus/.
  • A supported OS, for example Ubuntu 16.04. Be sure to also increase the disk size of the OS image to be 64 GB.

Click Create at the bottom of the form when you are done. This creates the new VM instance.

center
  1. Create a Firewall rule for Driverless AI. On the Google Cloud Platform left navigation panel, select VPC network > Firewall rules. Specify the following settings:
  • (Optional) Specify a unique name for this instance.
  • Change the Targets dropdown to All instances in the network.
  • Specify the Source IP range to be 0.0.0.0/0.
  • Under Protocols and Ports, specify the following: tcp:12345.

Click Create at the bottom of the form when you are done.

center
  1. On the VM Instances page, SSH to the new VM Instance by selecting Open in Browser Window from the SSH dropdown.
center
  1. H2O provides a script for you to run in your VM instance. Open an editor in the VM instance (for example, vi). Enter the following text and save the script as install.sh.
apt-get -y update
apt-get -y --no-install-recommends install \
  curl \
  apt-utils \
  python-software-properties \
  software-properties-common

add-apt-repository -y ppa:graphics-drivers/ppa
add-apt-repository -y "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -

apt-get update
apt-get install -y \
  nvidia-384 \
  nvidia-modprobe \
  docker-ce

wget -P /tmp https://github.com/NVIDIA/nvidia-docker/releases/download/v1.0.1/nvidia-docker_1.0.1-1_amd64.deb
dpkg -i /tmp/nvidia-docker*.deb && rm /tmp/nvidia-docker*.deb
  1. Type the following commands to run the install script.
chmod +x install.sh
sudo ./install.sh
  1. In your user folder, create the following directories as your user.
mkdir ~/tmp
mkdir ~/log
mkdir ~/data
mkdir ~/scripts
mkdir ~/license
mkdir ~/demo
mkdir -p ~/jupyter/notebooks
  1. Add your Google Compute user name to the Docker container.
sudo usermod -aG docker <username>
  1. Reboot the system to enable NVIDIA drivers.
sudo reboot
  1. Download the Driverless AI image, replacing X.Y.Z below with your Driverless AI Docker image version (for example, 1.0.16).
wget https://s3-us-west-2.amazonaws.com/h2o-internal-release/docker/driverless-ai-docker-runtime-rel-X.Y.Z.gz
  1. Load the Driverless AI Docker image, replacing X.Y.Z below with your Driverless AI Docker image version (for example, 1.0.16).
sudo docker load < driverless-ai-docker-runtime-rel-X.Y.Z.gz
  1. Start the Driverless AI Docker image with nvidia-docker. Note that you must have write privileges for the folders that are created below. You can replace ‘pwd’ with the path to your user /home/<username> or start with sudo nvidia-docker run.
# Start the Driverless AI Docker image
nvidia-docker run \
    --rm \
    -u `id -u`:`id -g` \
    -p 12345:12345 \
    -p 54321:54321 \
    -p 9090:9090 \
    -v `pwd`/data:/data \
    -v `pwd`/log:/log \
    -v `pwd`/license:/license \
    -v `pwd`/tmp:/tmp \
    opsh2oai/h2oai-runtime

Driverless AI will begin running:

---------------------------------
Welcome to H2O.ai's Driverless AI
---------------------------------
   version: X.Y.Z

- Put data in the volume mounted at /data
- Logs are written to the volume mounted at /log/YYYYMMDD-HHMMSS
- Connect to Driverless AI on port 12345 inside the container
- Connect to Jupyter notebook on port 8888 inside the container
  1. Connect to Driverless AI with your browser:
http://Your-Driverless-AI-Host-Machine:12345

Stopping the GCE Instance

The Google Compute Engine instance will continue to run even when you close the aportal. To stop the instance:

  1. On the VM Instances page, click on the VM instance that you want to stop.
  2. Click Stop at the top of the page.
  3. A confirmation page will display. Click Stop to stop the instance.

Install on Google Compute with CPUs

This section describes how to install and start Driverless AI in a Google Compute environment with CPUs. Note that this uses Docker CE and not NVIDIA Docker. GPU support will not be available.

  1. In your browser, log in to the Google Compute Engine Console at https://console.cloud.google.com/.
  2. In the left navigation panel, select Compute Engine > VM Instances.
../_images/gce_newvm_instance.png
  1. Click Create Instance.
../_images/gce_create_instance.png
  1. Specify the following at a minimum:
  • A unique name for this instance.
  • The desired zone. Note that not all zones and user accounts can select zones with GPU instances. Refer to the following for information on how to add GPUs: https://cloud.google.com/compute/docs/gpus/.
  • A supported OS, for example Ubuntu 16.04. Be sure to also increase the disk size of the OS image to be 64 GB.

Click Create at the bottom of the form when you are done. This creates the new VM instance.

center
  1. Create a Firewall rule for Driverless AI. On the Google Cloud Platform left navigation panel, select VPC network > Firewall rules. Specify the following settings:
  • (Optional) Specify a unique name for this instance.
  • Change the Targets dropdown to All instances in the network.
  • Specify the Source IP range to be 0.0.0.0/0.
  • Under Protocols and Ports, specify the following: tcp:12345.

Click Create at the bottom of the form when you are done.

center
  1. On the VM Instances page, SSH to the new VM Instance by selecting Open in Browser Window from the SSH dropdown.
center
  1. H2O provides a script for you to run in your VM instance. Open an editor in the VM instance (for example, vi). Enter the following text and save the script as install.sh.
apt-get -y update
apt-get -y --no-install-recommends install \
  curl \
  apt-utils \
  python-software-properties \
  software-properties-common

add-apt-repository -y "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -

apt-get update
apt-get install -y docker-ce
  1. Type the following commands to run the install script.
chmod +x install.sh
sudo ./install.sh
  1. In your user folder, create the following directories as your user.
mkdir ~/tmp
mkdir ~/log
mkdir ~/data
mkdir ~/scripts
mkdir ~/license
mkdir ~/demo
mkdir -p ~/jupyter/notebooks
  1. Add your Google Compute user name to the Docker container.
sudo usermod -aG docker <username>
  1. Download the Driverless AI image, replacing X.Y.Z below with your Driverless AI Docker image version (for example, 1.0.16).
wget https://s3-us-west-2.amazonaws.com/h2o-internal-release/docker/driverless-ai-docker-runtime-rel-X.Y.Z.gz
  1. Load the Driverless AI Docker image, replacing X.Y.Z below with your Driverless AI Docker image version (for example, 1.0.16).
sudo docker load < driverless-ai-docker-runtime-rel-X.Y.Z.gz
  1. Start the Driverless AI Docker image with nvidia-docker. Note that you must have write privileges for the folders that are created below. You can replace ‘pwd’ with the path to your user /home/<username> or start with sudo nvidia-docker run.
# Start the Driverless AI Docker image
nvidia-docker run \
    --rm \
    -u `id -u`:`id -g` \
    -p 12345:12345 \
    -p 9090:9090 \
    -v `pwd`/data:/data \
    -v `pwd`/log:/log \
    -v `pwd`/license:/license \
    -v `pwd`/tmp:/tmp \
    opsh2oai/h2oai-runtime

Driverless AI will begin running:

---------------------------------
Welcome to H2O.ai's Driverless AI
---------------------------------
   version: X.Y.Z

- Put data in the volume mounted at /data
- Logs are written to the volume mounted at /log/YYYYMMDD-HHMMSS
- Connect to Driverless AI on port 12345 inside the container
- Connect to Jupyter notebook on port 8888 inside the container
  1. Connect to Driverless AI with your browser:
http://Your-Driverless-AI-Host-Machine:12345

Stopping the GCE Instance

The Google Compute Engine instance will continue to run even when you close the aportal. To stop the instance:

  1. On the VM Instances page, click on the VM instance that you want to stop.
  2. Click Stop at the top of the page.
  3. A confirmation page will display. Click Stop to stop the instance.