Install on AWS

This section describes how to install the Driverless AI AMI from AWS.

Watch the installation video here. Note that some of the images in this video may change between releases, but the installation steps remain the same.

Environment

Provider Instance Type Num GPUs Suitable for
AWS p2.xlarge 1 Experimentation
p2.8xlarge 8 Serious use
p2.16xlarge 16 Serious use
p3.2xlarge 1 Experimentation
p3.8xlarge 4 Serious use
p3.16xlarge 8 Serious use
g3.4xlarge 1 Experimentation
g3.8xlarge 2 Experimentation
g3.16xlarge 4 Serious use

Installing the EC2 Instance

  1. Log in to your AWS account at https://aws.amazon.com.
  2. In the upper right corner of the Amazon Web Services page, make sure that the location drop-down is US East (N Virginia).
../_images/ami_location_dropdown.png
  1. Select the EC2 option under the Compute section to open the EC2 Dashboard.
../_images/ami_select_ec2.png
  1. Click the Launch Instance button under the Create Instance section.
../_images/ami_launch_instance_button.png
  1. Under Community AMIs, search for h2oai, and then select the version that you want to launch.
../_images/ami_select_h2oai_ami.png
  1. On the Choose an Instance Type page, select GPU compute in the Filter by dropdown. This will ensure that your Driverless AI instance will run on GPUs. Select a GPU compute instance from the available options. (We recommend at least 32 vCPUs.) Click the Next: Configure Instance Details button.
../_images/ami_choose_instance_type.png
  1. Specify the Instance Details that you want to configure. Create a VPC or use an existing one, and ensure that “Auto-Assign Public IP” is enabled and associated to your subnet. Click Next: Add Storage.
../_images/ami_configure_instance_details.png
  1. Specify the Storage Device settings. Note again that Driverless AI requires 10 GB to run and will stop working of less than 10 GB is available. The machine should have a minimum of 30 GB of disk space. Click Next: Add Tags.
../_images/ami_add_storage.png
  1. If desired, add unique Tag name to identify your instance. Click Next: Configure Security Group.
  2. Add the following security rules to enable SSH access to Driverless AI and to (optionally) enable access to H2O Flow, then click Review and Launch.
Type Protocol Port Range Source Description
SSH TCP 22 Anywhere 0.0.0.0/0  
Custom TCP Rule TCP 12345 Anywhere 0.0.0.0/0 Launch DAI
Custom TCP Rule TCP 54321 Anywhere 0.0.0.0/0 Optional access to H2O Flow
../_images/ami_add_security_rules.png
  1. Review the configuration, and then click Launch.
  2. A popup will appear prompting you to select a key pair. This is required in order to SSH into the instance. You can select your existing key pair or create a new one. Be sure to accept the acknowledgement, then click Launch Instances to start the new instance.
../_images/ami_select_key_pair.png
  1. Upon successful completion, a message will display informing you that your instance is launching. Click the View Instances button to see information about the instance including the IP address. The Connect button on this page provides information on how to SSH into your instance.
  2. Open a Terminal window and SSH into the IP address of the AWS instance. Replace the DNS name below with your instance DNS.
ssh -i "mykeypair.pem" ubuntu@ec2-34-230-6-230.compute-1.amazonaws.com
  1. If you selected a GPU-compute instance, then enable persistence of the GPU. Note that this only needs to be run once. Refer to the following for more information: http://docs.nvidia.com/deploy/driver-persistence/index.html.
sudo nvidia-persistenced --user <USER>
sudo nvidia-smi -pm 1
  1. At this point, you can copy data into the data directory on the host machine using scp. (Note that the data folder already exists.) For example:
scp <data_file>.csv ubuntu@ec2-34-230-6-230.compute-1.amazonaws.com:/home//data

The data will be visible inside the Docker container.

  1. Connect to Driverless AI with your browser:
http://Your-Driverless-AI-Host-Machine:12345

Stopping the EC2 Instance

The EC2 instance will continue to run even when you close the aws.amazon.com portal. To stop the instance:

  1. On the EC2 Dashboard, click the Running Instances link under the Resources section.
  2. Select the instance that you want to stop.
  3. In the Actions drop down menu, select Instance State > Stop.
  4. A confirmation page will display. Click Yes, Stop to stop the instance.

Upgrading the Driverless AI Image

If you have a valid license and are running a Driverless AI image from a cloud offering, there are bash commands to make the upgrade process easy. This example shows how to upgrade Driverless AI from version 1.0.18 to the latest version.

WARNING: Experiments, MLIs, and MOJOs are not automatically upgraded when Driverless AI is upgraded.

  • Build MLI models before upgrading.
  • Build MOJO pipelines before upgrading.

If you did not build MLI on a model before upgrading Driverless AI, then you will not be able to view MLI on that model after upgrading. Before upgrading, be sure to run MLI jobs on models that you want to continue to interpret in future releases. If that MLI job appears in the list of Interpreted Models in your current version, then it will be retained after upgrading.

If you did not build a MOJO pipeline on a model before upgrading Driverless AI, then you will not be able to build a MOJO pipeline on that model after upgrading. Before upgrading, be sure to build MOJO pipelines on all desired models.

  1. SSH into the IP address of the image instance and copy the existing experiments to a backup location:
# Set up a directory of the previous version name
mkdir dai_rel_1.0.18

# Copy the data, log, license, and tmp directories as backup
cp -r ./data dai_rel_1.0.18/data
cp -r ./log dai_rel_1.0.18/log
cp -r ./license dai_rel_1.0.18/license
cp -r ./tmp dai_rel_1.0.18/tmp
  1. Use the following commands to upgrade the Driverless AI version:
# Stop Driverless AI
h2oai stop

# Run the following command
h2oai update

# Start Driverless AI
h2oai start

And to see all available Driverless AI commands, type h2oai.

  1. Connect to Driverless AI with your browser at http://Your-Driverless-AI-Host-Machine:12345.