Install on Azure¶
This section describes how to install the Driverless AI image from Azure.
Note: Prior versions of the Driverless AI installation and upgrade on Azure were done via Docker. This is no longer the case as of version 1.5.2.
Watch the installation video here. Note that some of the images in this video may change between releases, but the installation steps remain the same.
Environment¶
Provider |
Instance Type |
Num GPUs |
Suitable for |
---|---|---|---|
Azure |
Standard_NV6 |
1 |
Experimentation |
Standard_NV12 |
2 |
Experimentation |
|
Standard_NV24 |
4 |
Serious use |
|
Standard_NC6 |
1 |
Experimentation |
|
Standard_NC12 |
2 |
Experimentation |
|
Standard_NC24 |
4 |
Serious use |
About the Install¶
The ‘dai’ service user is created locally (in
/etc/passwd
) if it is not found by ‘getent passwd’. You can override the user by providing the DAI_USER environment variable during rpm or dpkg installation.The ‘dai’ service group is created locally (in
/etc/group
) if it is not found by ‘getent group’. You can override the group by providing the DAI_GROUP environment variable during rpm or dpkg installation.Configuration files are placed in
/etc/dai
and owned by the ‘root’ user:/etc/dai/config.toml
: Driverless AI config file (See Using the config.toml File section for details)./etc/dai/User.conf
: systemd config file specifying the service user./etc/dai/Group.conf
: systemd config file specifying the service group./etc/dai/EnvironmentFile.conf
: systemd config file specifying (optional) environment variable overrides.
Software files are placed in
/opt/h2oai/dai
and owned by the ‘root’ userThe following directories are owned by the service user so that they can be updated by the running software:
/opt/h2oai/dai/home
: The application’s home directory (license key files are stored here)./opt/h2oai/dai/tmp
: Experiments and imported data are stored here./opt/h2oai/dai/log
: Log files go here if you are not using systemd (if you are using systemd, then the use the standard journalctl tool).
By default, for Docker or DEB/RPM installs, Driverless AI looks for a license key in
/opt/h2oai/dai/home/.driverlessai/license.sig
. If you are installing Driverless AI programmatically, you can copy a license key file to that location. For TAR SH installs, the equivalent location is<tar.sh dir>/home/.driverlessai
, and after the license is imported, it is copied under~/.driverlessai
. If no license key is found, the application guides you through the process of adding one through the UI.systemd unit files are placed in
/usr/lib/systemd/system
.Symbolic links to the configuration files in
/etc/dai
files are placed in/etc/systemd/system
.
If your environment is running an operational systemd, that is the preferred way to manage Driverless AI. The package installs the following systemd services and a wrapper service:
dai: Wrapper service that starts/stops the other services.
dai-dai: Main Driverless AI process.
If you don’t have systemd, refer to Linux TAR SH for install instructions.
Installing the Azure Instance¶
Log in to your Azure portal at https://portal.azure.com, and click the Create a Resource button.
Search for and select H2O DriverlessAI in the Marketplace.
Click Create. This launches the H2O DriverlessAI Virtual Machine creation process.
On the Basics tab:
Enter a name for the VM.
Select the Disk Type for the VM. Use HDD for GPU instances.
Enter the name that you will use when connecting to the machine through SSH.
Enter and confirm a password that will be used when connecting to the machine through SSH.
Specify the Subscription option. (This should be Pay-As-You-Go.)
Enter a name unique name for the resource group.
Specify the VM region.
Click OK when you are done.
On the Size tab, select your virtual machine size. Specify the HDD disk type and select a configuration. We recommend using an N-Series type, which comes with a GPU. Also note that Driverless AI requires 10 GB of free space in order to run and will stop working of less than 10 GB is available. We recommend a minimum of 30 GB of disk space. Click OK when you are done.
On the Settings tab, select or create the Virtual Network and Subnet where the VM is going to be located and then click OK.
The Summary tab performs a validation on the specified settings and will report back any errors. When the validation passes successfully, click Create to create the VM.
After the VM is created, it will be available under the list of Virtual Machines. Select this Driverless AI VM to view the IP address of your newly created machine.
Connect to Driverless AI with your browser using the IP address retrieved in the previous step.
http://Your-Driverless-AI-Host-Machine:12345
Stopping the Azure Instance¶
The Azure instance will continue to run even when you close the Azure portal. To stop the instance:
Click the Virtual Machines left menu item.
Select the checkbox beside your DriverlessAI virtual machine.
On the right side of the row, click the … button, then select Stop. (Note that you can then restart this by selecting Start.)
Upgrading the Driverless AI Image¶
WARNINGS:
This release deprecates experiments and MLI models from 1.7.0 and earlier.
Experiments, MLIs, and MOJOs reside in the Driverless AI tmp directory and are not automatically upgraded when Driverless AI is upgraded. We recommend you take the following steps before upgrading.
Build MLI models before upgrading.
Build MOJO pipelines before upgrading.
Stop Driverless AI and make a backup of your Driverless AI tmp directory before upgrading.
If you did not build MLI on a model before upgrading Driverless AI, then you will not be able to view MLI on that model after upgrading. Before upgrading, be sure to run MLI jobs on models that you want to continue to interpret in future releases. If that MLI job appears in the list of Interpreted Models in your current version, then it will be retained after upgrading.
If you did not build a MOJO pipeline on a model before upgrading Driverless AI, then you will not be able to build a MOJO pipeline on that model after upgrading. Before upgrading, be sure to build MOJO pipelines on all desired models and then back up your Driverless AI tmp directory.
The upgrade process inherits the service user and group from /etc/dai/User.conf and /etc/dai/Group.conf. You do not need to manually specify the DAI_USER or DAI_GROUP environment variables during an upgrade.
Upgrading from Version 1.2.2 or Earlier¶
It is not possible to upgrade from version 1.2.2 or earlier to the latest version. You have to manually remove the 1.2.2 container and then reinstall the latest Driverless AI version. Be sure to backup your data before doing this.
Upgrading from Version 1.3.0 to 1.5.1¶
SSH into the IP address of the image instance and copy the existing experiments to a backup location:
# Set up a directory of the previous version name mkdir dai_rel_1.3.0 # Copy the data, log, license, and tmp directories as backup cp -a ./data dai_rel_1.3.0/data cp -a ./log dai_rel_1.3.0/log cp -a ./license dai_rel_1.3.0/license cp -a ./tmp dai_rel_1.3.0/tmp
wget
the newer image. Replace VERSION and BUILD below with the Driverless AI version.
wget https://s3.amazonaws.com/artifacts.h2o.ai/releases/ai/h2o/dai/VERSION-BUILD/x86_64/dai-docker-ubi8-x86_64-VERSION.tar.gz
Use the
docker load
command to load the image:
docker load < dai-docker-ubi8-x86_64-VERSION.tar.gz
Run
docker images
to find the new image tag.Start the Driverless AI Docker image and replace TAG below with the image tag. Depending on your install version, use the
docker run --runtime=nvidia
(>= Docker 19.03) ornvidia-docker
(< Docker 19.03) command.
Note: Use
docker version
to check which version of Docker you are using.# Start the Driverless AI Docker image docker run --runtime=nvidia \ --pid=host \ --init \ --rm \ --shm-size=2g --cap-add=SYS_NICE --ulimit nofile=131071:131071 --ulimit nproc=16384:16384 \ -u `id -u`:`id -g` \ -p 12345:12345 \ -v `pwd`/data:/data \ -v `pwd`/log:/log \ -v `pwd`/license:/license \ -v `pwd`/tmp:/tmp \ h2oai/dai-ubi8-x86_64:TAG# Start the Driverless AI Docker image nvidia-docker run \ --pid=host \ --init \ --rm \ --shm-size=2g --cap-add=SYS_NICE --ulimit nofile=131071:131071 --ulimit nproc=16384:16384 \ -u `id -u`:`id -g` \ -p 12345:12345 \ -v `pwd`/data:/data \ -v `pwd`/log:/log \ -v `pwd`/license:/license \ -v `pwd`/tmp:/tmp \ h2oai/dai-ubi8-x86_64:TAG
Upgrading from version 1.5.2 or Later¶
Upgrading to versions 1.5.2 and later is no longer done via Docker. Instead, perform the following steps if you are upgrading to version 1.5.2 or later. Replace dai_NEWVERSION.deb
below with the new Driverless AI version (for example, dai_1.8.4.1_amd64.deb
). Note that this upgrade process inherits the service user and group from /etc/dai/User.conf and /etc/dai/Group.conf. You do not need to manually specify the DAI_USER or DAI_GROUP environment variables during an upgrade.
We recommend to have NVIDIA driver >= 471.68 installed (GPU only) in your host environment for a seamless experience on all architectures, including Ampere. Driverless AI ships with CUDA 11.8.0 for GPUs, but the driver must exist in the host environment.
Go to NVIDIA download driver to get the latest NVIDIA Tesla A/T/V/P/K series drivers. For reference on CUDA Toolkit and Minimum Required Driver Versions and CUDA Toolkit and Corresponding Driver Versions, see here .
Note
If you are using K80 GPUs, the minimum required NVIDIA driver version is 450.80.02.
# Stop Driverless AI.
sudo systemctl stop dai
# Backup your /opt/h2oai/dai/tmp directory at this time.
# Upgrade Driverless AI.
sudo dpkg -i dai_NEWVERSION.deb
sudo systemctl daemon-reload
sudo systemctl start dai