Azure Blob Store Setup

Driverless AI allows you to explore Azure Blob Store data sources from within the Driverless AI application. This section describes how to enable the Azure Blob Store data connector in Docker environments.

Description of Configuration Attributes

azure_blob_account_name: The Microsoft Azure Storage account name. This should be the dns prefix created when the account was created (for example, “mystorage”).

azure_blob_account_key: Specify the account key that maps to your account name.

azure_connection_string: Optionally specify a new connection string. With this option, you can include an override for a host, port, and/or account name. For example,

azure_connection_string = "DefaultEndpointsProtocol=http;AccountName=<account_name>;AccountKey=<account_key>;BlobEndpoint=http://<host>:<port>/<account_name>;"

Start Driverless AI

This section describes how to enable the Azure Blob Storer data connector when starting Driverless AI in Docker. This can done by specifying each environment variable in the nvidia-docker run command or by editing the configuration options in the config.toml file and then specifying that file in the nvidia-docker run command.

Start DAI Using Environment Variables

This example enables the Azure Blob Store data connector. This allows users to reference data stored on your Azure storage account using the account name, for example: https://mystorage.blob.core.windows.net. Replace TAG below with the image tag.

nvidia-docker run \
  --pid=host \
  --init \
  --rm \
  --shm-size=256m \
  -e DRIVERLESS_AI_ENABLED_FILE_SYSTEMS="file,azrbs" \
  -e DRIVERLESS_AI_AZURE_BLOB_ACCOUNT_NAME="mystorage" \
  -e DRIVERLESS_AI_AZURE_BLOB_ACCOUNT_KEY="<access_key>" \
  -p 12345:12345 \
  -v /tmp/dtmp/:/tmp \
  -v /tmp/dlog/:/log \
  -v /tmp/dlicense/:/license \
  -v /tmp/ddata/:/data \
  -u $(id -u):$(id -g) \
  h2oai/dai-centos7-x86_64:TAG

Start DAI by Updating the config.toml File

This example shows how to configure Azure Blob Store options in the config.toml file, and then specify that file when starting Driverless AI in Docker.

  1. Configure the Driverless AI config.toml file. Set the following configuration options:
  • enabled_file_systems = "file, upload, azrbs"
  • azure_blob_account_name = "mystorage"
  • azure_blob_account_key = "<account_key>"
  1. Mount the config.toml file into the Docker container.
nvidia-docker run \
  --pid=host \
  --init \
  --rm \
  --shm-size=256m \
  --add-host name.node:172.16.2.186 \
  -e DRIVERLESS_AI_CONFIG_FILE=/path/in/docker/config.toml
  -p 12345:12345 \
  -v /local/path/to/config.toml:/path/in/docker/config.toml
  -v /etc/passwd:/etc/passwd:ro \
  -v /etc/group:/etc/group:ro \
  -v /tmp/dtmp/:/tmp \
  -v /tmp/dlog/:/log \
  -v /tmp/dlicense/:/license \
  -v /tmp/ddata/:/data \
  -u $(id -u):$(id -g) \
  h2oai/dai-centos7-x86_64:TAG