Minio Setup

This section provides instructions for configuring Driverless AI to work with Minio. Note that unlike S3, authentication must also be configured when the Minio data connector is specified.

Description of Configuration Attributes

  • minio_endpoint_url: The endpoint URL that will be used to access Minio.

  • minio_access_key_id: The Minio access key.

  • minio_secret_access_key: The Minio secret access key.

  • enabled_file_systems: The file systems you want to enable. This must be configured in order for data connectors to function properly.

Start Driverless AI

The following sections describe how to enable the Minio data connector when starting Driverless AI in Docker. This can done by specifying each environment variable in the nvidia-docker run command or by editing the configuration options in the config.toml file and then specifying that file in the nvidia-docker run command.

Enable Minio with Authentication

This example enables the Minio data connector with authentication by passing an endpoint URL, access key ID, and an access key. It also configures Docker DNS by passing the name and IP of the name node. This allows users to reference data stored in Minio directly using the endpoint URL, for example: http://<endpoint_url>/<bucket>/datasets/iris.csv. Replace TAG below with the image tag.

nvidia-docker run \
        --shm-size=256m \
        --add-host name.node:172.16.2.186 \
        -e DRIVERLESS_AI_ENABLED_FILE_SYSTEMS="file,minio" \
        -e DRIVERLESS_AI_MINIO_ENDPOINT_URL="<endpoint_url>"
        -e DRIVERLESS_AI_MINIO_ACCESS_KEY_ID="<access_key_id>" \
        -e DRIVERLESS_AI_MINIO_SECRET_ACCESS_KEY="<access_key>" \
        -e DRIVERLESS_AI_MINIO_SKIP_CERT_VERIFICATION="false" \
        -p 12345:12345 \
        --init -it --rm \
        -v /tmp/dtmp/:/tmp \
        -v /tmp/dlog/:/log \
        -v /tmp/dlicense/:/license \
        -v /tmp/ddata/:/data \
        -u $(id -u):$(id -g) \
        h2oai/dai-centos7-x86_64:TAG

Start DAI by Updating the config.toml File

This example shows how to configure Minio options in the config.toml file, and then specify that file when starting Driverless AI in Docker.

  1. Configure the Driverless AI config.toml file. Set the following configuration options.

  • enabled_file_systems = "file, upload, minio"

  • minio_endpoint_url = "<endpoint_url>"

  • minio_access_key_id = "<access_key_id>"

  • minio_secret_access_key = "<access_key>"

  • minio_skip_cert_verification = "false"

  1. Mount the config.toml file into the Docker container.

nvidia-docker run \
  --pid=host \
  --init \
  --rm \
  --shm-size=256m \
  --add-host name.node:172.16.2.186 \
  -e DRIVERLESS_AI_CONFIG_FILE=/path/in/docker/config.toml \
  -p 12345:12345 \
  -v /local/path/to/config.toml:/path/in/docker/config.toml \
  -v /etc/passwd:/etc/passwd:ro \
  -v /etc/group:/etc/group:ro \
  -v /tmp/dtmp/:/tmp \
  -v /tmp/dlog/:/log \
  -v /tmp/dlicense/:/license \
  -v /tmp/ddata/:/data \
  -u $(id -u):$(id -g) \
  h2oai/dai-centos7-x86_64:TAG