Exporting Artifacts

In some cases, you might find that you do not want your users to download artifacts directly to their machines. Driverless AI provides several configuration options/environment variables that enable exporting of artifacts instead of downloading. Artifacts can be exported to a file system directory, an Amazon S3 bucket, a Bitbucket repository, or Azure Blob storage.

Note: The option to download artifacts is automatically disabled when exporting is enabled.

Enabling Artifact Exports

The config.toml file exposes the following variables:

• enable_artifacts_upload: Replace all the downloads on the experiment page to exports, and lets users push to the artifact store with artifacts_store. This is disabled by default.

• artifacts_store: Specify one of the following storage methods:

  • file_system: Store artifacts in the file system directory specified by the artifacts_file_system_directory setting.

  • S3: Store artifacts in the S3 bucket specified by the artifacts_s3_bucket setting.

  • bitbucket: Store artifacts in a Bitbucket repository.

  • azure: Store artifacts in Azure Blob storage.

Specify the following for the storage method you selected:

File System Directory

• artifacts_file_system_directory: The file system location where artifacts will be copied. This is expected to be a directory on your server.

AWS S3

• artifacts_s3_bucket: The AWS S3 bucket where artifacts will be stored.

Bitbucket

• bitbucket_skip_cert_verification: Specify whether to skip certificate verification for Bitbucket when using a repository with HTTPS. This is disabled by default.

• bitbucket_tmp_relative_dir: Specify a local temporary directory to clone artifacts to (relative to data_directory).

Azure Blob Storage

• artifacts_azure_blob_account_name: Specify your Azure Blob Storage account name.

• artifacts_azure_blob_account_key: Specify your Azure Blob Storage account key.

• artifacts_azure_connection_string: Specify your Azure Blob Storage connection string.

• artifacts_azure_sas_token: Specify your Azure Blob Storage shared access signatures (SAS) token.

Note: The option to disable artifact downloads does not extend to datasets. Whether users can download datasets is controlled by the enable_dataset_downloading configuration option, which is set to true by default. Set this to false if you do not want users to download datasets to their local machine. There is currently no configuration option that enables exporting datasets to a file system.

Docker Image InstallsNative Installs

The following example shows how to enable artifact exporting to a file system when starting the Driverless AI Docker image.

docker run \
--pid=host \
--rm \
--shm-size=2g --cap-add=SYS_NICE --ulimit nofile=131071:131071 --ulimit nproc=16384:16384 \
-e DRIVERLESS_AI_ENABLE_ARTIFACTS_UPLOAD="true" \
-e DRIVERLESS_AI_ARTIFACTS_STORE="file_system" \
-e DRIVERLESS_AI_ARTIFACTS_FILE_SYSTEM_DIRECTORY="tmp" \
-u `id -u`:`id -g` \
-p 12345:12345 \
-v `pwd`/data:/data \
-v `pwd`/log:/log \
-v `pwd`/license:/license \
-v `pwd`/tmp:/tmp \
h2oai/dai-ubi8-x86_64:1.11.1.1-cuda11.8.0.xx

Exporting an Artifact

When the export artifacts options are enabled/configured, the menu options on the Completed Experiment Page page will change. Specifically, all “Download” options (with the exception of AutoDoc) will change to “Export.”

1. Click on an artifact to begin exporting. For example, click on Export Summary and Logs.

2. Specify a file name or use the default file name. This denotes the new name to be given to the exported artifact. By default, this name matches the selected export artifact name.

3. Now click the Summary and Logs: Export to Data Store button. (Note that this button name changes depending on the artifact that you select.) This begins the export action. Upon completion, the exported artifact will display in the list of artifacts. The directory structure is: <path_to_export_to>/<user>/<experiment_id>/

4. Continue exporting additional artifacts for this experiment.