Deployment configuration

deployment_aws_access_key_id

deployment_aws_access_key_id (String)

Default value ''

Default AWS credentials to be used for scorer deployments.

deployment_aws_secret_access_key

deployment_aws_secret_access_key (String)

Default value ''

Default AWS credentials to be used for scorer deployments.

deployment_aws_bucket_name

deployment_aws_bucket_name (String)

Default value ''

AWS S3 bucket to be used for scorer deployments.

triton_benchmark_runtime

triton_benchmark_runtime (Number)

Default value 5

Approximate upper limit of time for Triton to take to compute latency and throughput performance numbers when performing ‘Benchmark’ operations for a deployment. Higher values result in more accurate performance numbers.

triton_quick_test_runtime

triton_quick_test_runtime (Number)

Default value 2

Approximate upper limit of time for Triton to take to compute latency and throughput performance numbers after loading up the deployment, per model. Higher values result in more accurate performance numbers.

deploy_wizard_num_per_page

deploy_wizard_num_per_page (Number)

Default value 10

Number of Triton deployments to show per page of the Deploy Wizard

triton_host_remote

Hostname of remote Triton inference server. (String) (Expert Setting)

Default value ''

Hostname (or IP address) of remote Triton inference service (outside of DAI), to be used when auto_deploy_triton_scoring_pipeline and make_triton_scoring_pipeline are not disabled. If set, check triton_model_repository_dir_remote and triton_server_params_remote as well.

triton_model_repository_dir_remote

triton_model_repository_dir_remote (String) (Expert Setting)

Default value ''

Path to model repository directory for remote Triton inference server outside of Driverless AI. All Triton deployments for all users are stored in this directory. Requires write access to this directory from Driverless AI (shared file system). This setting is optional. If not provided, will upload each model deployment over gRPC protocol.

triton_server_params_remote

Remote Triton server parameters, used to connect via tritonclient (Dict) (Expert Setting)

Default value {'http-port': 8000, 'grpc-port': 8001, 'metrics-port': 8002, 'model-control-mode': 'explicit'}

Parameters to connect to remote Triton server, only used if triton_host_remote and

triton_model_repository_dir_remote are set. Note: ‘model-control-mode’ need to be set to ‘explicit’ in order to allow DAI upload model to remote triton server. .