kdb+ Setup

Driverless AI lets you explore kdb+ data sources from within the Driverless AI application. This section provides instructions for configuring Driverless AI to work with kdb+.

Note: Depending on your Docker install version, use either the docker run --runtime=nvidia (>= Docker 19.03) or nvidia-docker (< Docker 19.03) command when starting the Driverless AI Docker image. Use docker version to check which version of Docker you are using.

Description of Configuration Attributes

  • kdb_user: (Optional) User name

  • kdb_password: (Optional) User’s password

  • kdb_hostname: IP address or host of the KDB server

  • kdb_port: Port on which the kdb+ server is listening

  • kdb_app_jvm_args: (Optional) JVM args for kdb+ distributions (for example, -Dlog4j.configuration). Separate each argument with spaces.

  • kdb_app_classpath: (Optional) The kdb+ classpath (or other if the jar file is stored elsewhere).

  • enabled_file_systems: The file systems you want to enable. This must be configured in order for data connectors to function properly.

Example 1: Enable kdb+ with No Authentication

This example enables the kdb+ connector without authentication. The only required flags are the hostname and the port.

 nvidia-docker run \
   --pid=host \
   --init \
   --rm \
   --shm-size=2g --cap-add=SYS_NICE --ulimit nofile=131071:131071 --ulimit nproc=16384:16384 \
   --add-host name.node:172.16.2.186 \
   -e DRIVERLESS_AI_ENABLED_FILE_SYSTEMS="file,kdb" \
   -e DRIVERLESS_AI_KDB_HOSTNAME="<ip_or_host_of_kdb_server>" \
   -e DRIVERLESS_AI_KDB_PORT="<kdb_server_port>" \
   -p 12345:12345 \
   -v /tmp/dtmp/:/tmp \
   -v /tmp/dlog/:/log \
   -v /tmp/dlicense/:/license \
   -v /tmp/ddata/:/data \
   -u $(id -u):$(id -g) \
   h2oai/dai-ubi8-x86_64:1.11.1.1-cuda11.8.0.xx

Example 2: Enable kdb+ with Authentication

This example provides users credentials for accessing a kdb+ server from Driverless AI.

 nvidia-docker run \
   --pid=host \
   --init \
   --rm \
   --shm-size=2g --cap-add=SYS_NICE --ulimit nofile=131071:131071 --ulimit nproc=16384:16384 \
   -e DRIVERLESS_AI_ENABLED_FILE_SYSTEMS="file,kdb" \
   -e DRIVERLESS_AI_KDB_HOSTNAME="<ip_or_host_of_kdb_server>" \
   -e DRIVERLESS_AI_KDB_PORT="<kdb_server_port>" \
   -e DRIVERLESS_AI_KDB_USER="<username>" \
   -e DRIVERLESS_AI_KDB_PASSWORD="<password>" \
   -p 12345:12345 \
   -v /tmp/dtmp/:/tmp \
   -v /tmp/dlog/:/log \
   -v /tmp/dlicense/:/license \
   -v /tmp/ddata/:/data \
   -u $(id -u):$(id -g) \
   h2oai/dai-ubi8-x86_64:1.11.1.1-cuda11.8.0.xx

Adding Datasets Using kdb+

After the kdb+ connector is enabled, you can add datasets by selecting kdb+ from the Add Dataset (or Drag and Drop) drop-down menu.

Add Dataset

Specify the following information to add your dataset.

  1. Enter filepath to save query. Enter the local file path for storing your dataset. For example, /home/<user>/myfile.csv. Note that this can only be a CSV file.

  2. Enter KDB Query: Enter a kdb+ query that you want to execute. Note that the connector will accept any q qeuries. For example: select from <mytable> or <mytable> lj <myothertable>

  3. When you are finished, select the Click to Make Query button to add the dataset.

Add dataset