Profiles

The Profiles tab allow you to define individual cluster sizes and configurations. Admins can then give different users access to the different clusters by specifying a specific profile when launching a new cluster.

Enterprise Steam comes with four profiles:

  • default-h2o: This is enabled by default.
  • default-sparkling-internal: This is disabled by default. Configure Spark settings to enable this profile. (See the Spark section for more information.)
  • default-sparkling-external: This is disabled by default. Configure Spark settings to enable this profile. (See the Spark section for more information.)
  • default-dai: This is disabled by default. Configure Driverless AI settings to enable this profile. (See the Driverless AI section for more information.)

Note: The minimum Sparkling Water versions are 2.1.41, 2.2.27, 2.3.16, 2.4.*

From this page, you can edit any of the default profiles, add additional profiles, copy profiles, and delete profiles.

Cluster profiles page

Adding Profiles

  1. On the Configurations page, click the Profiles tab. This page shows a list of available profiles.
  2. In the Create New Profile section of this page, enter a name for the new profile and select an available type (H2O, Sparkling Water - Internal Backend, Sparkling Water - External Backend). Click Create when you are ready. This opens the Creating Profiles form. Note that this form varies depending on the Type.

H2O Type

  1. YARN Queues: Optionally specify a comma-separated list of YARN queues available for user of this profile. Leave empty if you want to let the user to specify this parameter when launching the cluster.
  2. LDAP Groups: Optionally specify a comma-separated list of LDAP groups that will have access to this cluster. Enter * to allow any LDAP user to access this profile. Leave empty if you want to manually assign each cluster profile to each user.
  3. SAML Groups: Optionally specify a comma-separated list of SAML groups that will have access to this cluster. Enter * to allow any SAML user to access this profile. Leave empty if you want to manually assign each cluster profile to each user.
  4. Cluster Limit Per User: Specify the maximum number of clusters that a user with this profile can launch.
  5. H2O Nodes: Specify the minimum, maximum, and default number of allowed H2O nodes (cluster size) for this profile.
  6. H2O Node Memory (GB): Specify the minimum, maximum, and default amount of memory to allocate to H2O for each node (in GB).
  7. H2O Node Threads: Specify the minimum, maximum, and default number of H2O threads (CPUs) to use for each node. 0 defaults to using all CPUs on the host.
  8. H2O Node Extra Memory (%): Specify the minimum, maximum, and default extra memory for internal JVM use outside of the Java heap. (This corresponds to the extramempercent Hadoop launch parameter.)
  9. Maximum Idle Time (hrs): Specify the minimum, maximum, and default idle time in hours.
  10. Maximum Uptime (hrs): Specify the minimum, maximum, and default uptime in hours.
  11. YARN Virtual Cores: Specify the minimum, maximum, and default number of YARN virtual cores.

Sparkling Water - Internal Backend Type

  1. YARN Queues: Optionally specify a comma-separated list of YARN queues available for user of this profile. Leave empty if you want to let the user to specify this parameter when launching the cluster.
  2. LDAP Groups: Optionally specify a comma-separated list of LDAP groups that will have access to this cluster. Enter * to allow any LDAP user to access this profile. Leave empty if you want to manually assign each cluster profile to each user.
  3. SAML Groups: Optionally specify a comma-separated list of SAML groups that will have access to this cluster. Enter * to allow any SAML user to access this profile. Leave empty if you want to manually assign each cluster profile to each user.
  4. Cluster Limit Per User: Specify the maximum number of clusters that a user with this profile can launch.
  5. Python Environments: Select the environment(s) that will be associated with this profile. This list of available environments comes from the Python Environments page.
  6. Spark Properties: Optionally enter additional Spark properties for this cluster. Specify one property per line using ‘key=value’ format.
  7. Driver Cores: Specify the minimum, maximum, and default number of driver cores.
  8. Driver Memory (GB): Specify the minimum, maximum, and default driver memory (in GB).
  9. Number of Executors: Specify the minimum, maximum, and default number of executors.
  10. Executor Cores: Specify the minimum, maximum, and default number of cores per executor.
  11. Executor Memory: Specify the minimum, maximum, and default amount of executor memory per node (in GB).
  12. H2O Node Threads: Specify the minimum, maximum, and default number of H2O threads (CPUs) to use for each node. 0 defaults to using all CPUs on the host.
  13. Startup Timeout (seconds): Specify the minimum, maximum, and default startup timeout in seconds. The cluster will terminate if it cannot start within this time.

Sparkling Water - External Backend Type

  1. YARN Queues: Optionally specify a comma-separated list of YARN queues available for user of this profile. Leave empty if you want to let the user to specify this parameter when launching the cluster.
  2. LDAP Groups: Optionally specify a comma-separated list of LDAP groups that will have access to this cluster. Enter * to allow any LDAP user to access this profile. Leave empty if you want to manually assign each cluster profile to each user.
  3. SAML Groups: Optionally specify a comma-separated list of SAML groups that will have access to this cluster. Enter * to allow any SAML user to access this profile. Leave empty if you want to manually assign each cluster profile to each user.
  4. Cluster Limit Per User: Specify the maximum number of clusters that a user with this profile can launch.
  5. Python Environments: Select the environment(s) that will be associated with this profile. This list of available environments comes from the Python Environments page.
  6. Spark Properties: Optionally enter additional Spark properties for this cluster. Specify one property per line using ‘key=value’ format.
  7. Driver Cores: Specify the minimum, maximum, and default number of driver cores.
  8. Driver Memory (GB): Specify the minimum, maximum, and default driver memory (in GB).
  9. Number of Executors: Specify the minimum, maximum, and default number of executors.
  10. Executor Cores: Specify the minimum, maximum, and default number of cores per executor.
  11. Executor Memory: Specify the minimum, maximum, and default amount of executor memory per node (in GB).
  12. H2O Nodes: Specify the minimum, maximum, and default number of allowed H2O nodes (cluster size) for this profile.
  13. H2O Node Memory (GB): Specify the minimum, maximum, and default amount of memory to allocate to H2O for each node (in GB).
  14. H2O Node Threads: Specify the minimum, maximum, and default number of H2O threads (CPUs) to use for each node. 0 defaults to using all CPUs on the host.
  15. Startup Timeout (seconds): Specify the minimum, maximum, and default startup timeout in seconds. The cluster will terminate if it cannot start within this time.

Driverlesss AI

  1. LDAP Groups: Optionally specify a comma-separated list of LDAP groups that will have access to this cluster. Enter * to allow any LDAP user to access this profile. Leave empty if you want to manually assign each cluster profile to each user.
  2. SAML Groups: Optionally specify a comma-separated list of SAML groups that will have access to this cluster. Enter * to allow any SAML user to access this profile. Leave empty if you want to manually assign each cluster profile to each user.
  3. Instance Limit Per User: Specify the maximum number of Driverless AI instances that a user with this profile can access.
  4. DAI Servers: Specify the Driverless AI servers that a user with this profile can access.
  5. Config Toml Override: Specify and Driverless AI configuration overrides that will be asssociated with this profile. A list of available configuration options is available in the Driverless AI config.toml file.
  1. Click Save when you are done.

Upon completion, the new profile will appear in the Existing Profile section. If necessary, you can update or delete existing profiles directly from this section.

Editing Profiles

Enterprise Steam comes with a default profile. You can edit this profile or other expstings profiles by following the steps below.

  1. On the Configurations page, click the Profiles tab. This page shows a list of available profiles.
  2. Click the Edit button beside the profile that you want to edit.
  3. Edit any properties that you want to change, then click Save at the bottom of the form.

Copying Profiles

Copying profiles is an easy way to create a new profile based on an existing one.

  1. On the Configurations page, click the Profiles tab. This page shows a list of available profiles.
  2. Click the Copy button beside the profile that you want to copy.
  3. Change the name of the profile and change any options that you want to be different from the existing profile.
  4. Click Save when you are done.

Upon completion, the new profile will appear in the Existing Profile section.

Deleting Profiles

  1. On the Configurations page, click the Profiles tab. This page shows a list of available profiles.
  2. Click the Delete button beside the profile that you want to delete.
  3. A confirmation page displays. Click Confirm to complete the deletion.