Configurations

The Configurations page allows Enterprise Steam Admins to add, edit, and deactivate users and roles. Users can be added either individually using the Enterprise Steam SQLite database or through an existing LDAP directory.

Note: Only Admins have access to the Configurations page.

Users page

The Configurations page consists of the following tabs.

  • The Users tab shows the current list of users and their assigned role(s).
  • The Roles tab provides a table of the permissions assigned to each role.
  • The Cluster Profiles tab allow you to define the size of clusters, the minimum and maximum memory, and the number of cores, threads, and queues for the profile.
  • The Authentication tab allows you to connect Enterprise Steam to your current user database.
  • The Licensing tab provides information on your Enterprise Steam License.
  • The YARN tab allows you to enter YARN configuration settings that will apply when users launch new clusters.
  • The Engines tab allows you to add a manager file and H2O engines.
  • The Spark tab allows you to enable Spark and provide paths to your Spark home directory and Hadoop configuration directory.
  • The Token tab allows you to create your own personal access tokens for use in scripts and on the command line.

Authentication

Enterprise Steam supports Local, LDAP, and SAML authentication. No additional configuration is required for Local authentication. Refer to the sections that follow for information on how to configure LDAP and SAML authentication.

Configure LDAP Connection Settings

Enterprise Steam ships with a built-in SQLite database. By default, Enterprise Steam uses this database to store user and cluster management metadata. You can use this database, or you can configure Enterprise Steam to work with your existing LDAP directory.

  1. Navigate to the Configurations page and select the Authentication tab.
  2. Select LDAP in the User DB Type drop down menu, then configure the LDAP connection settings. (Refer to the table below and the image that follows.)
Field Description Example
Host The LDAP host server address ldap.0xdata.loc
Port The LDAP server port 389
SSL-Enabled Enable this if your LDAP supports SSL.  
Bind DN The Distinguished Name used by the LDAP server if extended access is required. This can be left blank if anonymous bind is sufficient. cn=admin,dc=0xdata,dc=loc
Bind DN Password/Confirm The password for the Bind DN user h2o
User Base DN The location of the LDAP users, specified by the DN of your user subtree ou=users,dc=0xdata,dc=loc
User Base Filter The LDAP search filter used to filter users department=IT
User Name Attribute The User Attribute that contains the username uid
Group Names The Distinguished Name used for group synch cn=jettygroup,ou=groups,dc=0xdata,dc=loc
Group Base DN The location of your LDAP groups, specified by the DN of your user subtree ou=groups,dc=0xdata,dc=loc
Group Name Attribute The Group Attribute that contains the username cn
Static Member Attribute The attribute for static group entries memberUid
Search Request Size Limit Limit the size of search results. 0 indicates unlimited.  
Search Request Time Limit Limit the time allotted for completing search results. 0 indicates unlimited. 0
Cache Max Age (in mins) The maxium age in minutes of of LDAP record in cache before forcing a refresh. Use 0 for no cache (not recommended). 5
LDAP Configuration
  1. Click Test Config when you are done. A valid response message indicates that the configuration was successful.
  2. Click Save Config.

After LDAP is configured, users can log in to Enterprise Steam using their LDAP username and password.

Notes:

  • The Reset button clears all user-specified information in this form and resets any default values.
  • The Invalidate LDAP cache button invalidates the records in the LDAP cache and forces the cache to retrieve updated records for users.

Configure SAML Connection Settings

Perform the the following steps to configure Enterprise Steam to use SAML authentication.

  1. Navigate to the Configurations page and select the Authentication tab.
  2. Select SAML in the User DB Type drop down menu, then configure the following SAML settings:
  • SAML Settings
    • IDP Metadata Path: The path to the SAML Identity Provider (IdP) metadata file on the local file system.
    • Keystore Path: The path to the keystore file on the local file system.
    • Keystore Password: The keystore password.
    • Base URL: The base URL for Enterprise Steam. For example, http://steam.loc:8888.
  • Group Settings
    • User Name Attribute: The attribute of authorization token that contains usernames.
    • Group Name Attribute: The attribute of authorization token that contains group names.
    • Admin Group Name: The name of the admin group that will get privileges in Enterprise Steam.
  • Advanced
    • SAML Entity ID: The PartnerSpID value that will be passed to the IdP. This is optional.
    • Logout URL: Specify the URL where the user will be redirected to after logging out. This is optional. By default, users will see the “Logged Out” screen.
SAML Configuration
  1. Click Save and Enable when you are done.

Profiles

The Profiles tab allow you to define individual cluster sizes and configurations. Admins can then give different users access to the different clusters by specifying a specific profile when launching a new cluster.

Enterprise Steam comes with three profiles:

  • default-h2o: This is enabled by default.
  • default-sparkling-internal: This is disabled by default. Configure Spark settings to enable this profile. (See the Spark section for more information.)
  • default-sparkling-external: This is disabled by default. Configure Spark settings to enable this profile. (See the Spark section for more information.)

From this page, you can edit any of the default profiles, add additional profiles, copy profiles, and delete profiles.

Cluster profiles page

Adding Profiles

  1. On the Configurations page, click the Profiles tab. This page shows a list of available profiles.
  2. In the Create New Profile section of this page, enter a name for the new profile and select an available type (H2O, Sparkling Water - Internal Backend, Sparkling Water - External Backend). Click Create when you are ready. This opens the Creating Profiles form. Note that this form varies depending on the Type.

H2O Type

  1. Optionally specify a comma-separated list of YARN queues available for user of this profile. Leave empty if you want to let the user to specify this parameter when launching the cluster.
  2. Optionally specify the LDAP group(s) that will have access to this cluster. Enter * to allow any LDAP user to access this profile. Leave empty if you want to manually assign each cluster profile to each user.
  3. Specify the minimum, maximum, and default number of allowed H2O nodes (cluster size) for this profile.
  4. Specify the minimum, maximum, and default amount of memory to allocate to H2O for each node (in GB).
  5. Specify the minimum, maximum, and default number of H2O threads (CPUs) to use for each node. 0 defaults to using all CPUs on the host.
  6. Specify the minimum, maximum, and default extra memory for internal JVM use outside of the Java heap. (This corresponds to the extramempercent Hadoop launch parameter.)
  7. Specify the minimum, maximum, and default idle time in hours.
  8. Specify the minimum, maximum, and default uptime in hours.
  9. Specify the minimum, maximum, and default number of YARN virtual cores.

Sparkling Water - Internal Backend Type

  1. Optionally specify a comma-separated list of YARN queues available for user of this profile. Leave empty if you want to let the user to specify this parameter when launching the cluster.
  2. Optionally specify the LDAP group(s) that will have access to this cluster. Enter * to allow any LDAP user to access this profile. Leave empty if you want to manually assign each cluster profile to each user.
  3. Optionally enter additional Spark properties for this cluster. Specify one property per line using ‘key=value’ format.
  4. Specify the minimum, maximum, and default number of driver cores.
  5. Specify the minimum, maximum, and default driver memory (in GB).
  6. Specify the minimum, maximum, and default number of executors.
  7. Specify the minimum, maximum, and default number of cores per executor.
  8. Specify the minimum, maximum, and default executor memory per node (in GB).
  9. Specify the minimum, maximum, and default number of H2O threads (CPUs) to use for each node. 0 defaults to using all CPUs on the host.
  10. Specify the minimum, maximum, and default startup timeout in seconds. The cluster will terminate if it cannot start within this time.

Sparkling Water - External Backend Type

  1. Optionally specify a comma-separated list of YARN queues available for user of this profile. Leave empty if you want to let the user to specify this parameter when launching the cluster.
  2. Optionally specify the LDAP group(s) that will have access to this cluster. Enter * to allow any LDAP user to access this profile. Leave empty if you want to manually assign each cluster profile to each user.
  3. Optionally enter additional Spark properties for this cluster. Specify one property per line using ‘key=value’ format.
  4. Specify the minimum, maximum, and default number of driver cores.
  5. Specify the minimum, maximum, and default driver memory (in GB).
  6. Specify the minimum, maximum, and default number of executors.
  7. Specify the minimum, maximum, and default number of cores per executor.
  8. Specify the minimum, maximum, and default executor memory per node (in GB).
  9. Specify the minimum, maximum, and default number of allowed H2O nodes (cluster size) for this profile.
  10. Specify the minimum, maximum, and default amount of memory to allocate to H2O for each node (in GB).
  11. Specify the minimum, maximum, and default number of H2O threads (CPUs) to use for each node. 0 defaults to using all CPUs on the host.
  12. Specify the minimum, maximum, and default startup timeout in seconds. The cluster will terminate if it cannot start within this time.
  1. Click Save when you are done.

Upon completion, the new profile will appear in the Existing Profile section. If necessary, you can update or delete existing profiles directly from this section.

Editing Profiles

Enterprise Steam comes with a default profile. You can edit this profile or other expstings profiles by following the steps below.

  1. On the Configurations page, click the Profiles tab. This page shows a list of available profiles.
  2. Click the Edit button beside the profile that you want to edit.
  3. Edit any properties that you want to change, then click Save at the bottom of the form.

Copying Profiles

Copying profiles is an easy way to create a new profile based on an existing one.

  1. On the Configurations page, click the Profiles tab. This page shows a list of available profiles.
  2. Click the Copy button beside the profile that you want to copy.
  3. Change the name of the profile and change any options that you want to be different from the existing profile.
  4. Click Save when you are done.

Upon completion, the new profile will appear in the Existing Profile section.

Deleting Profiles

  1. On the Configurations page, click the Profiles tab. This page shows a list of available profiles.
  2. Click the Delete button beside the profile that you want to delete.
  3. A confirmation page displays. Click Confirm to complete the deletion.

Users

The Users tab shows all current Enterprise Steam users. This section describes how to add, edit, and deactivate users.

Adding Users

Admins can add users into the Enterprise Steam SQLite database from within the UI.

  1. At the top of the Configurations page, click the Create User button.
Create users button
  1. Enter the name of the user. Note that the name must match with a username in your YARN system.
  2. Specify and confirm a password for the user.
  3. Specify the role(s) for this user. Note that Enterprise Steam ships with two default roles: admin and standard user.
  4. Specify the cluster profile that this user will be part of. Note that Enterprise Steam ships with a default cluster profile.
  5. Click Create User when you are done.
Create user

Upon successful completion, the new user will appear in the list of Enterprise Steam users.

Editing Users

This section describes how to edit a user’s role.

On the Users tab, click the Edit link beside the user you want to edit. This opens the Edit User Details form. Change the user’s roles or cluster profile. You can also specify an authentication type of LDAP, Local, or SAML. Click Confirm when you are done.

Edit user

Note: A message will display in the UI if you remove all roles from a user.

Resetting a User’s Password

If a user is added with Local Authentication, then admins can reset the user’s password by clicking the Reset Local Password link for the desired user. A new password will display at the top of the screen for approximately 5 seconds. This new password should then be provided to the user so that he/she can log in to Enterprise Steam. Note that this option is not available for users added with LDAP or SAML authentication.

Reset user's password

Deactivating/Reactivating Users

On the Users tab, click the Deactivate Steam User link for the user whose Enterprise Steam access you want to revoke. Click Reactivate Steam User to once again grant access for that user.

Deactivate/reactivate user

Roles

Roles determine the activities/permissions that an Enterprise Steam user can perform within your environment. Enterprise Steam ships with two default roles: admin and standard user. These default roles are sufficient for most Enterprise Steam deployments and, in general, should not be changed. You can create additional roles, however, if you require more granularity in the way that your users access and utilize Enterprise Steam.

Creating Roles

  1. At the top of the Configurations page, click the Create Role button.
Create role button
  1. Specify a name and description for the role.
  2. Select the permissions that will be granted to this role.
  3. Click Create Role at the bottom of the form when you are done.
Create role

Changing Permissions

Admins can add or remove permissions for each role directly on the Roles tab.

  1. Select the checkbox for the correspoding permission and role that you want to change
  2. Click Review Changes at the bottom of the page. A popup displays, providing you with a summary of the changes.
  3. Click the Confirm button beside each change that you want to make, then click Save Changes to complete the update.
Change permissions

Deleting Roles

On the Roles tab, scroll down to the bottom of the page, and click the trashcan icon under the Role column that you want to delete. A confirmation page will display, prompting you to confirm the deletion. Click Confirm to remove the role.

Delete role

YARN

A YARN tab is available for Enterprise Steam customers that want to control the way that users launch new clusters, whether it is specifying a YARN queue or limiting the number of nodes that a user can create when launching a new cluster.

Adding a New YARN Config Entry

Perform the following steps to add a new config entry.

Note: Use caution when adding a new config entry. These values are currently not validated, and incorrect entries will result in a failure on cluster launch.

  1. Specify a valid config name. This name must be a valid Hadoop launch parameter (such as network, driverif, driverport, nodes, etc.)
  2. Specify a value for the config. This value must correspond to the name. For example, you cannot enter an IP address if the new entry is for nodes.
  3. Specify whether this configuration will act as a default or whether this will override any values that the user sets.
  4. Specify whether this entry is a Hadoop type (passed directly to the YARN command line) or an H2O Driver type (specifying how Enterprise Steam will run).
  5. Click Add Entry when you are done. The new entry will display on this page. Repeat these steps to add additional YARN config entries.
YARN config entry

Updating a YARN Config Entry

On the YARN tab, enter new values in the entry that you want to update, then click the Update Entry button.

Update YARN config entry

Deleting a YARN Config Entry

On the YARN tab, click the Remove Entry button below the entry that you want to remove.

Delete YARN config entry

Engines

The Engines tab allows you to upload h2o-3 drivers (engines) and optionally tie the Enterprise Steam environment to a Steam Hadoop helper file. Once added, these drivers will be available for Enterprise Steam users.

  1. On your local machine, download the h2odriver from the H2O Download page. Be sure to select the version that corresponds with your version of Hadoop. For example:
wget http://h2o-release.s3.amazonaws.com/h2o/rel-xia/4/h2o-3.22.0.1-hdp2.4.zip
  1. Navigate to the Configurations > Engines tab.
Engines tab
  1. Optionally upload a new manager file. This is the Steam Hadoop helper, which is used to prevent clusters from using impersonation. This can be deleted after being added.
  2. Browse to and select the h2o driver that you want to add.

Upon completion, the new driver will appear on this page and will be available in the Launch New Cluster > H2O Version dropdown menu.

You can delete drivers from this page by selecting the trashcan icon beside the driver that you want to remove.

Spark

The Spark tab allows you to enable Spark and provide paths to your Spark home directory and Hadoop configuration directory. When Spark is enabled, then the default-sparkling-internal and default-sparkling-external profiles will be available. (See the Profiles section for more information.)

  1. Navigate to the Configurations > Spark tab.
Spark tab
  1. Click Enabled to enable Spark.
  2. Specify the path to your Spark home directory (SPARK_HOME).
  3. Specify the path to your Hadoop configuration directory (HADOOP_CONF_DIR).
  4. Click Save Config when you are done.

Token

The Token tab allows you to generate a personal access tokens for use in scripts and on the command line. Note: Be careful, these tokens are like passwords so you should guard them carefully. The advantage to using a token over putting your password into a script is that a token can be revoked.

On the Configuration > Token tab, click Generate New Token to generaate and retrieve your token. Note: For security reasons the token will be shown only once after generating. If you lose your token, you must generate a new one. You can only have one token at a time.

Token tab