Enterprise Steam Release Notes

Change Log

Version 1.6.5 (Mar 2, 2020)

  • Fixed custom Hadoop temporary directory to be part of HADOOP_CLIENT_OPTS

Version 1.6.4 (Feb 12, 2020)

  • Fixed error in Hadoop Helper for Java 7 Hadoop distributions
  • Fixed error when creating Python environment with the same name as deleted environment
  • Improved validation and documentation for LDAPs with internal certificate authority
  • Improved documentation for Driverless AI config.toml override
  • Improved documentation for Driverless AI installation prerequisites

Version 1.6.3 (Jan 31, 2020)

  • Supports RSparkling 3.28
  • Use Hadoop Tmp Dir for Hadoop Helper commands to prevent slowdowns
  • Fixed an issue with old DAI servers preventing instances from loading
  • Fixed server panic with invalid LDAP over TLS configuration
  • Fixed initial LDAP config requiring server restart

Version 1.6.2 (Jan 20, 2020)

  • Supports Sparkling Water 3.28
  • Fixed LDAP over TLS
  • Fixed SAML token login in R client
  • Fixed H2O profile not saving YARN queues

Version 1.6.1 (Jan 14, 2020)

  • Fixed mutual TLS setup
  • Fixed Driverless AI version parser
  • Add datatable_strategy WRITE to default DAI launch config
  • Add Driverless AI instance password to the instance details
  • Docs: Updated copyright year to include 2020

Version 1.6.0 (Dec 18, 2019)

  • Complete UI overhaul of configuration page for administrators.
  • New administrator settings for individual products and deployment backends and other configuration.
  • Improved UI/UX experience.
  • Moved most of the settings from steam.yaml into the administrator configuration in Steam UI.
  • Configuration file steam.yaml is no longer required.
  • Enabling Hadoop backend now thoroughly validates all settings to ensure they are correct (Proxyuser, Kerberos, …).
  • Added ability to export and import configuration.
  • Several settings no longer require Enterprise Steam restart.
  • Added indicator when Enterprise Steam needs to restart to apply changes.
  • Added option to use existing Livy server for Sparkling Water as long as it is on the same host as Enterprise Steam.
  • HAProxy comes bundled and does not need to be installed separately.
  • Steam Hadoop Helper (Steam Manager) comes bundled and does not need to be installed separately.
  • Additional HADOOP_CLASSPATH options can be added when launching H2O clusters.
  • Added ability to download complete Enterprise Steam logs from the UI.
  • Added SAML configuration validation.
  • Added cacert option when connecting to Steam or H2O clusters via Python client.
  • Added option to ensure internal secure connections with H2O and Sparkling Water clusters.
  • Added warning when uploading non-Hadoop version of H2O engine.
  • Improved database migration.
  • Improved validation of H2O Launch Parameters.
  • Fixed bug where Sparkling Water could not launch when users reused cluster names.
  • Fixed bug where Sparkling Water could not launch when the cluster name contained spaces.
  • Fixed UI issue when uploading Python environments with the same name.
  • Fixed Python environment selection when no Python environments assigned.
  • Fixed auto-assigment of profiles to LDAP groups

Version 1.5.1 (Nov 21, 2019)

  • Fixed a database migration bug affecting new installations.

Version 1.5.0 (Oct 10, 2019)

  • Added ability to launch Driverless AI instances.
  • Added ability to use Conda environments with Sparkling Water

Version 1.4.11 (Oct 1, 2019)

  • Fix token authentication in Python client when SAML is used
  • Fix R and Python client when internal and external ports do not match
  • Support recent versions of Sparkling Water

Version 1.4.10 (Jul 19, 2019)

  • Added an ability to download H2O logs from Steam UI
  • Added a config option for custom KRB5_CONFIG (STEAM_KRB5_CONFIG)
  • Added support for templating in YARN queue names
  • Added a check to ensure the Spark version and Sparkling Water version match
  • Added a check to warn admin if Steam port is occupied
  • Improved fetching of H2O YARN job status to now use impersonation
  • Fixed h2osteam Conda package
  • Fixed issue when Sparkling Water cluster failed to start when SPARK_YARN_MODE property was set

Version 1.4.9 (May 21, 2019)

  • Added Conda packages for Python client
  • Added ability to delete multiple stopped clusters at the same time
  • Added Python PySpark path configuration to profile
  • Added YARN queue configuration to user setting
  • Removed the need for secondary external port
  • Configuration fields are now protected from trailing and leading whitespace characters
  • Updated pysparkling dependencies for Python2
  • Fixed an issue with YARN usernames being lowercased when they were not configured to do so

Version 1.4.8 (Apr 15, 2019)

  • Add support for RSparkling through Jupyter notebook
  • Add option to disable Jupyterhub
  • Expose internal address and node ID for H2O cluster in the Steam Web UI
  • Add option to specify path to custom Pyspark Python for Sparkling Water
  • Fix pysparkling dependencies for Python3
  • Updated engine upload backend

Version 1.4.7 (Apr 3, 2019)

  • Fix Sparkling Water proxy issue with upercase usernames
  • Improve uploading h2o-3 engines
  • Set SPARK_YARN_MODE correctly based on the hadoop distribution

Version 1.4.6 (Apr 1, 2019)

  • Added ability to choose H2O-3 Leader Node when starting a cluster
  • Added ability to control the number of clusters a user can spin per cluster profile
  • Added option to select default Sparkling Water backend
  • Added automatic redirection back to login with an expired session cookie
  • Added an ability to auto-assign Steam profiles according to SAML profiles
  • Docs: Add “Before you begin installation” section
  • Docs: Documented steam.yaml configuration options
  • Docs: Updated documentation
  • Fix an issue when Steam was hitting API endpoints of dead clusters
  • Fix and issue when hadoop-unjar files were not deleted from temp directory
  • Fix issue with upercase usernames and Sparkling Water on Hadoop

Version 1.4.5 (Mar 22, 2019)

  • Added Configurable Steam Web UI timeout (STEAM_WEB_UI_TIMEOUT_MIN)

Version 1.4.4 (Feb 20, 2019)

  • Make log file permissions configurable (STEAM_LOG_PERMISSIONS)
  • H2O: Communicate with cluster using leader node only
  • SW: Added support for Hive tables
  • SW: Disable Spark dynamic allocation for internal backend
  • SW: Bundle and distribute all pysparkling dependencies
  • LDAP group configuration is no longer mandatory
  • Bug fixes for Jupyterhub
  • Bug fixes for Sparkling Water params
  • Bug fixes for CDH5

Version 1.4.3 (Jan 29, 2019)

  • Added ability to provide secure HTTP headers (SERVER_STRICT_TRANSPORT, SERVER_X_XSS_PROTECTION, SERVER_CONTENT_SECURITY_POLICY)
  • Improved logging output, time in UTC
  • Accept encrypted TLS private keys
  • Log files permissions elevated for the group
  • Bug fixes for HDP Spark
  • Bug fixes for Sparkling Water integration
  • Bug fixes for Steam Python API
  • Bug fixes for LDAP connector
  • Bug fixes for Jupyterhub
  • Bug fixes for Steam Web UI

Version 1.4.2 (Jan 1, 2019)

  • Add ability to upload H2O engine jar from Python/R API
  • Remove cluster dialog waits until cluster has been shut down
  • Log successful logins
  • Log files permissions reduced
  • Steam nbextension shows “please wait” when changing clusters
  • Document setup and troubleshooting steps for Hadoop/Kerberos configuration
  • Bug fixes

Version 1.4.1 (Dec 3, 2018)

  • Enable Sparkling Water API from API
  • Hide disabled user with a checkbox
  • Display message if there are no clusters to show
  • Fix glibc dependency for steam binary
  • Fix documentation version
  • Add release notes to documentation

Version 1.4.0 (Nov 23, 2018)

  • Add Sparkling Water integration
  • Add SAML authentication
  • Add More detailed cluster profiles
  • New cluster overview
  • New launch cluster page
  • Add option to generate Personal access tokens