Skip to main content

H2O AI Hybrid Cloud release notes

v24.11.0 | Nov 19, 2024​

Important

End of support date: Nov 19, 2025.

For more information, see About version support.

New components​

  • H2O Orchestrator v1.0.4
  • H2O Token Server v0.2.2
  • H2O Model Hub v0.1.0
  • H2O Secure Store v1.0.0

Upgraded components​

Core components

The following components were upgraded in this major release:

  • AI App Store v1.4.2
  • H2O Enterprise Steam v1.9.12
  • H2O AI Engine Manager v0.8.4
  • H2O MLOps v0.67.4
  • H2O-3 v3.46.0.5 & v3.44.0.4
  • H2O Driverless AI v1.10.x
  • H2O Document AI v0.9.0
  • H2O eScorer v0.11.1
  • H2O Drive v3.3.0
  • AI Notebooks v0.3.0
  • H2O Cloud Discovery Service v2.4.2
  • H2O Telemetry v1.2.7
  • H2O AuthZ v4.4.0

Optional components

  • H2O Hydrogen Torch v1.4.2
  • H2O LLM Studio v1.13.0
  • H2O LLM Data Studio v0.6.4

New features​

AI App Store​

  • Added a globally configurable SHMSize limit (default to 2Gi).
  • Added a new component called H2O Orchestrator that handles orchestration and scheduling capabilities for workflows for sequential execution of scripts and notebooks.
  • Added a Refresh Service Account Token for telemetry reporting
  • Added the ability to copy the app.toml code directly from the Admin Secrets UI.
  • The AI App Store Server is now FIPS compliant.
  • Added a new identity and access management role for allowing/disabling full access to users (is_full_access and !is_full_access roles). Full access users can now set custom images by default.
  • The ContainerOnly mode and Runtime image field can be left empty for container apps.
  • All AI App Store users must now be assigned to a role.
  • The registry and version can now be templated for custom images ({{VERSION}} {{REGISTRY}}) by setting config.appRegistry.
  • Added extraIngress and extraEgress to the network policy.
info

For a full list of features, improvements, and bug fixes in AI App Store, see the AI App Store v0.36.4-v1.4.2 release notes.

H2O MLOps​

  • Per project monitoring data retention period can be set for Influx DB during the MLOps installation or upgrade.
  • Added the functionality to log out from the Wave app.
  • Added support for:
    • New HT Flexible Runtimes for Python 3.10, including GPU and CPU variants.
    • MLflow Model Scorer and Dynamic Model Scorers for Python 3.10 and 3.11.
    • Token-based authentication for deployments.
    • H2O Driverless AI (DAI) 1.10.7.2 Python scoring pipeline and H2O DAI runtime versions 1.10.6.3, 1.10.7.1, 1.10.7.2, and 1.11.0.
  • Added component configuration support for applying tolerations, node selectors, and affinity settings to cron jobs.
  • Added CA certificate support to the API Gateway deployment.
  • Replaced Ambassador with Gateway API due to the removal of Emissary.
info

For a full list of features, improvements, and bug fixes in H2O MLOps, see the H2O MLOps v0.67.0 - v0.67.4 release notes.

H2O-3​

  • Ensured H2O-3 can load data from Snowflake using JDBC connector.
  • Updated how ModelSelection handles categorical predictors to preserve the best categorical predictor when the best categorical level performs well relative to other predictors.
  • Resolved that MOJO is working for Isolation Forest and Extended Isolation forest for implemented versions.
info

For a full list of features, improvements, and bug fixes in H2O-3, see the H2O-3 v3.44.0.4 & v3.46.0.5 release notes.

H2O Document AI​

  • Added ability to downgrade your pipeline version to the previous version. This lets you recover pipelines from failure after auto-upgrade.
  • Added visible logs you can access if your pipeline fails to start so you can debug it easier.
  • Added ability to put pipelines to sleep to keep them from consuming resources when not actively being used.
  • Added a validation rule to the pipeline that checks that the requested CPU or memory is not higher than the upper limit.
info

For a full list of features, improvements, and bug fixes in H2O Document AI, see the H2O Document AI v0.9.0 release notes.

H2O Hydrogen Torch​

  • H2O Hydrogen Torch now supports the following problem types:
    • Graph node classification
    • Graph node regression
    • Image + text classification (taking a combined input of image & text data)
  • For the following supported problem types, H2O Hydrogen Torch can now export a model to an open neural network exchange (ONNX) format:
    • Text regression and classification
    • Image classification, regression, and object detection
    • Audio classification and regression
  • The H2O Hydrogen Torch code base and scoring clients have been upgraded from Python 3.8 to Python 3.10.
info

For a full list of features, improvements, and bug fixes in H2O Hydrogen Torch, see the H2O Hydrogen Torch release notes.

Known issues and workarounds​

Known IssueWorkaround
An intermittent token authentication error may occur when using H2O Orchestrator.Restart the tokenserver pods.

v24.07.2 | Dec 05, 2024​

Important

End of support date: July 24, 2025.

For more information, see About version support.

Upgraded components​

The following components were upgraded in this minor release:

  • H2O Document AI v0.8.2

New improvements​

H2O Document AI​

  • Fixed conditional token labeling model evaluation.
  • Updated helm chart for the proxy to allow overriding scorer’s startup command. This can be used to install additional python libraries before scorer is started.

v24.07.1 | Aug 26, 2024​

Important

End of support date: July 24, 2025.

For more information, see About version support.

Upgraded components​

The following components were upgraded in this minor release:

Core components

  • H2O Driverless AI v1.10.7.1 - partial support, available upon request

Optional components

  • H2O AI Engine Manager v0.8.3
  • H2O Label Genie v0.4.3
  • H2O Document AI v0.8.1 with fixes for migration issues

New features​

H2O Driverless AI​

  • Enabled the following previously missing MLI explainers:
    • Friedman’s H-statistic
    • Absolute Permutation-Based Feature Importance
    • Relative Permutation-Based Feature Importance
  • Fixed an issue with the missing ufapt parameter during the refit process.
  • Added a new configuration option, snowflake_keycloak_broker_token_endpoint, corresponding to the environment variable DAI_SNOWFLAKE_KEYCLOAK_BROKER_TOKEN_ENDPOINT, for enhanced matching configuration.
  • Introduced the snowflake_keycloak_broker_token_type configuration option for the Snowflake connector, along with improvements to error messaging.
info

For a full list of features, improvements, and bug fixes in H2O Driverless AI, see the H2O Driverless AI v1.10.7.1 release notes.

H2O AI Engine Manager​

  • Added server-side apply to AIEM CRDs.
info

For a full list of features, improvements, and bug fixes in H2O AI Engine Manager, see the H2O AI Engine Manager v0.8.2 & v0.8.3 release notes.

H2O Label Genie​

  • Resolved critical security vulnerabilities.

H2O Document AI​

  • Fixed an issue related to migrating old H2O Document AI pipelines to a new version.
info

For a full list of features, improvements, and bug fixes in H2O Document AI, see the H2O Document AI v0.8.1 release notes.

Bug fixes​

  • Removed training GPU condition on the inference GPU pool.
  • Various MinIO-related fixes.
  • Fixed incorrect runtime name for the Python 3.10 GPU runtime.

v24.07.0 | July 25, 2024​

Important

End of support date: July 24, 2025.

For more information, see About version support.

New components​

  • H2O AuthZ v3.3.2
  • H2O Drive v2.0.2

Upgraded components​

The following components were upgraded in this major release:

Core components

  • AI App Store v0.36.4
  • H2O Cloud Discovery Service v2.3.3
  • H2O Driverless AI v1.10.6.2 & v1.10.7
  • H2O Enterprise Steam v1.9.11
  • H2O-3 v3.46.0.3 & v3.44.0.3
  • H2O MLOps v0.66.1
  • H2O Telemetry v1.2.5

Optional components

  • H2O AI Engine Manager v0.8.1
  • H2O Document AI v0.8.1
  • H2O eScorer v0.10.1
  • H2O Label Genie v0.4.2

New features​

AI App Store​

  • The following AI Engines settings are only visible for super administrators:
    • Starting and stopping AI Engines such as H2O Driverless AI, H2O Hydrogen Torch, H2O-3 via the AI Engine Manager.
    • Adjusting all configurable settings in H2O AI Cloud.
  • Resolved issues when installing some Python 3.12 packages by removing SETUPTOOLS_USE_DISTUTILS=stdlib for app launches, providing greater compatibility with existing Python packages.
  • Administrators of AI App Store are now allowed to set default values and constraints for H2O Driverless AI and H2O Engines.
  • Improved key:value pair editing in H2O AI Engine Manager and App Secrets Manager.
  • Improved error messaging on the CLI when app downloading is disabled.
  • Updated the end-user license agreement (EULA) to v04-18-2024.
  • The platformUsageEnabled variable was added to the helm chart. This variable allows platform admins to enable or disable the new Peak AI Unit Consumption UI page from the server config.
  • Users with Full Access or Visitors with the correct Visitor roles have access to run PUBLIC ON_DEMAND apps. For more information about user roles and access for H2O AI Cloud, see User roles in the H2O AI Cloud documentation.
  • Added the ability to copy app.toml code directly from the Admin Secrets UI.
info

For a full list of features, improvements, and bug fixes in AI App Store, see the AI App Store v0.34.0 - v0.36.0 release notes.

H2O AI Engine Manager​

  • H2O Engines can now be terminated to remain in the engines list.
  • H2O Engine now uses TERMINATING and TERMINATED states instead of PAUSING and PAUSED.
  • Added setting to turn DAIEngine Triton capability on/off.
  • Added h2o_engine_manager.__version__ to the Python client.
  • Added the following to the Admin UI:
    • DAI Engine version management
    • DAI Profile management
    • H2O Engine version management
info

For a full list of features, improvements, and bug fixes in H2O AI Engine Manager, see the H2O AI Engine Manager v0.8.0 & v0.8.1 release notes.

H2O Document AI​

  • H2O Document AI uses a single Helm chart to deploy all its components now. This chart uses a new namespace (document-ai). Scoring pipelines are deployed to document-ai-scorer namespaces, as it was the case earlier. Previously used document-ai-frontend, document-ai-viewer namespaces are now removed.
  • Added the ability to configure nodeSelector and tolerations for doc-proxy-scorer.
  • Implemented table support.
  • Implemented project collaboration between users using email invite.
  • Introduced the hybrid OCR processor to read PDF characters directly from documents and to extract text contained in images.
  • Implemented new pipeline backward compatibilities: Upgrading Document AI to a new verion (0.7 > 0.8) will cause all existing pipelines to automatically upgrade to the new version.
  • Updated Argus to v0.22.3.
  • Updated bulk scorer to v0.2.5.
  • Added a separate namespace for user workloads in Kubernetes.
  • Added SSL/SASL support for communication with Kafka.
  • Added gocloud.dev as the drop-in replacement for *sql.DB to handle DB connections from the API server.
  • Introduced pod disruption budgets for every deployment.
  • Added compatibility with pipelines published in previous versions of Document AI (v0.7.x).
  • Introduced the extra parameter to bulk scorer.
danger

There is a known issue with H2O Document AI pipelines failing to migrate. This will be resolved in the next release.

info

For a full list of features, improvements, and bug fixes in H2O Document AI, see the H2O Document AI v0.8.0 & v0.8.1 release notes.

H2O Driverless AI​

  • Added support for configuring a host name and port for the Snowflake connector when running Driverless AI in Snowpark Container Services.
  • Added recipe support for the Feature Store data connector.
  • Added a new configuration that lets users change the timeout duration when importing data from Hive, HDFS, JDBC, and kdb+ connectors.
  • Ability to navigate to the linked project from the experiment details page.
  • Added support for Python 3.12 in h2oai_client.
  • MLI:
    • Added the 2-D Partial Dependence explainer.
    • Added the Friedman H-statistic explainer.
    • Added PDP percentile plot.
    • Added ICE curves in Partial Dependence explainer at every decile of predicted probabilities. This gives an indication of local prediction behavior across the dataset.
  • Added a new configuration that lets users use their own service account when connecting to Google BigQuery (GBQ).
  • Added a new configuration that lets users optionally select which service account to impersonate for the Google BigQuery (GBQ) data connector.
  • Added a new configuration to control the experiment Leaderboard access globally for all users.
  • DAI now reports why certain explainers are not enabled in the MLI explainer drop-down list. This is usually limited by the experiment problem type, e.g., image, multinomial, etc.
  • Users are now warned when Shapley values are approximated in MLI.
info

For a full list of features, improvements, and bug fixes in H2O Driverless AI, see the H2O Driverless AI v1.10.6.2 & v1.10.7 release notes.

H2O eScorer​

  • Upgraded JDK to 17.
  • Upgraded Spring Boot to v3.1.9.
  • Upgraded Wave app to Wave SDK v1.0.2.
  • Models can be deployed to Remote eScorer Instances, and their stats can be monitored.
  • Python Client supports scoring Pandas DataFrames.
  • Model Registry can be filtered, sorted and searched.
  • Self-Signed Certificates can now be used with MLOps client.
  • Added initial LLM support.
  • Several bug fixes and improvements
info

For a full list of features, improvements, and bug fixes in H2O eScorer, see the H2O eScorer release notes.

H2O-3​

  • Added anomaly score metric to be used as a sort_by metric when sorting grid model performances for Isolation Forest with grid search.
  • Added weak_learner_params parameter for AdaBoost.
  • Added weak_learner="deep_learning" option for AdaBoost.
  • Implemented scoring and scoring history for Extended Isolation Forest by adding score_each_iteration and score_tree_interval.
  • Added support for Websockets to steam.jar.
info

For a full list of features, improvements, and bug fixes in H2O-3, see the H2O-3 v3.44.0.3 & v3.46.0.3 release notes.

H2O MLOps​

  • Added optional role-based access control (RBAC). You can now limit access to H2O MLOps APIs to users with specific roles.
  • Released Base Python Scorer v1.2.0 (BYOM).
  • Released Python-based runtimes v1.2.0 (BYOM).
  • Released H2O Hydrogen Torch runtime v1.2.0 (BYOM).
  • Released MLflow runtime v1.2.0 (BYOM).
  • Exposed authorization header with bear token through Env Vars in base scorer (BYOM).
  • You can now set configurable read time out in scorer proxy (Monitor Proxy).
  • Added deployer configurable monitor proxy timeout ()Deployer).
  • Upgraded base images to Java 17 eclipse-temurin:17.0.10_7-jre (Deployer).
  • Added the capability to disable model monitoring features when deploying H2O MLOps.
  • Support for FEDRAMP compliance.
  • Added an option to restrict model imports to specific types.
  • Added support for vLLM config model types.
  • When creating a deployment, added a deployment multi-issuer token security option.
  • The ListProjects API now returns all projects for admin users.
  • You can now set a specific timeout only for external registry import API.
  • You can now upload LLM experiments using the MLOps Wave app.
  • Added support for Authz user format.
  • Added support for DAI 1.10.7 and 1.10.6.2 runtimes.
  • Upgraded Rest scorer to Spring Boot 3 (1.2.0).
  • Added vLLM runtime support.
  • When creating a new deployment, there is now an option available to disable monitoring for the deployment.
  • Added validation for experiment file uploading.
  • Extended scoring API with new endpoint /model/media-score to support uploading multiple media files. The H2O Hydrogen Torch runtime is now supported with the ability to score image and audio files against the new endpoint /model/media-score.
  • The project page now includes an Events tab with pagination, search, and sorting.
  • You can now delete experiments.
  • Added pagination, search, sorting, and filtering by Tag on the Experiments page.
  • You can now update and delete tags. Note that tags can only be deleted if they are not associated with any entity.
  • The Create Deployment workflow now automatically populates K8s limits and requests with the suggested default settings.
  • The deployment state is now updated dynamically on the Deployments page.
  • Additional details about error deployment states are now displayed in the MLOps UI.
info

For a full list of features, improvements, and bug fixes in H2O MLOps, see the H2O MLOps v0.64.0 - v0.66.1 release notes.

Known issues and workarounds​

Known IssueWorkaround
MLOps error when upgrading to helm 0.16.0.Delete the <prefix-env>-mlops-environment-admin role
before upgrading the helm release:

kubectl -n mlops delete role/<prefix-env>-mlops-environment-admin
When manually setting the
SSL_CERT_FILE and REQUESTS_CA_BUNDLE to /etc/ssl/certs/appstore-ca-bundle.crt (where all wave apps set the ca cert),
the wave apps fail to install packages from PyPi.		1. Run the following commands (This only needs to be done once for a user.
If done before, go to step 2):

a. 
docker run -it --rm --entrypoint 
/bin/sh us-docker.pkg.dev/vorvan/dev/h2oai-appstore-runtimecpu39:v0.36.4
b. 
cat /etc/ssl/certs/ca-certificates.crt
c. Copy the contents and paste it in your ca-chain.crt

2. Add the following in
terraform/modules/applications/appstore/resources/google-values.yaml
and reapply terraform:

1. At line 43 add:
%{~ if enable_private_ca_certs ~}

  
caCertificateAppMountpoints:

    - 
/etc/ssl
/certs/appstore
-ca-bundle.crt
    - 
/etc/ssl
/certs/ca
-certificates.crt
  
%{~ endif ~}

2. At line 222 add:
%{~ 
if
 enable_private_ca_certs ~}
    - name: REQUESTS_CA_BUNDLE
      value: 
/etc/ssl
/certs/
ca-certificates.crt
    - name: SSL_CERT_FILE
      value: 
/etc/ssl
/certs/ca-certificates.crt
    %{~ endif ~}

resource "google_secret_manager_secret" "mlapi_admin_password" and
resource "google_secret_manager_secret_version" "mlapi_admin_password" are being created even when MLAPI Postgres is disabled.		Update the create_mlapi_postgres flag in
the terraform/modules/gcp-dependencies/secret-manager.tf file
(line 113 and 122) to the following condition:

count = var.enable_secret_management 
&& var.create_mlapi_postgres ? 1 : 0
Execution error when H2O Drive storage location variable is empty.Update terraform/modules/applications/gcp/locals.tf line 75 to the following:

drive_storage_location = local.install_drive && 
var.drive_storage_location == "" ? data.google_storage_bucket.drive_container[0].location : var.drive_storage_location
There is no flag controlling the deployment_server db in the /terraform/modules/azure-dependencies/postgres_common_server.tf, so it is always created.Control the count manually or add this line (along with associated variables) to
the terraform/modules/applications/azure/variables.tf file:

count = var.run_create_db_and_user_script && 
var.external_postgres_deployment? 1 : 0
When using the default MLOps deployment DB created in azure-dependencies, the connection string is not populated correctly in the Kubernetes secret when installing mlops_helm.
This bug will only affect you if you create the databases through the Terraform installer.		Set the variables like mlops_deployment_db_address,
mlops_deployment_db_name, etc., in
the terraform/modules/applications/azure file
When creating a deployment with the H2O Driverless AI Python Pipeline > Python Pipeline Scorer (DAI 1.10.7) in MLOps, the deployment pod fails with a
Read-only file system: './tmp error		Add the following to the terraform/modules/applications/mlops-helm/resources/values.yaml file at line 483:
writableVolume:
 
true

        
envs:

          
H2O_SCORER_LOADER_PATH:
 
"dai_pipeline_scorer.driverlessai:Loader"

          
H2O_SCORER_PORT:
 
8080

          
H2O_SCORER_IGNORE_SIGNAL_HANDLERS:
 
true

v23.10.3 | October 22, 2024​

Important

End of support date: Jan 23, 2025.

For more information, see About version support.

Upgraded components​

The following components were upgraded in this minor release:

  • H2O Document AI v0.7.3

Fixed issues​

H2O Document AI​

  • Fixed an issue where processor failed when all input pages were filtered out by the when miniprogram.
  • Fixed an issue where published conditional pipelines appeared stuck in a pending state.

v23.10.2 | April 05, 2024​

Important

End of support date: Jan 23, 2025.

For more information, see About version support.

Upgraded components​

The following components were upgraded in this minor release:

  • H2O MLOps v0.62.5.

New features​

H2O MLOps​

  • Added support for enabling health monitoring without drift. You can now enable Health monitoring (prediction volume and scoring latency) without Drift monitoring (which is dependent on raw data). This means that there are three total possible monitoring settings for a given deployment in H2O MLOps:
    • monitoring disabled
    • monitoring enabled for health only (no raw input data stored)
    • monitoring enabled for health and drift (raw input data stored)
      info

      H2O MLOps deployments will start with model monitoring disabled by default.

  • Added the ability to disable storing score transaction data in monitoring via the Deployer API
info

For a full list of features, improvements, and bug fixes in H2O MLOps, see the H2O MLOps v0.62.5 release notes.

H2O Enterprise Steam​

  • Fixed a bug where the platform host name was not propagated to H2O Cloud Discovery annotations.

AI App Store​

  • Fixed a bug with the custom image regex that was set for AI App Store.

Known issues and workarounds​

Known IssueWorkaround
Timeout during helm install for H2O Notebook application when using the GKE filestore, due to PVC creation delay.Add timeout = 600 to line 66 in terraform/modules/applications/notebook/notebook.tf.

v23.10.1 | March 15, 2024​

Important

End of support date: Jan 23, 2025.

For more information, see About version support.

Upgraded components​

The following components were upgraded in this minor release:

Core components​

  • H2O MLOps v0.62.4

Optional components​

  • H2O Document AI v0.7.2

New features​

Platform installer​

  • H2O Document AI and H2O MLAPI are now enabled separately on the installer. To enable both H2O Document AI and H2O MLAPI, set optional_applications_to_install = ["docai", "mlapi"].

H2O MLOps​

Frontend

  • Mitigating against potential XSS attacks.
    • Content Security Headers (CSP) are set on the frontend to minimise potential XSS attacks fetched through any untrusted sources
    • Sanitised all plain text inputs (name, description) to minimise potential XSS attacks.
    • Validated other text-based fields to avoid accepting XSS vulnerable characters (. < > " ') as the user input.
    • Sanitised all text-based downstream response fields (name, description) to minimise potential XSS attacks.
  • Introduced access control based on JWT claims. This means you can now control access to the H2O MLOps frontend UI based on user role.

Storage

  • The CreateArtifact API and UploadArtifact API now validates against a known list of mime types to allow or reject an artifact creation/upload respectively. The default configuration is to allow all mime types.
  • All user text inputs are escaped and stored within the database to avoid potential XSS issues.

Deployer

  • All user text inputs are escaped and stored within the component to avoid potential XSS issues.
info

For a full list of features, improvements, and bug fixes in H2O MLOps, see the H2O MLOps release notes.

H2O Document AI​

  • Added a new proxy version and updated the helm chart to reflect the new proxy URL.
note
  • If LinkerD is enabled, restart your H2O DocAI scorers after the upgrade. Otherwise the scorers may not work.
  • To enable LinkerD for the H2O DocAI viewer, add the LinkerD annotation (linkerd.io/inject: enabled) annotations on the api.podAnnotations and ui.podAnnotations sections. Do not annotate the document-ai-viewer namespace with the LinkerD annotation as archiveExtract jobs do not support the LinkerD injection yet.

Known issues and workarounds​

Known IssueWorkaround
Timeout during helm install of the H2O Notebook application when using the GKE filestore, due to PVC creation delay.Add timeout = 600 to line 66 in terraform/modules/applications/notebook/notebook.tf.
MLOps model upload fails after TF migration and fresh installs due to a UID mismatch on the pod and PVC.Add the following code snippet to line 609 in terraform/modules/applications/mlops-helm/resources/values.yaml. 

containerSecurityContext:
  # -- Security context for the main containers.
  enabled: true
  runAsNonRoot: true
  readOnlyRootFilesystem: false
  allowPrivilegeEscalation: false
  seccompProfile:
    type: "RuntimeDefault"
  capabilities:
    drop:
      - ALL
podSecurityContext:
  # -- Whether a pod security context is applied to each pod 
deployed in the environment.
  enabled: true
  runAsNonRoot: true
  runAsUser: 1001
  runAsGroup: 1001
  fsGroup: 1001

v23.10.0 | Jan 23, 2024​

Important

End of support date: Jan 23, 2025.

For more information, see About version support.

New components​

The following components are optional components of new installations of HAIC.

  • AI Notebooks v0.1.10
  • H2O eScorer v0.7.0
  • Enterprise h2oGPTe
note

Let your H2O contact know if you would like to install these applications into your AI App Store. Enterprise h2OGPTe is not yet a part of H2O AI Cloud, however, it can be installed within the same cluster.

Upgraded components​

Core components​

  • H2O Driverless AI v1.10.6.2
  • H2O MLOps v0.62.2
  • Enterprise Steam v1.9.7
  • AI App Store v0.29.1
  • H2O-3 v3.44.0.2
  • H2O Cloud Discovery Service v1.2.6
  • H2O Logging Service v0.1.4

Optional components​

Optional components of HAIC are provided to customers as needed based on your specific requirements. If you have any of the following components, note the following upgrades.

  • H2O Hydrogen Torch v1.3.3
  • H2O Document AI v0.7.0
  • H2O Model Validation v0.17.0
  • H2O Feature Store v0.19.3
  • H2O AI Engine Manager v0.5.5
  • H2O Autodoc v0.9.0
  • H2O Label Genie v0.4.1

New features​

Platform​

  • Added Private CA certificate support for H2O AI Hybrid Cloud runtimes. This means that H2O AI Hybrid Cloud now supports using certificates generated by a private certificate authority.
  • Supports backup and restore using Velero.
  • H2O AI Hybrid cloud now adopts a standardized image naming scheme for H2O MLOps DB servers that are deployed outside of the Kubernetes cluster. For example,
    • Old naming convention: <image-registry>/artifact-fetcher:0.62.2
    • New naming convention: <image-registry>/h2oai-modeldeployment-artifactfetcher:v0.62.2
  • Upgraded Postgres to v14.8
  • Compatibility up to Kubernetes v1.28
  • Deprecated the following AI App Store runtimes:
    • ub1804_cuda110_cudnn8_py37_wlatest: Python 3.7 GPU Runtime
    • deb10_py37_wlatest: Python 3.7 CPU Runtime.
  • The new default AI App Store runtime is deb11_py310_wlatest (Python 3.10 CPU Runtime).

H2O Driverless AI​

  • Added integration with h2oGPT that lets you optionally generate and view dataset or experiment summaries using a GPT model. For more information, see h2oGPT integration.
  • Added support for Okta SSO authenticator on the Snowflake connector.
  • Added support for custom telemetry recipes.
  • Added support for a PyTorch backend in Triton server.
  • Added support for sharing image datasets to H2O Storage. For more information, see H2O Storage (remote storage) integration.
  • You can now tag experiments inside Projects that are connected to H2O Storage (remote storage). These tags are also displayed in H2O MLOps. For more information, see Experiment tagging.
  • DAI Model Dashboard: You can now view a new DAI model dashboard that provides comprehensive insights into the performance of models created using DAI. For more information, see MLI Dashboard.
  • Performance Charts: You can now view performance charts specifically designed for Decision Tree (DT) and Random Forest (RF) surrogate models.
  • NLP LOCO Scoring Pipeline: Added the ability to construct a NLP LOCO scoring pipeline.
  • The Python client now supports the generation of plots for individual explainers. Supported explainers include:
    • Partial Dependence Plot
    • Shapley Summary Plot for Original Features (Naive Shapley Method)
    • Shapley Values for Transformed Features
    • Shapley Values for Original Features (Naive Method)
    • Surrogate Decision Tree
  • Added new configuations for the Google BigQuery (GBQ) data connector.
info

For a full list of features, improvements, and bug fixes in H2O Driverless AI, see the H2O Driverless AI v1.10.5.1, v1.10.6, v1.10.6.1, & v1.10.6.2 release notes.

H2O MLOps​

  • For GPU-enabled model deployments, you can now set the appropriate Kubernetes (K8s) requests and limits by clicking the GPU Deployment toggle when creating a deployment. For more information, see Deploy a model and Kubernetes options.
  • You can now create and assign experiment tags within a project. For more information, see Project page tabs and Add experiments.
  • You can now edit the names and tags of experiments. For more information, see Project page tabs.
  • Added support for up to H2O Driverless AI v1.10.6.1 runtime
  • The ListExperiments API can now be used to filter experiments by status (ACTIVE, DELETED). By default, the API returns ACTIVE experiments.
info

For a full list of features, improvements, and bug fixes in H2O MLOps, see the H2O MLOps v0.62.0 release notes.

H2O Enterprise Steam​

  • Helm: Added annotations and labels for H2O Cloud integration.
  • Helm: Added support to override the namespace where the chart is deployed to.
  • H2O Kubernetes: Added option to specify extra environmental variables.
  • Configuration: Added deprecation mode in the licensing configuration.

AI App Store​

  • Added support of affinity, nodeSelector, tolerations, and securityContext.
  • Added Python CPU 3.9 and 3.10 runtimes.
  • Users can now configure the timeout duration for bundle upload.
  • Users can now configure the database connection limits.
  • Screenshots can now be sorted in lexicographic order.
  • Added the ability to emit telemetry login events on token session exchange.
  • Implemented a check for custom image for federated container apps.
  • Added the h2o admin app import cli command to enable importing apps
  • Added the h2o app import --set-image flag for setting a container image for both regular and admin users
  • Added h2o app list --precondition for filtering apps based on their precondition status checks
  • Updated the user flow for creating AI Engines.
  • Added support for the node count field for H2O3 Engines
  • Added support for searching AI Engines
  • Added the Platform Usage Page and the PlatformUsageEnabled config option for enabling the Platform Usage Page.
  • Added Admin secret management
  • Added a Runtime.AppMode container for launching docker images as apps
  • The Precondition Checker API validates whether an app is runnable and scans apps periodically for validation.
  • Added the requireRuntimeVersion config option to force apps to have a configured runtime version when importing
  • Disabled the AI Unit scanning by default in preparation for the transition to telemetry service
  • Added precondition checks for Apps to detect deprecated runtime versions
info

For a full list of features, improvements, and bug fixes in AI App Store, see the AI App Store v0.23.0, v0.24.1, v0.25.0, v0.25.1, v0.25.2, v0.25.3, v0.26.0, v0.27.0, v0.28.0, v0.28.3, v0.28.4, v0.28.6, & v0.29.1 release notes.

H2O-3​

  • Added ability to publish models to MLOps via Python API.
  • Added ability to grid over Infogram.
  • Implemented Regression Influence Diagnostics for GLM.
  • Enhanced GBM procedures to output which records are used for each tree.
  • Added learning curve plot to H2O’s Explainability.
  • Added save_plot_path parameter for Fairness plotting allowing you to save plots.
  • Added official support for Python 3.9, 3.10, and 3.11.
  • Implemented AIC metric for all GLM model families.
  • Implemented Tweedie variance power maximum likelihood estimation for GLM.
  • Added ability to convert H2OAssembly to a MOJO2 artifact.
  • Implemented new Decision Tree algorithm.
  • Enabled H2OFrame to pandas DataFrame using multi-thread from datatable to speed-up the conversion process.
  • Added support for EMR 6.10.
  • Implemented new write_checksum parameter that allows you to disable default Hadoop Parquet writer systematically writing a .crc checksum file for each written data file.
  • Implemented make_metrics with custom AUUC thresholds, and MOJO support for UpliftDRF.
  • Implemented custom metrics for:
    • AutoML
    • Stacked Ensemble
    • Deep Learning
    • UpliftDRF
    • leaderboard
  • Implemented new AdaBoost algorithm for binary classification.
  • Implemented Shapley values support for ensemble models.
info

For a full list of features, improvements, and bug fixes in H2O-3, see the H2O-3 v3.40.0.1, v3.40.0.2, v3.40.0.4, v3.42.0.1, v3.42.0.2, v3.42.0.3, v3.42.0.4, v3.44.0.1, & v3.44.0.2 release notes.

H2O Hydrogen Torch​

  • A new data connector for Google Cloud Storage is now available.
  • Supports air-gapped environments without internet access.
  • Added MLAPI support which enables you to run Hydrogen Torch UI on a CPU instance and send GPU jobs (train experiment, predict experiment) through the API to MLAPI. This is beneficial for a number of reasons including saving GPU resources.
  • Added support for .MP3 audios in MLOPs deployments
info

For a full list of features, improvements, and bug fixes in H2O Hydrogen Torch, see the H2O Hydrogen Torch release notes.

H2O Feature Store​

  • Internal database used to store meta-data was changed from Mongo to Postgres and MongoDB collection data source introduced
  • Introduction of project history
  • Integration with H2O AI Cloud Discovery Service
  • Introduce APIs and for the following:
    • Pause and resume scheduled ingest tasks
    • Cancel a job and improve handing of cancelled jobs
    • Download a pre-generated notebook demonstrating retrieve flow
    • Upload and download artifacts to a specific feature set
    • Mark/unmark feature as target variable
    • View popular feature sets
    • View recent projects and recent feature sets
  • Ability to create feature sets, delete major feature set versions, and order projects, feature sets, or features via the GUI
  • Ability to list jobs, filter jobs based on type, and see progress of jobs via the GUI
  • Ability to create, list and revoke personal access tokens in the UI
  • Introduced approval process in CLIs and backend
  • Added support for LinkerD
  • Expose monitoring and custom data on feature schema
  • Azure Gen2 Jar is now published to maven central
  • Introduce feature set flow configuration - user can configure synchronization between online and offline stores
  • Integrate with H2O AI Cloud LoggingService
  • IAM support for Redis
  • Support for passing security context for containers
  • Ability to download pre-generated retrieve notebook via CLI and UI
  • Implemented review process in the UI
  • Ability to ingest and retrieve from UI
  • Ability for Feature Store administrator to specify maximum duration of a personal access token
  • Ability to pass affinity specification to Feature Store pods via Helm
  • Ability to obtain JDBC connection string for core PostgreSQL database from existing secret
  • Ability for namespace override in Helm
  • Add telemetry.cloud.h2o.ai/include: true annotation to Spark driver and executors
  • Add ownership attribution labels/annotations to feature store resources
info

For a full list of features, improvements, and bug fixes in H2O Feature Store, see the H2O Feature Store v0.14.3, v0.14.4, v0.15.0, v0.16.0, v0.17.0, v0.18.0, v0.18.1, v0.19.0, v0.19.1, v0.19.2, & v0.19.3 release notes.

H2O Document AI​

  • Introduced the universal scoring pipeline.
  • Introduced ability to automatically purge training artifacts.
  • Introduced ability to schedule the deletion of your whole project and all of its resources.
  • Introduced new base models for training a model in Publisher.
  • Introduced learning rate for model training in Publisher.
  • Implemented role-based access control (RBAC) to Document AI on HAIC.
info

For a full list of features, improvements, and bug fixes in H2O Document AI, see the H2O Document AI v0.7.0 release notes.

H2O Model Validation​

  • Added support for the following tests:
    • Calibration score
    • Segment performance
    • Robustness
  • Ability to generate insights for a validation test or dataset after defining a large language model (LLM) source.
  • Manage all validation tests within the Tests card.
  • Supports connections to H2O AI Engine Manager v0.5.5 or a previous version and Driverless AI (DAI) version 1.10.6.2 or a previous version.
  • Does not allow deleting models or datasets that are linked to an existing test.
  • Removed the Worker feature
  • Improvements to drift detection, segment performance tests, adversarial similarity tests, and size dependency tests.
  • Ability to view pairwise correlations between numerical columns of a dataset.
info

For a full list of features, improvements, and bug fixes in H2O Model Validation, see the H2O Model Validation v0.16.0 release notes & v0.17.0 release notes.

H2O AI Engine Manager​

  • Enabled WebSockets for DAIEngine, which resolves issues with using H2O Driverless AI Wizards.
  • Added the GPUTolerations field to DAISetup and H2OSetup.
  • Migration tool now accepts the H2O Enterprise Steam admin username and password as an alternative to PAT (Personal Access Token).
  • Added a CONNECTING state to DAIEngine.
  • Basic integration with H2O Logging Service.
  • Added registration with H2O Cloud Discovery Service.
  • Published the Python client on PyPI.
  • DAIEngine: Added support for Triton in 1.10.5.
  • DAIEngine: The engine owner is added to the local administrator list.
  • DAIEngine: The home folder is now in a writeable location and persisted across restarts.
  • DAIEngine: All entities are migrated to the engine owner during startup (>= 1.10.5).
  • DAIEngine: Added API method to change ownership of an engine and migrate the DAI entities.
  • DAIEngine: Increased the network request timeout to engines to 24 hours.
  • DAIEngine: Ensure that the engine fails when any container exits with a non-zero exit code.
  • DAIEngine: Use DAI idle indicator instead of looking at running experiments.
  • DAIEngine: Set proper override_virtual_cores and virtual_cores_per_physical_core values.
  • DAIEngine: Allow pausing engines while in CONNECTING state.
  • DAISetup: Added MaxUnusedDuration, which, when set deletes the engines when they are paused for too long.
  • H2OEngine: Added support for H2O 3.40.
  • H2OEngine: Added support for multinode clusters.
  • H2OEngine: Added support for external XGBoost.
  • Engine: Exposed current idle and running durations.
  • Engine: Added advanced filtering to support concurrent searching and filtering functionality.
  • Python: When creating an engine, the engine_id argument is no longer mandatory and will be generated if not provided.
  • Migration: Change ownership of all files to a known user during migration.
  • Helm: Added configurable Pod security context.
  • Logging: Added logging of manager events into LoggingService.
  • Logging: Added promtail sidecar for logging data from DAI filesystem into LoggingService.
  • Logging: Added creator label to scraped logs.
  • Python: Allow connecting only to running engines.
  • Telemetry: Added telemetry usage labels to DAIEngine and H2OEngine pods.
  • Added configurable SSL cert verification for the Python client.
  • Added support for Steam migrator filtering by providing a list of instances to be included or excluded from migration.
  • H2OEngine: Added API for automatic sizing.
  • Added API for DAIVersion and H2OVersion.
  • Added API for DAIProfiles.
  • Added super admin role for global resources.
  • Enabled seamless Python client login.

H2O AutoDoc​

  • Added the ability to create an AutoDoc for an H2O Driverless AI experiment in AI Engine Manager
  • The default runtime for H2O AutoDoc has been set to Python 3.8

H2O Label Genie​

  • Ability to import datasets from H2O Drive and download/export already approved samples to H2O Drive.
  • Supports multi-label for all classification annotation tasks.
  • Supports the following new annotation tasks:
    • Instance segmentation
    • Text summarization
    • Text-generative AI
  • h2oGPT and OpenAI LLMs are available for text summarization and text-generative AI annotation tasks.
  • Added several keyboard shortcuts that enable you to speed up the annotation process of a dataset. For more information, see Hotkeys
  • Added box to polygon feature that enables you to improve the efficiency of manual labeling for an image instance segmentation task.
info

For a full list of features, improvements, and bug fixes in H2O Label Genie, see the H2O Label Genie v0.3.0 release notes and v.0.4.0 release notes.

Known issues and workarounds​

Known IssueWorkaround
When adding a file in Edit in Page View on H2O Document AI, the changes are not reflected on the Annotation Set list view.Refresh the page or click on another menu item (e.g. Document Sets) and
click back to annotation sets to see the updated number of pages.
When adding a file in Edit in Page View of an Annotation Set on H2O Document AI, it only supports image files. JSON files are not supported.Export the .pdf as a .jpg or .png and then add the file in edit page view.
After adding a file in Edit in Page View in the H2O Document AI publisher, if you leave edit in page view and then return to it again, the added file does not show up.Currently, no workaround available.
H2O Document AI pipeline creation fails on OpenShift when Linkerd is enabled.Exclude port 443 in document-ai-viewer namespace by annotating the namespace with 
config.linkerd.io/skip-outbound-ports: "443"
When using the Edit in Page View in the H2O Document AI publisher on the Apple Safari browser, the first label gets pre-populated resulting in the bounding box not being labeled as such.This issue does not occur when using Google Chrome.
Keycloak pods are not started by default when Linkerd is enabled by default.Keycloak pods are not injected to the Linkerd proxy for fresh installs even
though the Linkerd inject is enabled for the Keycloak namespace.
Restart the Keycloak pod after Linkerd pods come up in your Kubernetes cluster.
Variable gcp_region does not take affect for Kubernetes module installation.Add the following line in the file terraform/templates/kubernetes/main.tf,
under the module kubernetes_gcp{}. 
gcp_region = var.gcp_region
H2O Document AI proxy ships a network policy that blocks Egress traffic from post-processor pods. If your H2O AI Hybrid Cloud environment has Calico (or any CNI that enforces NetworkPolicy) + Linkerd enabled, the custom post processor pod fails after 120 seconds.If Linkerd is enabled, delete the networkpolicy in the
document-ai-scorer namespace after deployment.
Alternatively, you can also update the helm flag as follows. 
publish: 
networkPolicy: 
enabled: false
or
publish.networkPolicy.enabled=false
Unable to start H2O-3 in H2O AI Engine Manager when Linkerd is enabled by default.Disable Linkerd in H2O AI Engine Manager. 
linkerd_aiem_manager_annotations  = { 
       "linkerd.io/inject"  =  "disabled" 
}
H2O Driverless AI versions prior to v1.10.4.2 are inaccessible post-migration to private CAUpgrade H2O Driverless AI to v1.10.4.2 or a later version.
Permission issue in container resulted in Minio job that creates the Minio bucket failing on OpenShift.Create the buckets manually
The Enterprise Steam setup fails to run on airgapped environments as it requires creating a virtual environment and installing the Steam client.Configure Enterprise Steam on the UI manually.
The mlops_helm installation uses the wrong code to get db_admin_password/db_name/db_address for deployment of PostgreSQL Server.Update lines 54, 59 and 129 on the modules/applications/aws/data.tf file
to use var.external_postgres_deployment variable instead of var.external_postgres.
When running the deployments-tf-scale-down.sh or deployments-tf-scale-up.sh scripts during the upgrade, the script fails in the middle.Replace lines 24-26 of the relevant script with the following code snippet: 
optional_components=( 
    'ui' 
    'externalregistry' 
    'prometheus-server' 
) 
 
for oc in "${optional_components 
[ 
@ 
] 
}"; do 
  if kubectl -n "$ 
ns" get deployment " 
${prefix}-${oc}" >/dev/null 2> 
& 
1; then 
    components+=("${oc}") 
  fi 
done
The variable name used to define the H2O MLOps monitoring db name is mlops_monitoring_db but in dependencies it is named as mlops_monitoring_db_name.
The same variable name should be used to avoid any issues.		Include both variables in the config.tfvars file to avoid any issues
Restoring the external-registry-adapter fails.Update the following variables with the following values in templates and modules:
  • mlops_monitoring_db_username = "mlops_model_monitoring"
  • mlops_deployment_db_username = "deployment_server"
  • mlops_db_username = "h2oai_storage"
  • mlops_deployment_db_name = "deployment_server"
  • mlops_monitoring_db = "mlops_model_monitoring"
  • mlops_db_name = "h2oai_storage"
The image pull secret does not get created in the feature-store-redis namespace due to the namespace variable being empty.Include featurestore_redis_namespace = "feature-store-redis" in the config.tfvars file or update templates & modules to set feature-store-redis to the variable.

v23.01.2 | Aug 23, 2023​

Important

End of support date: April 14, 2024.

For more information, see About version support.

Upgraded components​

Optional components​

Optional components of HAIC are provided to customers as needed based on your specific requirements. If you have any of the following components, note the following upgrades.

  • H2O Document AI v0.6.2

New features​

  • Added the ability to update or add new templates in H2O Document AI

Bug fixes​

  • Fixed an issue in H2O Document AI where the logic used to pull the feature store web proxy image did not work if the image registry is empty.
  • Made an improvement to include input_dir content in the request going to the custom post-processor deployment.

Known issues and workarounds​

Known IssueWorkaround
Deploying an H2O Document AI pipeline with a long name fails.To avoid this issue, use pipeline names that are as short as possible (preferably 3-4 characters) in H2O Document AI. In a future release, the issue will be fixed by decoupling the pipeline resource names from the name provided by the customer.
After publishing a pipeline using a model trained on one file on H2O Document AI Viewer, the document will show zero results.This has more to do with the quality of the model trained. If you try to score with the same file used for training then it will probably, though not guaranteed, find at least one entity.
Additionally, it also depends on how the bounding box was created. If the bounding box is very small or useless, then it may result in this situation.
H2O Document AI proxy ships a network policy that blocks Egress traffic from post-processor pods. If your H2O AI Hybrid Cloud environment has Calico (or any CNI that enforces NetworkPolicy) + Linkerd enabled, the custom post processor pod fails after 120 seconds.If Linkerd is enabled, delete the networkpolicy in the document-ai-scorer namespace after deployment. Alternatively, you can also update the helm flag as follows.
publish:
networkPolicy:
enabled: false

or publish.networkPolicy.enabled=false

v23.01.1 | Jul 28, 2023​

Important

End of support date: April 14, 2024

For more information, see About version support.

New components​

H2O LLM Studio is an optional component of new installations of HAIC.

note

Let your H2O contact know if you would like to install this beta application into your AI App Store.

Upgraded components​

Core components​

  • H2O Driverless AI v1.10.5
  • H2O MLOps v0.61.1
  • Enterprise Steam v1.9.5

Optional components​

Optional components of HAIC are provided to customers as needed based on your specific requirements. If you have any of the following components, note the following upgrades.

  • H2O Hydrogen Torch v1.3.0
  • H2O Document AI v0.6.1
  • H2O AutoInsights v0.7.3
  • H2O Model Analyzer v0.7.2
  • H2O Model Validation v0.15.2

New features​

Platform installer

  • Upgraded Amazon EBS CSI from v2.4.1 to v2.17.1
  • Added metrics server to Amazon EKS
  • Added support for Linkerd on Azure and OpenShift
  • Added S3 for AI App Store on OpenShift
  • Replaced the MLOps legacy UI with the Wave UI
  • Upgraded Amazon RDS versions to v11.20 on AWS
  • Telemetry is upgraded to v0.25
  • Compatibility up to Kubernetes v1.26

H2O Driverless AI

  • Added support for NVIDIA Triton Inference Server (disabled by default and should remain disabled for H2O AI Hybrid Cloud Openshift customers)
  • Several improvements to the Experiment Wizard
  • Added an Experiment Results Wizard (beta) and an Experiment Comparison Wizard
  • Added CPU memory usage of C++ MOJO to experiment summary and to Deploy Wizard.
  • Added a page that lets admin users view system logs.
  • Added MLI python client along with explanatory videos for MLI explainers
  • Added support to transform a dataset with the experiment’s fitted pipeline
  • Numerous bug fixes
  • Several UI/UX and performance improvements
    info

    For a full list of features, improvements, and bug fixes in H2O Driverless AI, see the H2O Driverless AI v1.10.5 release notes.

H2O MLOps

  • Introduced a feature flag to enable the import third-party experiments (pickled experiments) flow with Conda.
  • Users can now create A/B Test and Champion/Challenger deployments through the UI.
  • Users can now create and view configurable scoring endpoints through the UI.
  • Concurrent Scoring Requests are now supported for Python-based Scorers. Scoring times for C++ MOJO, Scoring Pipeline, and MLflow types now support parallelization with the default degree of parallelization set to 2.
  • Support for H2O3 MLflow Flavors and import of MLflow wrapped H2O3 models.
  • Added support for Kubernetes v1.26
  • Added full support for DAI v1.10.5
  • Added support for the H2O Cloud Discovery service.
  • Added loglevel propagation for the deployment and scoring pipeline and increased the default loglevel from DEBUG/TRACE to INFO.
  • Bug fixes to the deployment pipeline, monitoring, and drift detection.
  • Add mutually exclusive lock protection when multiple update deployment requests are received against the same target deployment.
  • The standalone version of the wave app is now fully air-gap compatible.
    info

    For a full list of features, improvements, and bug fixes in H2O MLOps, see the MLOps v0.60.1 & v0.61.1 release notes.

H2O Hydrogen Torch

  • Added support for the following problem types:
    • Speech recognition
    • 3D image classification
    • 3D image regression
    • 3D image semantic segmentation
  • Users can now deploy built models to H2O MLOps directly from the H2O Hydrogen Torch UI.
  • The .whl package of a Python scoring pipeline now includes a Dockerfile that enables users to build a dedicated Docker image that can include all requirements to run the scoring pipeline for model inference (production).
  • The H2O Hydrogen Torch landing page now has a new design that provides an array of statistics and facts about the instance of the application.
  • Several new UX improvements to improve user experience.
    info

    For a full list of features, improvements, and bug fixes in H2O Hydrogen Torch, see the Hydrogen Torch v1.3.0 release notes.

H2O Document AI

  • Introduced H2O Document AI - Viewer for business users to score documents on built pipelines.
  • Introduced initial telemetry integration.
  • Added the ability to score PDFs with page ranges.
  • Added support for Kubernetes 1.26.
  • Updated telemetry implementation to make scored documents more efficiently retrieved. ::info For a full list of features, improvements, and bug fixes in H2O Document AI, see the Document AI v0.6 release notes. :::

H2O AutoInsights

  • Support for basic Treemap chart for hierarchical category analysis
  • Enable ability to download flagged anomalies (Multivariate)

H2O Model Analyzer

  • DAI 2023 License update
  • Minor UI/UX improvements

Enterprise Steam

  • Kubernetes: Added support for Kubernetes 1.26
  • Kubernetes: Configurable seccomp profile
  • Helm: Fixed support for custom CA certificates
  • Helm: Removed unnecessary RBAC permissions
  • Added support for the H2O Cloud Discovery Service
  • H2O Driverless AI: Added support for Driverless AI 1.10.5
  • H2O Driverless AI: Automatic entity migration when changing the ownership on an instance
  • H2O Driverless AI: Added option to toggle Triton (and CAP_SYS_NICE capability) on/off
  • Python: Reduced excessive logging of client requests
  • Hadoop: Added support for Hadoop 3.3
    info

    For a full list of features, improvements, and bug fixes in Enterprise Steam, see the Enterprise Steam v1.9.1, v1.9.2, v1.9.3, v1.9.4, & v1.9.5 release notes.

Known issues and workarounds​

Known IssueWorkaround
H2O Document AI pipeline creation fails on OpenShift when Linkerd is enabled.Exclude port 443 in document-ai-viewer namespace by annotating the namespace with config.linkerd.io/skip-outbound-ports: "443".
Keycloak pods are not started by default when Linkerd is enabled by default.Keycloak pods are not injected to the Linkerd proxy for fresh installs even though the Linkerd inject is enabled for the Keycloak namespace. Restart the Keycloak pod after Linkerd pods come up in your Kubernetes cluster.
Variable gcp_region does not take affect for Kubernetes module installation.Add the following line in the file terraform/templates/kubernetes/main.tf, under the module kubernetes_gcp{}.

gcp_region = var.gcp_region
Typo in the image name for H2O DocumentAI Viewer.Set a value for image_registry on your bundle.
Unable to start H2O DriverlessAI wizard from AI Engine Manager.Fixed in AI Engine Manager v0.2.3. Optionally, you can use Enterprise Steam if the nitro wizard is required.
Deploying an H2O Document AI pipeline with a long name fails.To avoid this issue, use pipeline names that are as short as possible (preferably 2-3 characters) in H2O Document AI. In a future release, the issue will be fixed by decoupling the pipeline resource names from the name provided by the customer.
When adding a file to an annotation set on H2O Document AI publisher in Edit in Page View, the number of documents and pages may not get properly updated.Refresh the page, or click on another menu item (e.g. Document Sets) and then click back to Annotation Sets to see the updated number of pages.
Only image files are supported when adding a file in Edit in Page View in the H2O Document AI publisher.To add a file that is not an image, first export the PDF as a JPG or PNG. Then, add the file in Edit in Page View.
After adding a file in Edit in Page View in the H2O Document AI publisher, if you leave edit in page view and then return to it again, the added file does not show up.Currently, no workaround available.
When using the Edit in Page View in the H2O Document AI publisher on the Apple Safari browser, the first label gets pre-populated resulting in the bounding box not being labeled as such.This issue does not occur when using Google Chrome.
If the image registry is empty, the logic used to pull feature store web proxy image will not work.Change line 11 of the file terraform/modules/applications/featurestore/locals.tf to the following:

featurestore_web_proxy_image = var.image_registry != "" ? "${var.image_registry}/${var.featurestore_web_proxy_image}" : "gcr.io/vorvan/h2oai/${var.featurestore_web_proxy_image}"

v23.01.0 | Apr 14, 2023​

Important

End of support date: April 14, 2024

For more information, see About version support.

New components​

AI Engine Manager (AIEM) is an optional component of new installations of HAIC, and is recommended if you are using any of the following product versions:

  • H2O Driverless AI v1.10.4 or a later version
  • H2O-3 v3.38.0.4, v3.38.0.3, or v3.36.1.5

However, if you are currently using, or wish to use an older version of H2O Driverless AI or H2O-3, it is recommended to use Enterprise Steam instead of AI Engine Manager.

The goal of AIEM is to be the central service integrating with Kubernetes for model building tools like H2O Driverless AI and H2O-3, enabling other teams to focus on making the best ML products instead of managing Kubernetes resources. AIEM also features an interface found directly within the HAIC cloud interface to improve user flow.

Upgraded components​

Core components​

  • AI App Store v0.22.0
  • H2O Driverless AI v1.10.4.2 & v1.10.4.3
  • H2O-3 v3.38.0.3 & v3.38.0.4
  • H2O MLOps v0.59.0, 0.59.1, & v0.60.1
  • Enterprise Steam v1.9.0

Optional components​

Optional components of HAIC are provided to customers as needed based on your specific requirements.

  • H2O Feature Store v0.14.4 is currently only available as a preview for H2O Hybrid Cloud users on AWS and Azure.
  • H2O Document AI v0.5.0 provides automated document processing and extracts insights from text, images, and tables. For more information, see the H2O Document AI release notes.
  • H2O Hydrogen Torch v1.2.0 is an application for training and deployment of deep learning models.

New features​

AI App Store

  • Improvements to the 'My Apps' page:
    • Added support for Python Apps
    • Easily manage apps from the UI
    • Find the app you’re looking for more easily
    • Easily copy the UUID
  • Improvements to the 'My Instances' page:
    • Find instances more easily
    • Quickly find all versions of the exact app you’re seeking
    • Enhanced app filtering
  • Improvements to the 'Admin Apps' page:
    • Find the app you’re looking for more easily
    • Easily copy the UUID
  • Improvements to the 'Admin Instances' page:
    • Sort app instances in the admin instances UI
    • Improve scalability of app instances

Document AI

  • Added optical character recognition (OCR) language support for:
    • Latin (e.g. Spanish)
    • Arabic (e.g. Persian)
  • Added Document Text Recognition (DocTR) EfficientNet models to better recognize handwritten documents
  • Added ability to set batch size and number of epochs for model training
  • Upgraded the ML API to v0.4.0
  • Refactored and improved the training user interface for better usability
  • Added role based access control (RBAC) for Doc AI visibility in HAIC to gate access to H2O Document AI based on a user's role
  • Added command-line bulk scorer to score a large number of documents (optional feature and shipped separately from the main product)

H2O Driverless AI

Hybrid Cloud integration & deployment

  • MLOps/Remote Storage integration:
    • Added functionality in the Python/R API and Flow to push an H2O-3 model to Remote Storage/MLOps
  • Added automated testing:
    • Environments for fresh install testing
      • v23.01.0 on AWS
      • v23.01.0 on Azure
      • v23.01.0 on GCP
    • Environments for upgrade testing to v23.01.0
      • v22.10.1 on AWS, GCP, and Azure
      • v22.10.0 on AWS
      • v22.07.2 on AWS
      • v22.04.1 on AWS
  • Modular auto-installer
    • Integrate Document AI to deployment templates
    • Integrate Feature Store to deployment templates
    • Support for Kubernetes v1.23
    • Support for Kubernetes v1.24
    • Ongoing Improvements based on v22.10.0 installation/upgrade feedback from the field

H2O MLOps

  • Added static scoring endpoints
  • Supports downloading deployment logs from the MLOps Wave App and MLOps API
  • Runtime support for H2O Driverless AI v1.10.4.1, v1.10.4.2, & v1.10.4.3
  • Replaces a Kubernetes configmap with a Kubernetes secret
    info

    For a full list of features, improvements, and bug fixes in MLOps, see the MLOps v0.60.1, 0.59.0 & v0.59.1 release notes.

H2O-3

  • Implemented p-value calculation for GLM with regularization
  • Implemented normal (non-monotonic) splines that can support any degrees
  • Verified the minimum number of knots each spline type can support for GAM
    info

    For a full list of features, improvements, and bug fixes in H2O-3, see the H2O-3 v3.38.0.3 & v3.38.0.4 release notes.

Platform UI

  • Access AI Engines from the Hybrid Cloud UI

Enterprise Steam

  • Driverless AI: Added support for single-user multinode clusters and fixed maximum uptime detection
  • Sparkling Water: Added support for the latest version of Sparkling Water
  • Kubernetes: Disabled auto-mounting of service account tokens
  • Kubernetes: Set specific secure default for Pod seccomp profile
  • Python: Removed tokens from Python client logging
  • Helm: Added support for custom CA certificates
    info

    For a full list of features, improvements, and bug fixes in Enterprise Steam, see the Enterprise Steam v1.9.0 release notes.

v22.10.1 | Dec 19, 2022​

Important

End of support date: November 16, 2023

For more information, see About version support.

Component upgrades​

  • H2O MLOps v0.58.0

New features​

Kubernetes 1.23 support

H2O AI Cloud now supports Kubernetes 1.23 for the following deployments:

  • Amazon AWS EKS
  • Google GCP GKE
  • Microsoft Azure AKS

H2O MLOps

  • Added support for linking and deploying H2O Driverless AI unsupervised models

  • H2O Driverless AI flavours for MLflow models are now supported through integration with Azure Databricks

  • H2O eScorer integration to support batch deployment workflows

  • UX optimizations and additional improvements to the H2O MLOps Wave app

  • Fixed various issues with the H2O MLOps admin dashboard

  • Shapley values can now be calculated for H2O Driverless AI Python pipelines and MOJOs

  • Improvements and bug fixes related to Python scoring pipelines runtimes

info

For a full list of features, improvements, and bug fixes in H2O MLOps, see the H2O MLOps v0.58.0 release notes.

Installer

  • Improvements and bug fixes to the HAIC installer
  • Resolved incompatibility with newer versions of the ebs-csi driver on AWS

H2O Driverless AI

  • Added the H2O MLOps v1.10.3.2 runtime

Known limitations​

H2O Driverless AI

Note that the H2O Driverless AI 1.10.4 Python client is not backwards-compatible with H2O Driverless AI v1.10.3.2. Instead, you can use this v1.10.3.2 Python client with DAI v1.10.3.2.

The DAI v1.10.4.1 Python client includes backward-compatibility for v1.10.3.2 and will be available with the upcoming H2O AI Cloud v23.01.0 release.

Release FAQ​

Do I need to upgrade to v22.10.1?

Upgrading is recommended if you are on v22.10.0 and need H2O MLOPs v0.58.0 or Kubernetes 1.23.

I am currently using a version prior to v22.10.0. Can I directly upgrade to this version without going to v22.10.0?

Yes. You can upgrade from v22.07.0 or v22.04.0 to v22.10.1.

When v23.01.0 is released, will I need to first upgrade to v22.10.1, or can I upgrade directly to v23.01.0?

Depending on the version currently installed, the environment may have to be upgraded to an intermediate version before v23.01.0 can be installed. However, v22.10.1 specifically may not be required in the migration path. For more information about H2O AI Cloud upgrade paths, contact your H2O.ai support team.

v22.10.0 | Nov 16, 2022​

Important

End of support date: November 16, 2023

For more information, see About version support.

Component upgrades​

  • AI App Store v0.21.0
  • Driverless AI v1.10.4
  • Hydrogen Torch v1.2.0
  • H2O-3 v3.36.1.5
  • H2O MLOps v0.57.3
  • Enterprise Steam v1.8.14

New features​

AI App Store

  • Developers can now retrieve the app.toml file and app bundles from the server using H2O CLI commands.

  • Support for Python 3.8 runtimes

Driverless AI

  • New GUI-based wizards to configure and start experiments, join datasets, and perform business value analyses.

  • Improvements in data handling and detection.

  • New BinnerTransformer for one-dimensional binning of numeric features.

  • Support for prediction intervals for regression experiments in Java MOJO scoring (for both C++ and Java MOJO runtimes).

  • New feature_store_mojo recipe type to create a MOJO to be used as a feature engineering pipeline in the H2O Feature Store.

  • Users can now run Disparate Impact Analysis on external datasets.

  • Users can now bulk abort multiple experiments in a project.

    info

    For a full list of features, improvements, and bug fixes in Driverless AI, see the Driverless AI v1.10.4 release notes.

Hydrogen Torch

  • New Azure Data Lake data connector

  • Support for several dataset (data) formats for:

    • image object detection experiments
    • image semantic segmentation experiments
    • image instance segmentation experiments

  • Users can now merge imported datasets into one as well as extend datasets with new data.

  • Users can organize experiments into projects and compare experiments in a project.

  • Improved experiment settings and support for several grid search modes.

    info

    For a full list of features, improvements, and bug fixes in Hydrogen Torch, see the Hydrogen Torch v1.2.0 release notes.

H2O-3

H2O MLOps

Enterprise Steam

  • Users are now able to change the ownership of Driverless AI instances.

  • Bug fixes related to updating user roles and session ready time.

    info

    For a full list of features, improvements, and bug fixes in Enterprise Steam, see the Enterprise Steam v1.18.4 release notes.

Installer

  • Restructured installer and modules

  • CI/CD pipeline for deployment package automation

  • Upgrade guide for existing customers

  • OpenShift support for Hybrid Cloud and updated Azure platform support

Feedback