CPU-only deployment
Enterprise h2oGPTe can run without GPUs. In a CPU-only deployment, the models service runs OCR, embeddings, guardrails and document layout models on CPU, and every LLM is an external endpoint that you configure. Ingestion and retrieval are slower than on a GPU deployment.
CPU-only deployment is supported from version 1.7.7.
Enable the CPU-only profile​
The Helm chart ships a values file, values-cpu-only.yaml, next to values.yaml. Add it after your own values file:
helm upgrade --install h2ogpte ./helm/h2ogpte-chart -f my-values.yaml -f values-cpu-only.yaml
The profile:
- removes the
nvidia.com/gpurequest and limit from the models pod - runs the models service and its embedding model on CPU (
coreEncoderDevice: cpu,H2OGPTE_CORE_MODELS_MUST_USE_GPU=false,CUDA_VISIBLE_DEVICES=-1) - does not start the GPU-only model queues (see below) through
H2OGPTE_CORE_MODELS_QUEUE_FILTER - makes Tesseract the default OCR engine (
H2OGPTE_PARSE_OCR_MODEL=tesseracton crawl, core and chat)
Helm replaces lists instead of merging them. If your own values already set extraEnv for models, crawl, core or chat, copy the profile's entries into your lists instead of layering the file, or your entries are dropped.
Keep h2ogpt.enabled: false and configure external LLM endpoints. Hosting LLMs inside the cluster needs GPUs.
Do not combine the profile with the modelsCpu or modelsHeavy tier deployments: the profile sets the models queue filter directly.
What is not available on CPU​
| Feature | Behavior in a CPU-only deployment |
|---|---|
PaddleOCR (paddleocr-* OCR models: Chinese, Japanese, Korean, Arabic, Cyrillic, Devanagari, Thai and other non-Latin scripts) | Not available. OCR with an explicit paddleocr-* model fails quickly with an error that names the cause. |
Automatic OCR model selection (auto) | Language detection runs on PaddleOCR, so it is skipped and docTR is used for every page. |
Handwriting OCR (mississippi-800m) and handwriting detection | Not available. |
| Audio and video ingestion | Not available. |
| VLM-based language identification | Not available. |
| LLMs hosted in the cluster | Not available. Use external LLM endpoints. |
A request for a model the models service does not run fails within seconds with an error such as Model get_paddleocr_preds is not served by this models pod. The models pod /health endpoint reports "status": "degraded" and lists dead_queues when the workers for a queue could not start.
Performance​
OCR, embeddings, layout detection and guardrails all run on CPU, so expect ingestion to be several times slower than on a GPU deployment. Throughput depends on the CPU and memory you give the models pod; size models.resources for your document volume and test with representative documents.
To take embedding work off the models pod, configure a remote embedding model.
- Submit and view feedback for this page
- Send feedback about Enterprise h2oGPTe to cloud-feedback@h2o.ai