Skip to main content

Executor Pool Settings

An Executor Pool is the main entity that handles the execution of tasks or runnables within a Workflow. You can configure the following parameters of an Executor Pool in H2O Orchestrator.

Max Queue Size per Executor​

Defines the maximum number of tasks or Runnables that can be queued before a new executor is created.

Default behavior: If this limit is not set, only one executor will be created to handle all tasks in the pool.

Max Executors​

Specifies the maximum number of executors that can be created in this pool.

Default behavior: If not set, the pool can create an unlimited number of executors.

Queue Size​

Represents the number of tasks or Runnables that are currently waiting in the queue for a new executor to be created.

amountOfGpu​

Defines the number of GPUs that each executor will be allocated.

Use case: Essential for workloads requiring GPU resources, like machine learning or scientific computation tasks.

memoryBytesLimit​

Sets the maximum memory limit (in bytes) for each executor.

Purpose: Ensures that executors do not exceed a certain memory usage, helping in effective resource management.

milliCpuLimit​

Defines the maximum CPU limit (in millicores) that each executor can consume.

Purpose: Prevents executors from using excessive CPU resources.

maxIdleDuration​

Specifies the maximum idle time allowed for an executor. Once this duration is reached, the executor will be terminated.

Purpose: Efficient resource management by freeing up executors when they are no longer needed.

Kernel Image​

Defines which kernel image should be used for executing code.

Use case: Critical for environments where specific kernels are required for different tasks (e.g., Python, R).

memoryBytesRequest​

The minimum amount of memory (in bytes) requested by each executor at the time of creation.

Purpose: Ensures that the system reserves enough memory for the executor to operate efficiently.

milliCpuRequest​

Defines the minimum CPU resources (in millicores) requested by each executor.

Purpose: Guarantees that executors have enough CPU resources for their tasks.

language​

Specifies the programming language supported by the executors.

Purpose: Determines the runtime environment for executing code. Executors may support languages like Python, R.

Feedback