Problem

In the compute configuration UI, runtimes for the ML and Standard categories are displayed by default, but you want to restrict users to only configurations from a predefined category (for example, just Standard). 

Cause

In the user interface, multiple categories are displayed by default when you create a cluster, each with several choices. You want to avoid accidentally selecting configurations that don't align with an organization's requirements.

Solution

To ensure only valid runtime selections are available to select, use the appropriate regex pattern based on the runtime category or configuration type.

For Standard Runtimes

To limit the selection to standard runtime versions, define the following regex pattern in the compute policy.

{
  "spark_version": {
    "type": "regex",
    "pattern": "(([1-9])|([0-9][0-9])).[0-9].x-scala[0-9].[0-9][0-9]"
  }
}

For ML Runtimes

To limit the selection to ML runtime versions, define the following regex pattern in the compute policy.

{
  "spark_version": {
    "type": "regex",
    "pattern": "(([1-9])|([0-9][0-9])).[0-9].x-cpu-ml-scala[0-9].[0-9][0-9]"
  }
}

For more details on defining compute policies, refer to the Create and manage compute policies (AWS | Azure | GCP) documentation.