Set instance_profile_arn as optional with a cluster policy

Use a cluster policy to set the AWS attribute instance_profile_arn as optional.

Written by ravirahul.padmanabhan

Last published at: March 5th, 2023

In this article, we review the steps to create a cluster policy for the AWS attribute instance_profile_arn and define it as optional.

This allows you to start a cluster with a specific AWS instance profile. You can also start a cluster without an instance profile.

Delete

Note

You must be an admin user in order to manage cluster policies.

Create a new cluster policy

  1. Open your Databricks workspace.
  2. Click Compute.
  3. Click Cluster Policies.
  4. Click Create Cluster Policy.
  5. Enter a Name for the policy.
  6. Enter this JSON code in the Definitions field.
    {
      "aws_attributes.instance_profile_arn": {
        "type": "allowlist",
        "values": [
          "arn:aws:iam::123456789012:instance-profile/allow-this-role"
        ],
        "isOptional": true
      }
    }
  7. Click Permissions.
  8. Assign the new policy to users in your workspace.
  9. Click Create.

You can now create a new cluster using the policy.

Edit an existing cluster policy

  1. Open your Databricks workspace.
  2. Click Compute.
  3. Click Cluster Policies.
  4. Click on an existing policy.
  5. Click Edit.
  6. Add this JSON code to the policy in the Definitions field.
    {
      "aws_attributes.instance_profile_arn": {
        "type": "allowlist",
        "values": [
          "arn:aws:iam::123456789012:instance-profile/allow-this-role"
        ],
        "isOptional": true
      }
    }
  7. Click Permissions.
  8. Verify that the policy is assigned to the correct users in your workspace.
  9. Click Update.

You can now create a new cluster using the policy.