Updated March 7th, 2023 by ravirahul.padmanabhan

Users unable to view job results when using remote Git source

Problem You are running a job using notebooks that are stored in a remote Git repository (AWS | Azure | GCP). Databricks users with Can View permissions (who are not a workspace admin or owners of the job) cannot access or view the results of ephemeral jobs submitted via dbutils.notebook.run() from parent notebook. Cause When job visibility control ...

2 min reading time
Updated September 27th, 2022 by ravirahul.padmanabhan

Upload large files using DBFS API 2.0 and PowerShell

Using the Databricks REST API to interact with your clusters programmatically can be a great way to streamline workflows with scripts. The API can be called with various tools, including PowerShell. In this article, we are going to take a look at an example DBFS put command using curl and then show you how to execute that same command using PowerShe...

2 min reading time
Updated March 4th, 2022 by ravirahul.padmanabhan

Set instance_profile_arn as optional with a cluster policy

In this article, we review the steps to create a cluster policy for the AWS attribute instance_profile_arn and define it as optional. This allows you to start a cluster with a specific AWS instance profile. You can also start a cluster without an instance profile. Note You must be an admin user in order to manage cluster policies. Create a new clust...

0 min reading time
Load More