Problem You are trying to create a token on behalf of a service principal , using /2.0/token-management/on-behalf-of/tokens in the REST API but are getting a PERMISSION_DENIED error. { "error_code": "PERMISSION_DENIED", "message": "User xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx does not have permission to use tokens." } Cause This happens when the servic...

The majority of Databricks customers use production Databricks Runtime releases (AWS | Azure | GCP) for their clusters. However, there may be certain times when you are asked to run a custom Databricks Runtime after raising a support ticket. Warning Custom Databricks Runtime images are created for specific, short-term fixes and edge cases. If a cust...

Problem You are trying to install the PyStan PyPi package on a Databricks Runtime 6.4 Extended Support cluster and get a ManagedLibraryInstallFailed error message. java.lang.RuntimeException: ManagedLibraryInstallFailed: org.apache.spark.SparkException: Process List(/databricks/python/bin/pip, install, pystan, --disable-pip-version-check) exited wit...

Problem You have imported Python libraries, but when you try to execute Python code in a notebook you get a repeating message as output. Example 1: INFO:py4j.java_gateway:Received command c on object id p0INFO:py4j.java_gateway:Received command c on object id p0INFO:py4j.java_gateway:Received command c on object id p0INFO:py4j.java_gateway:Received ...

In this article we show you how to display detailed timestamps, including the date and time when a file was created or modified. Use ls command The simplest way to display file timestamps is to use the ls -lt <path> command in a bash shell. For example, this sample command displays basic timestamps for files and directories in the /dbfs/ folde...

Problem You are using the Databricks Connect client with Databricks Runtime 6.4 and receive an error message which states that the client does not support the cluster. Caused by: java.lang.IllegalArgumentException: The cluster is running server version `dbr-6.4` but this client only supports Set(dbr-5.5). You can find a list of client releases at ht...

Problem You are trying to create a Parquet table using TIMESTAMP, but you get an error message. Error in SQL statement: QueryExecutionException: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.UnsupportedOperationException: Parquet does not support timestamp. See HIVE-6384 Example code %sql CREATE EXTERN...

Problem Your job fails with a Java IndexOutOfBoundsException error message: java.lang.IndexOutOfBoundsException: index: 0, length: <number> (expected: range(0, 0)) When you review the stack trace you see something similar to this: Py4JJavaError: An error occurred while calling o617.count. : org.apache.spark.SparkException: Job aborted due to s...

Problem You are using a cluster running Databricks Runtime 7.3 LTS and above. You have enabled table access control for your workspace (AWS | Azure | GCP) as the admin user, and granted the SELECT privilege to a standard user-group that needs to access the tables. A user tries to access an object in the database and gets a SecurityException error me...

Problem Notebook or job execution stops and returns either of the following errors: Run result unavailable: job failed with error message Context ExecutionContextId(1731742567765160237) is disconnected. Can’t attach this notebook because the cluster has reached the attached notebook limit. Detach a notebook and retry. Cause When you attach a noteboo...