Forbidden error while accessing S3 data

Problem While trying to access S3 data using DBFS mount or directly in Spark APIs, the command fails with an exception similar to the following: com.amazonaws.services.s3.model.AmazonS3Exception: Forbidden; Request ID: XXXXXXXXXXXXX, Extended Request ID: XXXXXXXXXXXXXXXXXXX, Cloud Provider: AWS, Instance ID: XXXXXXXXXX (Service: Amazon S3; Status Co...

Table creation fails with security exception

Problem You attempt to create a table using a cluster that has Table ACLs enabled, but the following error occurs: Error in SQL statement: SecurityException: User does not have permission SELECT on any file. Cause This error occurs on a Table ACL-enabled cluster if you are not an administrator and you do not have sufficient privileges to create a ta...

Set an unlimited lifetime for service principal access token

Info This article applies to clusters using Databricks Runtime 11.2 and above on AWS. You are using Databricks service principals to delegate permissions to automated tools and systems. In order to secure the service principals, Databricks recommends creating an access token for the service principal. Please review the create a Databricks access tok...