Append to a DataFrame

Learn how to append to a DataFrame in Databricks....

How to improve performance with bucketing

Learn how to improve Databricks performance by using bucketing....

How to handle blob data contained in an XML file

Learn how to handle blob data contained in an XML file....

Simplify chained transformations

Learn how to simplify chained transformations on your DataFrame in Databricks....

How to dump tables in CSV, JSON, XML, text, or HTML format

Learn how to output tables from Databricks in CSV, JSON, XML, text, or HTML format....

Get and set Apache Spark configuration properties in a notebook

...

Hive UDFs

Learn how to create and use a Hive UDF for Databricks....

Prevent duplicated columns when joining two DataFrames

Learn how to prevent duplicated columns when joining two DataFrames in Databricks....

Revoke all user privileges

Use a regex and a series of for loops to revoke all privileges for a single user....

How to list and delete files faster in Databricks

Learn how to list and delete files faster in Databricks....

How to handle corrupted Parquet files with different schema

Learn how to read Parquet files with a specific schema using Databricks....

No USAGE permission on database

User does not have USAGE permission on the database....

Nulls and empty strings in a partitioned column save as nulls

Learn why nulls and empty strings in a partitioned column save as nulls in Databricks....

Behavior of the randomSplit method

Learn about inconsistent behaviors when using the randomSplit method in Databricks....

Job fails when using Spark-Avro to write decimal values to AWS Redshift

Learn how to resolve job failures when writing decimal values to AWS Redshift with Spark-Avro....

Generate schema from case class

Learn how to generate a schema from a Scala case class....

How to specify skew hints in dataset and DataFrame-based join commands

Learn how to specify skew hints in Dataset and DataFrame-based join commands in Databricks....

How to update nested columns

Learn how to update nested columns in Databricks....

Incompatible schema in some files

Learn how to resolve incompatible schema in Parquet files with Databricks....

Access denied when writing to an S3 bucket using RDD

Learn how to resolve an access denied error when writing to an S3 bucket using RDD....

Invalid timestamp when loading data into Amazon Redshift

Learn how to resolve an invalid timestamp error when loading data into AWS Redshift....

Unable to infer schema for ORC error

Apache Spark returns an error for ORC files if no schema is defined when reading from an empty directory or a base path with multiple subfolders....

Access files written by Apache Spark on ADLS Gen1

Configure permissions to allow access to files that Apache Spark writes to ADLS Gen1 storage....

Object ownership is getting changed on dropping and recreating tables

Use TRUNCATE or REPLACE for tables and ALTER VIEW for views instead of dropping and recreating them....

User does not have permission SELECT on ANY File

Regular users cannot create tables without permission when access control is enabled....