Start Here
Search & Browse the Databricks Knowledge Base.Help Topics
Whether it's a very specific question, or a vague one, we have it covered in our knowledge base.
Popular Articles
Unexpected cluster termination
Sometimes a cluster is terminated unexpectedly, not as a result of a manual termi...
Apache Spark job fails with Parquet column cannot be converted error
Problem You are reading data in Parquet format and writing to a Delta table when ...
Spark job fails with Driver is temporarily unavailable
Problem When running notebooks or jobs on a cluster, they run successfully multip...
Prevent duplicated columns when joining two DataFrames
If you perform a join in Spark and don’t specify your join correctly you’ll end u...
Create a DataFrame from a JSON string or Python dictionary
In this article we are going to review how you can create an Apache Spark DataFra...
Get and set Apache Spark configuration properties in a notebook
In most cases, you set the Spark config (AWS | Azure ) at the cluster level. Howe...
Append to a DataFrame
To append to a DataFrame, use the union method. %scala val firstDF = spark.range(...
How to list and delete files faster in Databricks
Scenario Suppose you need to delete a table that is partitioned by year, month, d...
Generate unique increasing numeric values
This article shows you how to use Apache Spark functions to generate unique incre...
Forbidden error while accessing S3 data
Problem While trying to access S3 data using DBFS mount or directly in Spark APIs...