Get and set Apache Spark configuration properties in a notebook
In most cases, you set the Spark configuration at the cluster level. However, there may be instances when you need to check (or set) the values of specific Spark configuration properties in a notebook.
This article shows you how to display the current value of a Spark configuration property in a notebook. It also shows you how to set a new value for a Spark configuration property in a notebook.
Get Spark configuration properties
To get the current value of a Spark configuration property, evaluate the property without including a value.
Set Spark configuration properties
To set the value of a Spark configuration property, evaluate the property and assign a value.
Note
You can only set Spark configuration properties that start with the spark.sql
prefix.