In most cases, you set the Spark configuration at the cluster level. However, there may be instances when you need to check (or set) the values of specific Spark configuration properties in a notebook.
This article shows you how to display the current value of a Spark configuration property in a notebook. It also shows you how to set a new value for a Spark configuration property in a notebook.
To get the current value of a Spark configuration property, evaluate the property without including a value.
To set the value of a Spark configuration property, evaluate the property and assign a value.
You can only set Spark configuration properties that start with the
library(SparkR) sparkR.session() sparkR.session(sparkConfig = list(spark.sql.<name-of-property> = "<value>"))