Rendering an R markdown file containing sparklyr code fails

Learn how to resolve failures when rendering an R markdown file containing sparklyr.

Written by Adam Pavlacka

Last published at: May 20th, 2022


After you install and configure RStudio in the Databricks environment, when you launch RStudio and click the Knit button to knit a Markdown file that contains code to initialize a sparklyr context, rendering fails with the following error:

failed to start sparklyr backend:object 'DATABRICKS_GUID' not found Calls: <Anonymous>… tryCatch -> tryCatchList-> tryCatchOne -> <Anonymous> Execution halted


If you try to initialize a sparklyr context in a Markdown notebook with code similar to the following, the Markdown page fails to render because the knitr process spawns a new namespace that is missing the 'DATABRICKS_GUID' global variable.


case library(sparklyr)
sc <- spark_connect(method = "databricks")


Instead of rendering the Markdown page by clicking the Knit button in the R Markdown console, use the following script:



Render the Markdown file using the R console, and then you can access the file in the RStudio Files tab.

RStudio image with files tab highlighted.

Was this article helpful?