Problem
After you install and configure RStudio in the Databricks environment, when you launch RStudio and click the Knit button to knit a Markdown file that contains code to initialize a sparklyr context, rendering fails with the following error:
failed to start sparklyr backend:object 'DATABRICKS_GUID' not found Calls: <Anonymous>… tryCatch -> tryCatchList-> tryCatchOne -> <Anonymous> Execution halted
Cause
If you try to initialize a sparklyr context in a Markdown notebook with code similar to the following, the Markdown page fails to render because the knitr process spawns a new namespace that is missing the 'DATABRICKS_GUID' global variable.
%r case library(sparklyr) sc <- spark_connect(method = "databricks")
Solution
Instead of rendering the Markdown page by clicking the Knit button in the R Markdown console, use the following script:
%r rmarkdown::render("your_doc.Rmd")
Render the Markdown file using the R console, and then you can access the file in the RStudio Files tab.