When working with Databricks you will sometimes have to access the Databricks File System (DBFS).
Accessing files on DBFS is done with standard filesystem commands, however the syntax varies depending on the language or tool used.
For example, take the following DBFS path:
Under Spark, you should specify the full path inside the Spark read command.
When you are using DBUtils, the full DBFS path should be used, just like it is in Spark commands. The language specific formatting around the DBFS path differs depending on the language used.
%fs ls dbfs:/mnt/test_folder/test_folder1/
dbfs: is not required when using DBUtils or Spark commands. The path
dbfs:/mnt/test_folder/test_folder1/ is equivalent to
Shell commands do not recognize the DFBS path. Instead, DBFS and the files within, are accessed with the same syntax as any other folder on the file system.