Updated February 3rd, 2023 by emad.rizkallah
Slowness when fetching results in Databricks SQL
Problem Databricks SQL uses cloud fetch to increase query performance. This is done by default. Instead of using single threaded queries, cloud fetch retrieves data in parallel from cloud storage buckets (such as AWS S3 and Azure Data Lake Storage). Compared to a standard, single threaded fetch, you can see up to a 10X increase in performance using ...
0 min reading timeUpdated March 15th, 2023 by emad.rizkallah
ZORDER results in "Hilbert indexing can only be used on 9 or fewer columns" error
Problem You are running an OPTIMIZE ZORDER BY (AWS | Azure | GCP) command in Databricks SQL (AWS | Azure | GCP) when you get an Apache Spark exception error: Hilbert indexing can only be used on 9 or fewer columns. Error in SQL statement: ExecutionException: org.apache.spark.SparkException: Hilbert indexing can only be used on 9 or fewer columns Cau...
0 min reading timeLoad More