Slowness when fetching results in Databricks SQL

Problem Databricks SQL uses cloud fetch to increase query performance. This is done by default. Instead of using single threaded queries, cloud fetch retrieves data in parallel from cloud storage buckets (such as AWS S3 and Azure Data Lake Storage). Compared to a standard, single threaded fetch, you can see up to a 10X increase in performance using ...

0 min reading time

ZORDER results in "Hilbert indexing can only be used on 9 or fewer columns" error

Problem You are running an OPTIMIZE ZORDER BY (AWS | Azure | GCP) command in Databricks SQL (AWS | Azure | GCP) when you get an Apache Spark exception error: Hilbert indexing can only be used on 9 or fewer columns. Error in SQL statement: ExecutionException: org.apache.spark.SparkException: Hilbert indexing can only be used on 9 or fewer columns Cau...

0 min reading time